About Disk Access Strategy

Random disk access は、memory accessに比べて、遅いことは多く知られているかと思うが、Sequential disk accessは、memory accessに比べて早くなることがある

• Random disk access
• Random memory access
• Sequential disk access

以下のコードはそれぞれの速度を計測するコードであり、実行結果は以下の通りになった。

Random disk access time: 4317 ms
Random memory access time: 6 ms
Sequential disk access time: 1 ms

import java.io.RandomAccessFile
import java.nio.file.Files
import kotlin.random.Random
import kotlin.system.measureTimeMillis

fun main() {
    val path = Files.createTempFile("test", "txt").toFile()

    val size = 1024 * 1024 * 5 // 1MB
    val buffer = ByteArray(size) { 0 }
    val randomAccessIndices = IntArray(size / 4) { Random.nextInt(size / 4) * 4 }

    val randomAccessFile = RandomAccessFile(path, "rw")

    // Measure random access to disk
    // In the measurement of random disk access, data of 4 bytes is written to a random location within the file.
    // Before each write operation, the seek method is used to set the position of the file pointer.
    val randomDiskAccessTime = measureTimeMillis {
        for (i in randomAccessIndices) {
            randomAccessFile.seek(i.toLong())
            randomAccessFile.write(buffer, i, 4)
        }
    }
    println("Random disk access time: $randomDiskAccessTime ms")

    // Measure random access to memory
    // For the measurement of random memory access, a value is written to a random location within the buffer.
    val randomMemoryAccessTime = measureTimeMillis {
        for (i in randomAccessIndices) {
            val index = i % buffer.size
            buffer[index] = 123
        }
    }
    println("Random memory access time: $randomMemoryAccessTime ms")

    // Measure sequential access to disk
    // In the measurement of sequential disk access, all data from the buffer is written once, starting from the beginning of the file.
    val sequentialDiskAccessTime = measureTimeMillis {
        randomAccessFile.seek(0)
        randomAccessFile.write(buffer, 0, size)
    }
    println("Sequential disk access time: $sequentialDiskAccessTime ms")

    randomAccessFile.close()
    path.delete()
}

Reference

https://kafka.apache.org/documentation/#design:~:text=The key fact,random memory access