Why Does Node.js Need a Virtual File System?

The Motivation

While browsing Hacker News, I stumbled upon an article titled "Why Node.js needs a virtual file system."

I thought, "Wait, isn't the file system something the OS handles? A VFS for Node.js??" so I decided to look into it.

---

What is VFS (Virtual File System) anyway?

VFS is a mechanism that provides a "virtual file structure" separate from the actual physical disk.

For example, the Linux kernel has a VFS layer that allows both ext4 and NFS to be handled using the same open() and read() interfaces.

The key point is that "you can operate using the same interface regardless of where the physical location is."

---

The Current State of Node.js and Its Issues

Node.js allows you to read and write files normally using the fs module.

import { readFileSync } from 'node:fs';
const content = readFileSync('./config.json', 'utf-8');

This in itself isn't a problem, but it doesn't work in edge environments or within WebAssembly.

What are edge environments?

They refer to ultra-lightweight execution environments running on CDN servers, such as Cloudflare Workers or Vercel Edge Functions.

• Memory and CPU are strictly limited.
• The fs module (= access to the OS file system) cannot be used.
• Yet, there is still a need to "read configuration files" or "use template files."

The same applies to WebAssembly

When running WASM (WebAssembly) modules in a browser or Node.js, even if the code inside WASM wants to "read a file," the browser does not have a file system.

How do Deno and Bun handle this?

Deno

Deno controls file access granularly using permission flags like --allow-read. Furthermore, it has become possible to manage data independently of the file system using APIs like Deno.openKv().

Bun

Bun has an API called Bun.file() implemented in its own way, which abstracts file operations further.

---

What is the benefit of having a VFS in Node.js?

If a VFS existed, you would be able to do things like this:

// Virtual example (does not exist in the standard library currently)
import { createVFS } from 'node:vfs';

const vfs = createVFS();

// Create a virtual file in memory
vfs.writeFile('/virtual/config.json', JSON.stringify({ port: 3000 }));

// Can read with the same interface in any environment
const config = vfs.readFile('/virtual/config.json');

You would be able to run the same code in edge environments, WASM, and test environments.

---

Use cases in testing

The concept of VFS is already utilized in testing.

A library called memfs is exactly this; it allows you to simulate a file system in memory without using an actual disk.

import { fs, vol } from 'memfs';

// Prepare files in memory before testing
vol.fromJSON({
  '/app/config.json': '{"port": 3000}',
  '/app/data.txt': 'hello world',
});

// Testing can be done without actual file I/O!
const config = fs.readFileSync('/app/config.json', 'utf-8');

Tests become faster and independent of the environment.

---

Summary

Issue	With VFS...
Cannot use fs in edge environments	Can be replaced with a virtual FS
File access within WASM	The host side can provide a VFS
File I/O in tests	High-speed testing possible with memfs, etc.
Portability	Same interface regardless of the environment

I took for granted that "Node.js has a file system," but I realized that in the world of edge and WASM, it is not taken for granted at all.

I feel I have also gained an understanding of why Deno and Bun design things this way.🧐

---

This was a note learned from today's Hacker News stroll!