Exploring how Next.js runs with bun dev

https://github.com/oven-sh/bun

Bun's runtime includes a mode for launching a Next.js server. I was curious about what this actually does, so I observed the behavior and examined the source code.

• bun dev serves as the command to start the HTTP server.
• This is likely because Bun was originally created with the goal of speeding up dev-servers.
• Whether to use Next is specified in the bunfig.toml configuration file, but it can also be explicitly stated with bun --use next.

bunfig.toml

framework = "next"

• Running bun bun --use next generates two binaries: ./node_modules.bun and ./node_modules.server.bun.
• bun dev executes these .bun files. They can also be specified via --bunfile and --server-bunfile.
• If bun dev is run without options and not specified in the configuration file, routes will not be resolved, resulting in a 404 error.
• Therefore, let's look at what processing is added when this flag is enabled.

dev_command.zig

const Server = @import("../http.zig").Server;
pub const DevCommand = struct {
    pub fn exec(ctx: Command.Context) !void {
        Global.configureAllocator(.{ .long_running = true });
        try Server.start(ctx.allocator, ctx.args, @TypeOf(ctx.debug), ctx.debug);
    }
};

• Global.configureAllocator seems a bit like magic, and its exact function is unclear.
• global.zig and __global.zig are defined, suggesting the implementation might be swapped somewhere.
• The Server implementation is in http.zig, so let's look at that.

http.zig

• Requests are processed using the Server struct and a Handler function in the HTTP server.

Entry point: Server.start()
https://github.com/oven-sh/bun/blob/c3bf97002d6f0b117060dabf2aa281eedd85b7fd/src/http.zig#L3941

pub fn start(allocator: std.mem.Allocator, options: Api.TransformOptions, comptime DebugType: type, debug: DebugType) !void

• The second argument options includes the next flag.
  • The process of how it is transformed into Api.TransformOptions is unclear.
• Allocating the Server.
• Attaching the Bundle to the server.
• Calling configureLinker() and configureRouter() of the Bundler.
• Since options is used here, this is the part where it integrates with the application.

server.bundler = try allocator.create(Bundler);
server.bundler.* = try Bundler.init(allocator, &server.log, options, null, null);
server.bundler.configureLinker();
try server.bundler.configureRouter(true);

bundler.zig

• configureLinker() is a function that initializes the Linker field.
• configureRouter() is a function that defines the behavior of the Linker field based on the information in options.routes.
• When Bundler.init() is executed, Options.fromApi() is called, and the framework configuration information is included in its arguments.

bun-framework-next

• An npm package included in the repository.
• Since it is added to the devDependencies of applications created with Bun, this module acts as an intermediary for the Server.
• It doesn't do anything particularly special; it serves as glue code to handle Next.js application requests using Bun.serve.
• As an experiment, I tried deleting this module after generating the binary with bun bun, and the routes stopped working, confirming that it is loaded at runtime.

node_modules.bun, node_modules.server.bun

❯ file node_modules.server.bun  
node_modules.server.bun: sticky a /usr/bin/env bun script executable (binary data)

I didn't quite understand what these binaries were at first.

Since they were executables, I initially thought the bundled result had been converted into native code, but looking inside, that wasn't the case.

#!/usr/bin/env bun

o<cd>
^@
var BUN_RUNTIME=(()=>{var fr=Object.create;var G=Object.defineProperty;var cr=Object.getOwnPropertyDescriptor;var sr=Object.getOwnPropertyNames;var ...

By running bun node_modules.bun, you can extract the binary as the JavaScript code shown above.

This file contains references to paths under node_modules/, which explains why it stops working if you delete that directory.

Hot Module Replacement

Regarding how Bun implements the HMR mechanism of next-dev-server, the following code, which runs in the browser, was executed at bundle time.

bundler/generate_node_modules_bundle.zig

fixed_buffer_writer.print(
    \\if ('window' in globalThis) {{
    \\  (function() {{
    \\    BUN_RUNTIME.__injectFastRefresh(${x}());
    \\  }})();
    \\}}

The entity of __injectFastRefresh() is a function implemented in TS using react-refresh (it is added to the application's devDependencies).

runtime/hmr.ts

  function injectFastRefresh(RefreshRuntime) {
    if (!FastRefreshLoader.hasInjectedFastRefresh) {
      RefreshRuntime.injectIntoGlobalHook(globalThis);
      FastRefreshLoader.hasInjectedFastRefresh = true;
    }
  }

What I Discovered

• An HTTP server written in Zig is built-in, and JS dependency modules are bundled into a proprietary binary and loaded at startup. This is likely faster than a server started with Node, which processes JS for each request (Hypothesis).
• Processes that would typically be handled in the core of Next.js, such as bundling, refresh, and request handling, were also included in Bun.
  • This suggests that bun dev is intended to serve as a high-speed interface for next dev.
• However, since the JS under node_modules/ is still necessary for bun dev to function, not everything in the application is converted to a binary.
  • This is because bun-framework-next itself runs in the JSC context and references the Next.js rendering code from within.
  • It seems the proprietary binary and node_modules/ work together.

1. bun-framework-next running on the Bun HTTP server returns responses via the Next.js interface.
2. The rendering utilizes the code from the Next.js core npm package.
3. Bun's proprietary binary bundle and node_modules/ are referenced at runtime.
4. For client-side Hot Reloading, Bun injects the react-refresh JS code into the bundle.