What Is the Native ESM Era?

Recently, the term "Native ESM era" has been circulating in the Japanese frontend community. The main topic of discussion is what build tools will look like in this Native ESM era.

Personally, I find it an interesting concept and hope it catches on, so I've prepared some basic material to explain what the Native ESM era is.

What "Native ESM era" refers to may vary depending on the person. This article explains the author's perspective. Thank you for your understanding.

What is Native ESM in the first place?

Native ESM refers to ES Modules. In other words, it refers to the module system defined as part of the ECMAScript specification. ES Modules is what is widely used in modern frontend development today—importing with import declarations and exporting with export declarations.

Specifically, ES Modules is a module system that is directly understood by browsers. The word "Native" emphasizes this fact.

From here on, this article will use the term "frontend development" to refer to modern frontend development that uses ES Modules for development.

What is the Native ESM Era?

The Native ESM era is an era where ES Modules are used even after the build process.

This is because, in current frontend development, source code is not distributed as-is; it requires a build. Browsers load the artifacts obtained through the build. Tools such as Babel, esbuild, and Webpack represent the tools responsible for this build step.

In the currently dominant method, multiple JavaScript files connected by import and export are combined into a single JavaScript file by a tool called a bundler during the build process^[1]. This is the artifact, and the browser loads that single consolidated JavaScript file. The fact that there is only one file means there is no longer a need to load other files. In other words, even if you were using ES Modules before the build, ES Modules are no longer used in the post-build JavaScript^[2].

The idea of the Native ESM era is to break this situation and ensure that the use of ES Modules is included in the post-build artifacts (in other words, the post-build artifacts consist of multiple files (modules)).

I believe there are two stages to the Native ESM era. Namely, a stage where ES Modules are used only in the development build (the build when running tools in watch mode locally) and things remain as before in the production build (the build to create artifacts for deployment), and a stage where ES Modules are used in both the development and production builds. Of course, the former will arrive first. Furthermore, regarding the former, it can be considered to have already arrived due to the development of related tools. The following article is a good reference.

https://zenn.dev/mizchi/articles/native-esm-age

What is Solved in the Native ESM Era?

Next, I will explain why it is necessary to migrate to the Native ESM era, or in other words, what problems exist in the traditional approach and how they are solved in the Native ESM era.

Problems are broadly divided into two categories: build performance issues and runtime performance issues. As you can guess from the terms, the former relates to both development and production builds, while the latter relates only to production builds. The state where only the former is solved is the arrival of the Native ESM era in the development environment, and when the latter is also solved, it becomes the full Native ESM era. In this article, let's call the former the Development Native ESM era and the latter the Full Native ESM era (these are terms coined for this article).

Development Native ESM Era: Build Performance Issues

First, let's look at build performance issues. What is particularly relevant here is the part of the build handled by the bundler (a tool that analyzes import and export between multiple JavaScript files and combines them into a single JavaScript file^[3]).

To put it simply, the process of bundling itself is slow. In the bundling process (to put it very roughly), multiple JavaScript codes are combined into one large piece of code, which involves transforming it slightly to make it work correctly or adding runtime code to maintain consistency. Generating a large piece of code that contains the entire content of the application inevitably becomes slower as the application grows.

Furthermore, especially in development environments, the mainstream practice is to re-bundle immediately every time a file is changed and saved. During development, you open the artifact of the current source build in a browser and develop while checking the behavior. At this time, by automatically reloading the browser when a file changes, the impact of the change (the result of the new build) is immediately reflected in the browser, helping with high-speed app development. Additionally, a feature called Hot Module Replacement (HMR) can further increase efficiency by avoiding a full reload.

Bundling frequently means that slow bundling directly leads to decreased development efficiency. This is the build performance issue. Bundlers also make efforts to reduce unnecessary work as much as possible during re-bundling, but the current situation is that it is not uncommon to feel stress due to the time bundling takes. Other problems also arise, such as the need to cache a large amount of data in memory in preparation for re-bundling.

The solution in the (Development) Native ESM era is not to bundle in the first place. In other words, if there are import or export statements in the source file, they are loaded directly by the browser as they are. However, transformations such as slightly rewriting the import destination for Hot Module Replacement support are performed by the server. With a clever mechanism, this transformation can be done on a per-module basis. By making it possible to perform transformations per module, the bottleneck process of bundling can be skipped, achieving improved parallelism and a reduction in necessary transformation processing. Furthermore, when performing HMR (an optimization that replaces only the modified modules rather than a full reload), there is no need to perform tedious processing such as creating partial bundles, which is efficient.

By the way, as a side note, to enable HMR, you need to define HMR boundaries (explaining with Vite as an example; terms may differ in other tools). An HMR boundary is a declaration that says, "When this module (and its dependent modules) is rewritten, only this module needs to be reloaded, and files depending on this module do not need to be reloaded." If there is an HMR boundary, when a module inside it is updated, only the inside of the boundary is reloaded while the outside remains as is. HMR boundaries are important for improving development efficiency.

However, even if only a part of the project is reloaded, ES Modules do not have a language feature to later replace the contents of something already imported. In other words, when defining an HMR boundary, you need to manually "gracefully replace what is visible from the modules using it when reloaded," which requires significant effort (brute force). In the case of Vite, for example, for React projects, @vite/plugin-react-refresh provides that brute force. As a result, modules that export only React components become HMR boundaries, and the performance of builds and reloads is optimized.

In this way, in the Development Native ESM era, the module graph^[4] formed by the project's source files is delivered to the browser in its original form. By having the browser resolve the module graph (tracing import statements and loading all necessary files for the program), the bundling process of combining multiple files into one can be omitted, leading to faster builds during development.

By the way, among the Web standards necessary for the arrival of the Development Native ESM era, the newer ones are dynamic import and import.meta, both of which were formally adopted in ES2020, but browser support was already in place by 2017–2018. Therefore, the Development Native ESM era became possible not so much through the evolution of Web standards, but rather through the research and development of development software.

Full Native ESM Era: Runtime Performance Issues

Combining a project that uses ES Modules into a single non-ES Modules file using a bundler also has runtime performance issues. Current bundler artifacts include a runtime to emulate the imports that were in the original source files. In other words, the module graph information is hidden within that runtime, and the browser cannot directly recognize the module graph. This likely results in inferior performance compared to an ideal state.

In fact, when I measured the performance of a project I am involved in, about 15% of the time spent on initial rendering was consumed by this runtime. By adopting Native ESM and having the browser resolve import directly instead of emulating it via runtime, there is room to reduce this overhead to some extent.

For this reason, there is motivation to aim for the use of Native ESM even in production builds. However, does this mean we can use the same approach as in the Development Native ESM era? The answer is no.

In the Development Native ESM era, during development, the code containing import and export—that is, the project's module graph—was delivered to the browser in its original form. Actually, this is only permissible because it is a development environment. This is because, in a development environment, the server delivering the build artifacts (the development server) is basically on your own PC. Therefore, the communication time between the browser and the server can essentially be ignored. In fact, it is because of this situation that we were able to deliver the project's module graph as-is.

Delivering the project's module graph as-is even in production builds still has a significant performance issue. That is, it takes an incredibly long time to load. Due to the specifications of ES Modules, program execution cannot begin until the entire module graph has been loaded. Furthermore, which module a certain module imports cannot be determined without first loading the importing module. In other words, multiple round trips between the browser and the server are required to fully load the module graph. In a production environment where round trips to the server take time, making unnecessary round trips becomes a major problem.

In website performance optimization, the basic principle is to complete everything in two round trips. Specifically, the HTML document is returned in the first round trip, and the JavaScript files whose URLs are listed in it are retrieved in the second round trip^[5].

If we were to deliver the project's module graph as-is, we would need to send the entire necessary module graph all at once during the second round trip^[6], but it is known that this still has problems. This is because trying to have the browser pre-load the original source code's module graph as-is means having it load hundreds or thousands of files at once. This is not good for performance^[7].

Therefore, in production environments where performance is required, it is essential to merge multiple files into one (or a small number of files). Actually, this is exactly what bundlers have traditionally done. In my opinion, as long as the premise of "having to reduce the number of files" exists, the work of the bundler will not disappear. However, a Full Native ESM era is possible. Currently, relevant specifications are being developed in preparation for a future Full Native ESM era.

And the following two are noteworthy as things moving in that direction:

• Module Fragments: An ECMAScript proposal to enable "modules within modules." Module Fragments allow for things like "importing from another module within the same file." This means the original module graph can be embedded as multiple modules within a single file, achieving both "being a single file" and "containing many aggregated modules."
• Resource Bundles (likely to be renamed to Bundle Preloading?): A WICG proposal for delivering not just JavaScript files but also other types of resources like images bundled together in a single file (Web Bundle). Furthermore, browsers can individually cache the specific resources contained within the Web Bundle.

If these kinds of technologies evolve and become established, it will be possible to achieve both "improving performance by having the browser directly interpret ES Modules" and "preventing the simultaneous loading of many files," making the production use of Native ESM a reality.

Summary

In this article, I explained what the Native ESM era is, dividing it into ES Modules usage in development environments and production environments.

In development environments, I explained that for apps originally developed as many modules, the development experience can be improved by having the browser load the module graph directly and through a bit of effort. This is the era of Native ESM in development environments.

Furthermore, while Native ESM is more ideal even in production environments from a performance perspective, for that to happen—that is, for the Full Native ESM era to arrive—specifications still need to mature.

脚注

1. There are techniques called code splitting where the artifacts become multiple instead of one, but this article omits that explanation for simplicity.

2. It may be used via patterns like the module-nomodule pattern, but that is not very relevant to the topic of this article.

3. Some tools, like esbuild, aim for higher speeds by performing several build steps that go beyond just bundling together. In fact, in an era where speed is required, this may become the mainstream.

4. A graph where source files are nodes and import relationships are edges.

5. When using Server-Side Rendering (SSR), the state that would be obtained by running the JavaScript (the result of the initial rendering) is already included in the HTML of the first round trip. By loading the JavaScript in the second round trip, the content becomes interactive (so-called hydration). Also, putting the CSS necessary for the initial display (so-called critical CSS) into the HTML is a common optimization. In principle, JavaScript could also be inlined into the HTML, but this is likely not done much due to issues such as cache efficiency.

6. With current technology, this is somewhat achievable by using <link rel="modulepreload">.

7. As stated in the Resource Bundles explainer: “Modern Web sites are composed of hundreds or thousands of resources. Fetching them one by one has poor performance”