From 5 Minutes to 1! Dramatically Reducing yarn install Time

This is a record of investigating why yarn install (frontend dependency installation) was taking so long and how I resolved it.

(In this case, I'm using yarn v1, but you can perform a similar investigation in npm by using npm ls instead of yarn why.)

Background

In a certain product, yarn install (frontend dependency installation) suddenly started taking about 5 minutes.

At first glance, it might seem like something you don't need to worry about since it only affects the initial installation in the local environment. However, it is actually a significant problem when you think about it.

• Slow local environment setup increases onboarding effort.
• Development efficiency decreases because yarn install is required in each local environment when updating dependencies.
• Review speed decreases because CI results are not available immediately.
• Speed from merge to release decreases because CD takes more time.
• CI/CD costs increase.

Therefore, I decided to spend some time investigating and fixing the issue.

Identifying the libraries that take a long time to install

To reinstall all libraries, I delete the entire node_modules directory.

rm -rf node_modules

Then, I install them again.

yarn install --immutable

During the installation, I keep an eye on the terminal.

I pay close attention when the status becomes Building fresh packages.... In this product, it was stuck on the following screen for about 2 minutes.

[4/4] Building fresh packages...
[-/11] waiting...
[2/11] fsevents
[-/11] waiting...
[-/11] waiting...
[8/11] node-sass

After that, it moved to the next screen and took another 2 minutes.

[4/4] Building fresh packages...
[-/11] waiting...
[-/11] waiting...
[-/11] waiting...
[-/11] waiting...
[10/11] grpc

From the above results, it's clear that building node-sass and grpc takes a total of 4 minutes.

Initially, I considered saving and analyzing the logs in verbose mode, but identifying the time-consuming libraries by sight was sufficient.

For node-sass, node-sass@4.13.1 was specified in the dependencies.

However, I hadn't specified grpc. So, I checked which library it was a dependency of.

yarn why grpc

yarn why v1.22.19
[1/4] Why do we have the module "grpc"...?
[2/4] Initialising dependency graph...
[3/4] Finding dependency...
[4/4] Calculating file sizes...
=> Found "grpc@1.24.2"
info Reasons this module exists
   - "firebase#@firebase#firestore" depends on it
   - Hoisted from "firebase#@firebase#firestore#grpc"
Done in 0.44s.

I found that firebase depends on it. Indeed, firebase@7.8.1 is installed.

Checking the README of the relevant libraries

Now that we know the causes are node-sass@4.13.1 and firebase@7.8.1, let's check the README for the specific versions of these libraries. There might be some hints.

node-sass

First, let's look into node-sass.

Looking at the README at the time, it says:

Supported Node.js versions vary by release, please consult the releases page.

Since it mentions checking the releases page, I looked at the releases page and found the cause. We had just recently updated Node to version 14.

Supported Environments
Node 0.10, 0.12, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13

Install runs only two Mocha tests to see if your machine can use the pre-built LibSass which will save some time during install. If any tests fail it will build from source.

Normally, it just copies a pre-built binary, but since we were using an unsupported Node version, I realized it was building from source every time.

To minimize the impact, I decided to update to version 4.14.1, which is the closest version supporting Node 14. Since there were no changes that would affect the product, I regretted not moving sooner.

https://github.com/sass/node-sass/compare/v4.13.1...v4.14.1

firebase

Next, let's check the release notes for firebase, which depends on grpc.

I found the following release note for version 7.14.0:

Replaced grpc with @grpc/grpc-js in the Node.js builds. As a result, the minimum supported NodeJS version is now 8.13.0.

It seems that for versions below 7.14.0, the grpc source code was built every time, but from 7.14.0 onwards, it started depending on the pre-built @grpc/grpc-js.

I decided to update this to 7.14.0 as well.

Summary

I was able to reduce the yarn install time from 5 minutes to 1 minute 🎉!

The benefit was particularly significant due to the faster CI speed. I was able to greatly improve development efficiency.

I plan to continue making improvements, such as migrating from node-sass to dart-sass.