NEWT: From Multi-Repos to Monorepo Development

Hello!

Hello! I'm Rodrigo, a Full-stack developer at Reiwa Travel.

On April 5, 2022 we have released our service NEWT, an overseas travel reservation application.

If you like or plan to traveling abroad, please use it!

This article is part of the advent calendar for the NEWT release, and It will briefly explain our reasons for moving from multi-repos to a monorepo development strategy. It will also share how we organize it and which tools/libraries we are using.

If you want to know more about our Frontend and Backend stack, please refer to these blogs:

• NEWT: Front-end Technology Selection
• Backend Technology Selection for Building a digital travel agency

Context

At the beginning of the project, we started from the design of our application to have a more realistic representation of the WHY → HOW → WHAT of our service. At the same time, we decided which functionalities we would include in our MVP.

Later on, we started the development of our app; we were a small team, and most of the members were side job contractors.

Our teams were divided into 4 groups.

• iOS
• Android
• Backend (API + Admin Panel)
• Web (App Webview)

Multi-repo

Because each team was moving at a different speed and not all of them started at the same time to develop our service, we decided to go with a multi-repo strategy to allow each team to move freely by deciding their own structure, libraries, and setting up their CI/CD without impacting each other.

(*) simple representation

Issues

A multi-repo strategy was a good decision for our MVP and our team's organization, but as the team started to grow, we encountered some problems that started to affect the overall speed of our development.

• Development Speed & Code Separation

  Introducing new functionality or changing an existing API required us to create a PR on the API repository + on the other affected repositories (Admin, Web, iOS, Android).

• Team Coordination

  It was difficult to understand the context or the impact of one change because each team was moving at their own speed, and the code visibility was not good between projects, especially when we wanted to refactor our code or change our API.

• Deployments

  Because each team generated PR on different repositories, it required us to coordinate the PR merge & deployments order to don’t cause downtimes and always have backward compatibility.

• Testing Environments

  If we wanted to create a testing environment for a specific feature, we needed to identify the PR related to it and deploy + link them manually.

Monorepo

After identifying our current problems in our development flow, we started looking for solutions to increase our development speed, and that’s why we decided to go with a monorepo strategy.

(*) simple representation

To be more precise, at this moment, we are using a hybrid style; we are moving our Web, API & Admin repositories into a monorepo, but our iOS & Android repositories are still on their own repositories.

✅ Pros

Most of the benefits of a monorepo are related to development speed. Let me mention some of them.

One source of truth
Instead of having a lot of repositories, we can have a single one with all the projects, making it easier to manage & It gives us visibility of code used in every project.

Easily sharing code between projects
If there is a common code that has to be used on different projects, we can actually share them easily.

Refactoring across all projects
We can add a new feature or refactor our code easily across all our projects without introducing breaking changes.

Deployments
Since all the code needed to deploy a new feature to production or our testing environments are concentrated in a single repository & branch, we could deploy that single branch to our environments.

⚠️ Cons

Having all the projects on a single repo also introduces some problems, but most of them can be solved by introducing some tools.

Package linking
Package linking can get annoying to don't repeat our code.
→ solution: using a build tool (turborepo)

Limitations Around Access Control
Because of a single repository, we can’t limit the access to single areas, and all the members will have access to all applications code.
→ solution: Nothing we can do here, but since we want to improve our code visibility, this is not a problem. If we want to restrict access to a project, we may need to use a different repository.

Commits could get messy
Making changes inside a monorepo could make your commits difficult to understand.
→ solution: Use a commit linter that enforces commit rules for the repository.

for more details about all the benefits of a monorepo, I recommend checking the following website:

https://monorepo.tools/

Structure

The repository structure of NEWT looks as follow.
The root directory is divided into 3 categories (frontend, backend, shared)

(*) simple representation

Tooling

Let me introduce some of the tools we are using on our monorepo and their specific usage.

Turborepo
Turborepo is a fast build system for JavaScript/TypeScript monorepos.
https://turborepo.org/

Husky
Setup easily git hooks to run your test, linters, etc., before you push your changes.
https://typicode.github.io/husky/#/

Commit Linter
We use conventional commit naming conventions to keep our commits easy to understand.

• https://www.conventionalcommits.org/en/v1.0.0/
• https://github.com/conventional-changelog/commitlint

<type>(<scope>): <description>

- refactor(newt-ui): remove unused Button component
- fix(newt-ui): include onClick prop Button component
- test(newt-ui): add test for Button component
- chore(global): ignore some folder in .gitignore

Vercel
Since we are using Typescript, we can easily create testing environments for our frontend apps & APIs for each branch, allowing us to check the changes quickly.
https://vercel.com/docs/concepts/git/monorepos

Github Actions
We are hosting our code on Github, and we use Github Actions to run our CI/CD.
With GitHub you can create an action for each project by specifying the path.

on:
  push:
    paths:
      - "frotend/storybook/**"

Next

We have already migrated our front-end apps to the monorepo, but our APIs are on the way.
Once all teams actively start using our monorepo, new problems will come out, and I would like to share our learning on a different blog.

We are hiring!

If you are a talented developer who loves to solve problems and discuss with other professionals the best ways to overcome them definitely, you will love working with us.

Feel free to contact us from the recruitment page:
https://www.reiwatravel.co.jp/recruit