Verdaccio is a project with a big ecosystem, composed by several projects, each one with its own configurations and ways to manage. This fact transforms a simple maintenance operation, like updating a common dependency, into a real hard work.
We saw a problem, these configurations entropy made harder to work with all the projects. So, we needed to simplify and unify them to make it easier. We need a monorepo.
A monorepo is a project configuration to manage a collection of dependencies in a simple and unified way. There are many examples out there that Javascript developer use nowadays, like <a href="https://babeljs.io/">Babel</a>, <a href="https://create-react-app.dev/">Create React App</a> or <a href="https://material-ui.com/">Material UI</a>.
Now, we are proud to announce our <a href="https://github.com/verdaccio/monorepo">monorepo</a>, our big ecosystem joined in only one repository. This article is the first part of a series of articles where we will try to explain our motivation about to set up by your own, improve the management and workflows (CI, code quality, etc).

<h2>Background and tools</h2>
A year ago, <a href="https://twitter.com/jotadeveloper">Juan</a> and I met in Madrid, Spain, and were talking about the roadmap for Verdaccio 4, the scope of the projects and more. We had some concerns about the Verdaccio ecosystem we want to build, such amount of repositories requires hard work for maintenance by each one with their own dependencies, scripts, configurations, etc.
We decided to unify all configurations, because handle several repositories would not be realistic and would have all things replicated in all repositories. In order to achieve our goal, and we found <a href="https://lerna.js.org/">Lerna</a> and <a href="https://yarnpkg.com/lang/en/docs/workspaces/">Yarn Workspaces</a>.
As a first step, we created the basic architecture of the monorepo and moved the first dependencies in, an ESLint config and a Babel preset.
After Verdaccio 4 release, Typescript migration and more, we saw that many projects shares the same architecture, so it could be a good moment to achieve our goals.
There are more tools for this purpose, but we will focus on Lerna and Yarn Workspaces.
<h3>Lerna</h3>
Lerna is a tool to manage several Javascript projects with multiple packages (called monorepos), optimizing the workflows around them.
You can install it globally with <code>npm install --global lerna</code>, <code>yarn global add lerna</code> or your favourite package manager, to run commands with <code>lerna &#x3C;command></code>. Also, if you don't want to install it, you can use package runners such as <code>npx</code>.
<h3>Yarn Workspaces</h3>
Yarn Workspaces is a way to setup package architecture where all packages dependencies are installed together with a single <code>yarn install</code>.
This involves two things that you could not see at the first moment.
<ul>
<li>All the packages in the workspace uses a common lockfile as a single source of truth.</li>
<li>If a package has a dependency on other package in the workspace, they are linked without affecting your global environment.</li>
</ul>
Since Yarn 1.0, this feature is enabled by default, you only need a root <code>package.json</code> to setup them.
<h2>Creating the monorepo</h2>
The initial setup is really simple, you only need to create a new repository and run <code>lerna init</code> to initialize the monorepo. This will generate files like <code>package.json</code> or <code>lerna.json</code> (packages structure is shown as example, but not generated).
<pre><code>lerna.json
package.json
packages/
├── pkg1
│ └── package.json
├── pkg2
│ └── package.json
└── pkg3
 └── package.json
</code></pre>
Let's see the main configuration for <code>lerna.json</code> and root <code>package.json</code>.
<h3>lerna.json</h3>
After initial setup, this file will looks like:
<pre><code class="language-json">{
 "packages": ["packages/*"],
 "version": "0.0.0"
}
</code></pre>
<ul>
<li>packages: this array defines the location for all packages that conforms the monorepo. They could be explicit (<code>packages/pkg1</code>) or, if a folder has several packages, you can use <code>*</code> wildcard. In our case, we use <code>core/*</code>, <code>plugins/*</code> and <code>tools/*</code>.</li>
<li>version: the version of the packages. It could be a semver value (called fixed mode) or <code>independent</code> if you want to let packages define their own version. Take care that independent mode will create one tag for each package that will be published with its own version. We prefer fixed mode to keep all packages with the same version and reduce headaches to users.</li>
</ul>
Other interesting settings are:
<ul>
<li>npmClient: you can define in you want to use Yarn, Npm or your favourite client.</li>
<li>useWorkspaces: if you want to enable Yarn Workspaces, you will have to tell it to Lerna, setting this option to <code>true</code>.</li>
<li>stream: if you want to have the output of a child process inmediately in the console, you have to enable this option. Also, this will prefix each line with the package name that generate them.</li>
</ul>
<h3>package.json</h3>
After initial setup, you need some configuration to allow Lerna work with Yarn Workspaces:
<ul>
<li>workspaces: this is the array where we define the packages that are part of the workspace. In a simple way, this is the same you have defined in <code>lerna.json</code> under the packages key.</li>
<li>private: as the root package should be a simple container, you should keep it with <code>true</code> value to not publish it.</li>
</ul>
Later, you will learn how to define more settings in the root <code>package.json</code>.
<h2>Creating and importing packages</h2>
You have a monorepo, but it's not useful at this moment. Let's create and import some packages.
<h3>lerna create</h3>
To create new packages, you can use <code>lerna create &#x3C;package_name></code> like you would do with <code>npm init</code> or <code>yarn init</code>. The wizard will ask you for some fields like package description, author or license. Other way to give that information is using command options (<code>--description</code>, <code>--author</code>, etc).
Then, you have the package ready for work with it, add stuff like Babel or ESLint, dependencies and scripts to <code>package.json</code>...
It's important to say that if your package is a scoped package, you have to add the next codeblock in <code>package.json</code>, but do not add it if your package is not scoped, because <code>lerna publish</code> will fail in this case:
<pre><code class="language-json">"publishConfig": {
 "access": "public"
}
</code></pre>
<h3>lerna import</h3>
If you have a project you want to import, you don't have to create a new one in the monorepo, you can import it using <code>lerna import &#x3C;path_to_project></code>.
This command will read all the Git history from the project specified and apply commit-by-commit into your monorepo to avoid losing the original history. If there were commits with conflicts, the import process will fail, but CLI propose you to use the <code>--flatten</code> option to bypass it. Also, if you want to keep original authors and committers, you can pass <code>--preserve-commit</code> option.
After the import completes, you can remove unnecessary stuff like CI settings, old scripts, hoisted devDependencies...
<h2>Managing dependencies and devDependencies</h2>
Each package will contain their own dependencies and devDependencies like if the package is not in a monorepo, but there are some interesting things you can do with devDependencies, hoist them in the root <code>package.json</code>.
Let's see with an example, pkg1 and pkg2 defines ESLint as devDependency, so you have defined that in two packages, duplicating the definition and management. You can extract it from both and set it as devDependency in the root <code>package.json</code>. This way, all the projects that need it will have it available. Now, we are going to add ESLint to pkg3, you only have to add its own configuration, because package has been hoisted right now.
But the same doesn't work with dependencies, because they are needed when the package is published.
A good practice is to hoist every devDependency so they will be available for every package, except in two cases:
<ul>
<li>A package needs a specific version of the package. In this case, you can have a root definition for all the packages and the specific version for the package that requires it. This will create a <code>node_modules</code> for the specific package, but not another <code>yarn.lock</code>.</li>
<li>Those devDependencies that are part of the workspace must not be defined in the root <code>package.json</code>, because if you do that, you will create a cycle. An example is if pkg1 and pkg2 define pkg3 as devDependency, it could not be defined in the root because pkg3 will depend with itself.</li>
</ul>
<h2>Running scripts</h2>
Like dependencies, each package will have their own scripts, so you should define them in their specific <code>package.json</code>.
But what happens when you want to run scripts for many packages at the same time? You don't need to extract them to the root <code>package.json</code> because they will contain specific arguments/options for each project. You can invoke scripts or commands from the root package using two Lerna commands, <code>lerna run</code> and <code>lerna exec</code>.
The first, <code>lerna run &#x3C;script></code>, will perform the script provided looking for what packages have it defined in their <code>packages.json</code>. This command is useful when you want to build package or run tests, because not all packages would have them defined.
<center>
 <img src="https://cdn.verdaccio.dev/blog/managing-multiples-projects-with-lerna-and-yarn-workspaces/lerna_run_example.png" alt="lerna run example" />
</center>
The second, <code>lerna exec &#x3C;command></code>, will execute the command (not script) in all the packages. This is useful if you want to run tools like ESLint in all the packages and you have it installed globally. In this case, the command invoked must be in your system PATH (<code>ls</code>, <code>cat</code>, npm binaries, etc).
<center>
 <img src="https://cdn.verdaccio.dev/blog/managing-multiples-projects-with-lerna-and-yarn-workspaces/lerna_exec_example.png" alt="lerna exec example" />
</center>
Both commands shares options like <code>--scope=&#x3C;packages></code> and <code>--ignore=&#x3C;packages></code>, where the first will run only in packages specified and the last will ignore them.
<h2>Versioning the monorepo</h2>
As we mentioned in <a href="#lerna.json">lerna.json</a> section there are two versioning ways for packages in monorepos: fixed mode and independent mode.
We will focus in fixed mode because:
<ul>
<li>We want to use the same version for all the packages in the ecosystem.</li>
<li>Independent mode creates one git tag for each package and version in each release. With the example before, in a release we will create three tags, pkg1@0.1.0, pkg2@2.1.0, pkg3@0.5.2. In larger codebases you will create more than 10 tags at the same time.</li>
</ul>
In fixed mode, the version set in the lerna.json is a reference for all the packages but, if a package has no changes between releases, that package will not be published except when the version bump is major (from X.Y.Z to X+1.0.0).
Because Lerna is going to manage the versioning, you should change your mind to use <code>lerna version</code> and <code>lerna publish</code> commands.
<ul>
<li><code>lerna version</code> will update version for the packages that has changes from the release (you can review with <code>lerna changed</code>). This will launch a wizard except if you pass the version or a semver keyword (major, minor, patch, etc) as the first argument or use Conventional Commits (see below). This command updates version in all affected packages, commit changes, create tags and pushes to the remote automatically. 
Also, you can use <code>--conventional-commits</code> option if your commits follows <a href="https://www.conventionalcommits.org">Conventional Commits spec</a>, automating the changelog generation. Additionally, for Github and Gitlab users, you can use <code>--create-release &#x3C;github | gitlab></code> to create release with changes. Keep in mind that you have to provide an auth token (GH_TOKEN or GL_TOKEN).</li>
<li><code>lerna publish</code> will act as <code>lerna version</code> and publish packages if you don't provide <code>from-git</code> or <code>from-package</code> arguments, or only publish if provide one. You would like to use <code>from-git</code> to version first, using Git as a single source of truth.</li>
</ul>
<h2>Conclusion and thanks</h2>
Lerna and Yarn Workspaces are a great combination for creating monorepos. In this first part, you have learned how to setup a monorepo, add packages, improve dependency management, scripts and versioning. In the next chapters, you will see more configuration and tooling (and their settings for monorepos), and how to automate some things using Continuous Integration tools.
We have to thank the teams and community behind both projects, specially <a href="https://twitter.com/left_pad">Henry Zhu</a> and <a href="https://twitter.com/evocateur">Daniel Stockman</a> for Lerna, and <a href="https://twitter.com/arcanis">Maël Nison</a> for Yarn. Also, but not less important, we want to thank all the people that makes Verdaccio possible, contributing, donating, documenting, and more.
Thanks for support Verdaccio ! 👏👏👏👏.

Sergio Herrera

My Blog Posts