{"id":59,"date":"2019-08-01T15:08:13","date_gmt":"2019-08-01T13:08:13","guid":{"rendered":"https:\/\/www.codemotion.com\/magazine\/npm-from-its-conception-to-its-future\/"},"modified":"2019-12-10T11:39:20","modified_gmt":"2019-12-10T10:39:20","slug":"npm-from-its-conception-to-its-future","status":"publish","type":"post","link":"https:\/\/www.codemotion.com\/magazine\/backend\/npm-from-its-conception-to-its-future\/","title":{"rendered":"npm: from its conception to its future"},"content":{"rendered":"\n

npm<\/strong> is a dependency manager for the NodeJS<\/strong> ecosystem, composed of two main elements:<\/p>\n\n\n\n

  1. a registry with all the packages that can be installed in a NodeJS project<\/li>
  2. the npm CLI, which provides the ability to workout which dependencies are required for the project<\/li><\/ol>\n\n\n\n

    npm is bundled together with NodeJS<\/strong>, so if you have a NodeJS environment on your machine, there is also npm installed with that.<\/p>\n\n\n\n

    The packages in the registry are immutable for each version: this means that it’s not possible to update a package submitted to the npm registry, but rather it is mandatory to publish a new version for it.<\/p>\n\n\n\n

    How npm CLI works?<\/h2>\n\n\n\n

    It all starts with npm install<\/code>.<\/p>\n\n\n\n

    The npm CLI is the tool that computes the dependencies required by the project, fetches them from the registry (if not downloaded yet, i.e. when adding a new library to an existing project) and executes building scripts, putting all the results into the node_modules<\/code> folder. The best feature of npm CLI is the local dependency trees for each project, so that no conflict can arise for dependencies when the server is hosting multiple applications.<\/p>\n\n\n\n

    First releases of npm CLI were using pretty simple approaches, downloading recursively the dependencies for each package: that led to very nested folders inside node_modules<\/code> and pretty “fat” project folders, hence the meme of the heaviest object in the universe:<\/p>\n\n\n\n

    <\/figure><\/div>\n\n\n\n

    The process was also slow as many packages had to be downloaded from the registry and a lot of disk I\/O was required to complete the process.<\/p>\n\n\n\n

    Managing the dependency tree is a tricky task, as we\u2019ll see later in the \u201clock file\u201d section. Indeed it requires not only a better logistic approach for the download and disk I\/O but also the foundation for the reproducibility of the installation process.<\/p>\n\n\n\n

    Since npm CLI version 3, the approach has been smarter, so that shared dependencies are worked out to build a flatten tree, less download and disk I\/O in general was required. Newer versions of npm CLI refined this approach to reduce as much as possible the overhead required to install, challenged by another competitor arised meanwhile: yarn<\/code>.<\/p>\n\n\n\n

    What is yarn?<\/h2>\n\n\n\n

    yarn<\/code> is another CLI application, a npm registry client, which aims to map the same feature of npm, but faster<\/strong>. The yarn<\/code> project was created by some big companies like Google and Facebook amongst the others to provide a secure and faster alternative to npm. It periodically makes a copy of the npm registry and provides a lock file (yarn.lock<\/code>) by default.<\/p>\n\n\n\n

    Since yarn<\/code> introduced the lock file by default, also npm started providing one (package-lock.json<\/em>), which was before an optional feature behind a flag.<\/p>\n\n\n\n

    But what exactly is this lock file? Why yarn<\/code> decided it was so important to have it by default?<\/p>\n\n\n\n

    The lock file<\/h2>\n\n\n\n

    The lock file<\/strong> aims to solve a typical problem of dependency management systems: the reproducibility of the installation process<\/strong>. With a single package.json<\/em> file (as in the original npm<\/code> approach), the developer has the ability to choose which versions of the libraries s\/he wants to use, but has no power on the rest of the dependency tree (which can be very deep). This means that an author of a dependency expressed in the package.json<\/em> (or even a sub-dependency at any level of the tree) could arbitrarily update his library in a way that can lead to a conflict in the final project.<\/p>\n\n\n\n

    The lock file makes a snapshot of the whole tree once installed and use this snapshot, when available, to replicate the same setup: a sort of “work on my machine” extended to the dependency management.<\/p>\n\n\n\n

    The next generation<\/h2>\n\n\n\n

    So far we’ve been talking about npm and its history, as well as about yarn<\/code>. But have they reached the tip of the mountain in terms of speed and user experience for dependency management? Well, not yet.<\/p>\n\n\n\n

    Both projects are still pursuing the performance target to improve the CLI and have an almost zero-time install experience. Interestingly, during the latest JSconf EU 2019 both the projects showcased two completely opposite approaches to this goal.<\/p>\n\n\n\n

    npm presented the tink<\/strong> project, which eventually will be merged into the next npm CLI project. The idea is to let tink<\/code> compute the dependencies just before runtime, fetching only the strictly required files from each dependency. This means no more packages in node_modules<\/code> with test and documentation folders, source code or unused dependencies. For production environments, a special command will be provided to bundle this list of files and install them.<\/p>\n\n\n\n

    On the other hand the yarn project is going in the completely opposite direction, keeping the node_modules<\/code> folder and fill it with compressed package files<\/strong>. Avoiding to uncompress each package saves a lot of CPU and disk I\/O time during the installation process, and additional security checks can be applied to them. In the end, their idea is to commit the node_modules<\/code> folder to bring the next level of lock safeness to the project.<\/p>\n\n\n\n

    With both projects presenting their (opposite) ideas on how to improve the install experience, we can now try to figure out what the future will bring.<\/p>\n\n\n\n

    What’s next?<\/h2>\n\n\n\n

    The state of the art for both npm CLI and yarn<\/code> has been presented now, but no particular evolution has been seen on the registry side of npm for many years. This year, at the JSconf EU 2019 conference, a new project on this topic has been presented, by no-less than the former npm CTO C. J. Silverio<\/strong>.<\/p>\n\n\n\n

    We recommend to watch the full talk, as it provides very interesting insights on the npm registry world which only few people in the world can have:<\/p>\n\n\n\n

    \n