The most frequent and problematic of these issues included a lack of consistency across different machines and users when installing dependencies, and the increasing amount of time required to pull dependencies in. There were also security concerns regarding the automatic execution of code from certain npm dependencies.
“Yarn guarantees that the exact same set of dependencies are downloaded on every install, reducing the potential for conflicts across different machines.”
“It’s no secret that major frameworks like Ember, Angular and React stress the limits of the official npm client,” wrote Ember.js co-creator Yehuda Katz, in a blog post about the release. Katz came on board in the later stages of Yarn’s development as an active contributor to the project along with “a couple of dozen other people from a bunch of companies.”
Facebook was far from the only company experiencing npm growing pains, and the original Yarn.js (not to be confused with YARN, the popular Hadoop scheduler) development team reached out to other engineers in the community.
Exponent, Google, and Tilde all contributed brainpower to build out the Yarn client to work seamlessly on every major JS framework, as well as be extensible to other languages.
Yarn is designed to outright replace the npm client in existing workflows while remaining fully compatible with the npm registry. Yarn’s creators say it has the same feature set — while operating faster, more securely, and more reliably — is also set up as a fully open source community project with its own GitHub organization, using the same governance model that has been effective for Rust and Ember.
According to a Facebook blog post:
- “Yarn is ultra fast. After installing a project for the first time, Yarn allows developers to add and update dependencies in just seconds. Projects of any scale will see the benefits of Yarn; the larger the code base, the larger the benefit. In our tests, we saw improvements of 10x on average.
- Yarn guarantees that the exact same set of dependencies are downloaded on every install, reducing the potential for conflicts across different machines. With Yarn, engineers working at any scale still have access to the hundreds of thousands of packages in the npm registry that they need for their projects, with the additional benefits of moving fast and having better control over the way code is executed.”
This “guarantee” is based on rethinking part of the Node ecosystem. The npm client functions non-deterministically, meaning that dependencies go into a given node_modules directory in the order they’re installed by a given user — thus, the structure of a node_modules directory can differ from one person to another, potentially introducing maddeningly obscure “well it works on my machine” bugs.
To resolve versioning issues, Yarn uses lockfiles and a deterministic install algorithm. This locks the installed dependencies to a specific version and ensures that every install produces the same exact file structure in node_modules across all instances. Further, the written lockfile is concise, to ensure that review is simple, and uses ordered keys to ensure that changes are easily made.
“Yarn enables engineers to move faster and with confidence when using shared code so they can focus on what matters — building new products and features,” wrote Facebook engineers Sebastian McKenzie, Christoph Pojer and James Kyle, in the blog post.
Does this mean that npm is broken? Far from it, say those involved most closely with Yarn’s development.
“For all of the complaints people have about the official client, it does a whole lot that people rely on, and the npm team has done a lot to improve it over the years. I genuinely respect their work, and believe that the hard work associated with maintaining a project the size and scope of the npm CLI client is vastly underappreciated, “ Yehuda Katz wrote.
Those not working at the extreme scale of Facebook or Google are unlikely to experience the issues that drove the creation of Yarn as an alternative. Really it’s a “more the merrier” situation, say those most involved in the Node.js universe.
Still, others are happy for an alternative to npm.
Let’s create the next big thing together!
Coming together is a beginning. Keeping together is progress. Working together is success.