How to Solve the Global npm Module Dependency Problem

By Joe Zimmerman

The Node Package Manager (a.k.a. npm) has given web developers easy access to a lot of awesome JavaScript modules and has made our lives considerably easier when trying to find and manage dependencies for our applications. It also makes it easy for developers to create and publish their own modules, meaning that other developers can grab them with a simple npm install -g your-tool and start using them any time they want to. It’s utopia! Right?

Err, actually …

We’ve Got a Bit of a Problem

I will never say never use the -g option when installing an npm module, but I do have to say that we are causing problems by using it too much. There are a couple reasons that I think we should cut down on our use of global module installation, especially in the case of build, test, or linting tools such as Gulp, Karma, JSHint, and countless others. I’ll be referring primarily to Gulp throughout this article because it’s quite popular and it’s fun to say, but if you don’t like Gulp, just mentally replace it with whatever you prefer.

First of all, global modules are not listed as dependencies in your projects, even though your project depends on them, which causes extra steps for others using your application. You know that you need to use Gulp in order to get your project ready for production, so you install it globally and use it. When someone else wants to start working on, or using your wonderful open source project, they can’t just type npm install and get going. You end up having to throw directions into your README file saying something along the lines of

To use this project, follow these steps:

  • git clone the repo
  • Run npm install
  • Run npm install -g gulp
  • Run gulp to build

I see two issues with this: firstly, you are adding the extra step of installing Gulp globally and secondly, you are running gulp directly. I see an extra step that could have been avoided (globally installing Gulp) and I see that the user is required to know that your app uses Gulp in order to build the project. This first issue is the main one I’m going to address in this article, and although the second one isn’t as big of an issue, you’ll need to update the instructions if you end up switching tools. The solution I discuss later should fix both of these issues.

The second big issue relating to installing modules globally is that you can run into conflicts due to having the wrong version of the module installed. This is illustrated by the following two examples:

  • You created your project six months ago and you used the latest version of Gulp at that time. Today, someone has cloned your project’s repo and tried to run gulp to build it, but runs into errors. This is because the person who cloned your project is either running an older version or a newer version of Gulp that has some breaking differences.
  • You created a project six months ago that used Gulp. Since then you’ve moved on to other projects and updated Gulp on your machine. Now you go back to this old project and try to run gulp and you experience errors because you’ve updated Gulp since the last time you touched the project. Now you are forced to update your build process to work with the new version of Gulp before you can make any more progress on the project, instead of putting it off until a more convenient time.

These are potentially very crippling issues. Like I said earlier though, I wouldn’t make a blanket statement telling you never to install something globally. There are exceptions.

A Brief Note on Security

By default, on some systems, installing a npm module globally requires elevated privileges. If you find yourself running commands like sudo npm install -g a-package, you should change this. Our beginners guide to npm shows you how.

Exceptions to the Rule

So what can you install globally? To put it simply: anything your project doesn’t depend on. For example, I have a global module installed called local-web-server. Whenever I just have some HTML files I want to view in the browser, I’ll just run ws (that’s the command for local-web-server) and it’ll set the current folder as the root for localhost:8000 and I can pop open any documents under there in my browser and test them out.

I also run into situations where I want to minify JavaScript files that aren’t part of a project, or at least aren’t part of a project where I’m allowed to set up a formal build process (for silly “corporate” reasons). For this, I have uglify-js installed and I can easily minify any script from my command line in seconds.

The Solution

Now that we know where issues can arise, how do we prevent them? The first thing you need to do is remove that -g when you install modules. You should replace that with --save-dev so you can save the module as a development dependency and it will always be installed when someone runs npm install. That only solves one of the minor issues that I mentioned, but it’s a start.

What you need to know is that when you install a dependency locally, if it has any scripts that are meant to be run from the command line, they will be placed in ./node_modules/.bin/. So, right now, if you just install Gulp locally, you could run it by typing ./node_modules/.bin/gulp in your command line. Of course, no one wants to type that whole thing in. You can fix this with npm scripts.

Inside your package.json file, you can add a scripts property that looks something like this:

    "scripts": {
        "gulp": "gulp"

Now you can run npm run gulp any time you want to run the local version of Gulp. npm scripts will look for a local copy of an executable command in the ./node_modules/.bin/ directory before checking your PATH for it. If you want, you can even pass other arguments to Gulp by adding -- before those arguments, e.g. npm run gulp -- build-dev is equivalent to gulp build-dev.

It sucks that you still need to type in more than you would if you used Gulp globally, but there are two ways around that. The first way, which also solves one of the issues I brought up earlier, is to use npm scripts to create aliases. For example, you shouldn’t necessarily tie your app to Gulp, so you could create scripts that run Gulp, but do not mention Gulp:

    "scripts": {
        "build": "gulp build-prod",
        "develop": "gulp build-dev"

This way, you can keep your calls to Gulp shorter and you keep your scripts generic. By keeping them generic, you can transparently remove Gulp any time and replace it with something else and no one needs to know (unless they work on the build process, in which case, they should know about it already and probably should have been part of the conversation to move away from Gulp). Optionally, you can even throw a postinstall script in there to automatically run the build process immediately after someone runs npm install. This would clean up your README quite a bit. Also, by using npm scripts, anyone who clones your project should have simple and immediate documentation regarding all the processes you run on your project right in the package.json file.

In addition to using npm scripts, there is another trick that will let you use your local installations of command line tools: a relative PATH. I added ./node_modules/.bin/ to my path, so that as long as I am in a project’s root directory, I have access to the command tools simply by typing in the name of the command. I learned this trick from a comment on a different post I wrote (thanks Gabriel Falkenberg).

These tricks cannot necessarily replace every situation where you’d want to use something like Gulp, and they do take a bit of work to set up, but I do believe it should be a best practice to include those tools listed in your dependencies. This will prevent version clashing (which is one of the main reasons behind dependency managers in the first place) and will help simplify the steps necessary for someone to pick up your project.

Going above and Beyond

This may be a bit excessive, but I also believe that Node and npm are dependencies for your project which have several different versions that can clash. If you want to be sure your application will work for EVERYONE, then you need some way to ensure that the user has the correct versions of Node and npm installed as well.

You can install local copies of Node and npm into your project! This doesn’t make everything fine and dandy, though. First of all, Node isn’t the same on every operating system, so each individual would still need to make sure they download the one that works with their operating system. Secondly, even if there was a way to have a universal Node installed, you would need to make sure that each person has a simple way to access Node and npm from their command line, such as ensuring that everyone adds the path to the local copy of Node and npm to their PATH. There’s no simple way to guarantee this.

So, as much as I’d love to be able to enforce specific versions of Node and npm per project, I can’t think of a good way to do so. If you think it’s a good idea and come up with a good solution, let us all know about it in the comments. I’d love to see a simple enough solution that this could become a standard practice!

The Final Word

I hope you can now see the importance of keeping your tools listed as versioned dependencies for your projects. I also hope that you’re willing to do the work required to implement these practices in your own projects so we can push these practices forward as a standard. Unless of course, you’ve got a better idea, in which case speak up and let the world know about it!

  • I want to show great work opportunity… three to five hours of work daily… Weekly paycheck… Bonus opportunities…Payscale of $6k to $9k /a month.#.. Just few hours of your free time, any kind of computer, elementary understanding of web and stable connection is what is required…Get informed more about it by visiting my profile>page

  • I want to show great work opportunity… three to five hours! of work daily… Weekly paycheck… Bonus opportunities…Payscale of $6k to $9k /a month… Just few hours of your free time, any kind of computer, elementary understanding of web and stable connection is what is required…Get informed more about it by visiting my profile>page

  • I have to share this great internet ~freelancing opportunity… 3 to 5 hours of work /a day… Payment each week… Performance depending bonuses…Payscale of $6k-$9k /month… Just few hours of free time, desktop or laptop, most basic understanding of web and trusted internet connection is what is required…See more on my disqus–page

  • I want to show great work opportunity… three to five hours of work daily… Weekly paycheck… Bonus opportunities…Payscale of $6k to $9k /a month… Just few hours of your free time, any kind of computer, elementary understanding of web and~ stable connection is what is required…Get informed more about it by visiting my profile>page

  • henry jurk

    importance of getting professional website designer in business :- The most major Mobile App Development New Orleans or an essential other choices may be able to be by managing eth work of its own New Orleans App Development means. The other point that are termed will work changes may have App Development New Orleans to be stimulatingly giving you with Web Design New Orleans the aim related goal met. The article is all managed through the process and it gives better chance.

  • Yea, I’m happy to see that this is starting to work its way around the community now and become a little more pervasive, but it still has a long way to go. Thanks for contributing toward making things better!

  • You could make a fully self-contained package (including npm and node) using Docker. You have the additional benefit to be able to have whatever version of npm/node available “globally”.

    • Quite true and that could be a very good solution, but of course then you’re banking on everyone who wants to pick up or contribute to your project using Docker.

  • Shawn Inder

    I used to do things as proposed here, but I eventually went back to global installs. The problem was that all my projects needed the same build tools, so it made all projects really slow to `npm install`. The builds tools in question have dependencies on phantomjs and other really big modules, so the slowness was really ruining our productivity (waiting for 5 full minutes just to install a tiny module because it has tests that need to run in phantomjs, for example). Unfortunately, I am of course back to square one now, experiencing all the problems you describe :(

    To me this is simply a missing feature in npm. One should be able to declare global dependencies along with semantic versioning strings, same as local dependencies. The version declared would be globally installed on `npm install`. In npm scripts, when an “global executable” is called, npm could look in the package to determine which version to run. The only issue left would be about calling these modules outside of npm scripts.. Pretty big issue actually :p Anyone have a clever solution for this? Trying to get the best of all worlds here :p

    • I once thought that being able to specify globals would be nice as well, but as you stated, there are some pretty nasty potential issues that could come up. I’m not really sure why you need to keep re-installing phantomjs all the time? Unless you’re talking about a CI tool taking a really long time to run because it needs to install the deps. Otherwise, you should be able to keep the dependency installed on your machine.

      An alternative, though, could be a local copy of the npm registry so that you and/or your CI tool doesn’t need to go as far to retrieve those deps.

      • Shawn Inder

        CI is indeed a good example. But also on-boarding new employees, people changing projects, setting up new computers… The most problematic case comes from the fact that we’re developing a multitude of inter-related modules at the same time, so for that to be workable, we use npm links a lot. But then we want to test that it also works without linking (verify that everything that needs to be published has been and stuff like that). So the simplest way is `rm -rf node_modules && npm install`. That’s when the slowness really hurts. So most of us ended up having one fully linked setup and another setup with no links at all. But since all our modules are getting new versions daily, we end running `npm install` a lot anyways, just to get the new versions of all modules (instead of doing them all one by one, which is also long). The local npm registry is a good idea though, I’ll look into it when I get a little time :)

  • David Nguyen

    I solved it with installing packages locally and then put it all to “configs” key of npm and then making configurations at “scripts” key. Then use npm run to run. :)

  • Alex Mills

    I have written a couple NPM modules. Suman is the latest and is a new test runner for Node.js. Currently the way it works is if there is a globally installed package and the user runs the global executable – the global executable will look for the locally installed executable and run that instead. Because there should always be a locally installed package, it works great, and this would work for gulp too, because gulp is usually required locally in the gulpfile aka – require(‘gulp’). Also, another solution is to create bash functions that serve as aliases for your executable. In that way, we can do away with the need to ever installed global packages. In the postinstall script for any locally installed package, we write out the bash script that contains the bash functions that serve as aliases to any local project. The user is responsible for sourcing the bash script – they add one line to .bash_profile or whatever they use for zsh.

Get the latest in JavaScript, once a week, for free.