JavaScript
Article

Give Grunt the Boot! A Guide to Using npm as a Build Tool

By Peter Dierx

Front-end build and workflow tools are available in abundance: Grunt, Gulp, Broccoli, and Jake to name but a few. These tools can automate almost anything you find yourself doing repeatedly in a project, from minifying and concatenating source files, to running tests or compiling code. But the question is, do you need them? Do you really want to introduce an additional dependency to your project? The answer is “No!”. There is a free alternative that can do the majority of these tasks for you and it comes bundled with Node.js. Of course I’m talking about npm.

In this article we’ll discuss what npm is capable of as a build tool. If you’d like a quick primer on npm before starting, please refer to our beginner’s guide to npm. If you’d like to follow along, you can find the code used in this article on GitHub.

npm Scripts

To start our discussion, we’re going to create a directory for our new demo project, that we’ll call “buildtool”. Once done, we’ll move into this folder and then run the command npm init to create a package.json file:

$ mkdir ~/buildtool && cd ~/buildtool
$ npm init

You’ll be asked several questions. Feel free to skip all or part of them as you’ll replace the final content of the package.json file with the following content:

{
  "name": "buildtool",
  "version": "1.0.0",
  "description": "npm as a build tool",
  "dependencies": {},
  "devDependencies": {},
  "scripts": {
    "info": "echo 'npm as a build tool'"
  },
  "author": "SitePoint",
  "license": "ISC"
}

As you can see, we have a scripts object with a property called info. The value of info is going to be executed in the shell as a command. We can see a list of the scripts properties (also known as commands) and values defined in a project by running the command:

$ npm run

If you run the previous command in our project folder, you should see the following result:

Scripts available in buildtool via `npm run-script`:
  info
    echo 'npm as a build tool'

In case you want to run a specific property, you can run the command:

$ npm run <property>

So, to run the info command we defined in the package.json file, we have to write:

$ npm run info

It’ll produce the following output:

$ npm run info
> buildtool@1.0.0 info /home/sitepoint/buildtool
> echo 'npm as a build tool'

npm as a build tool

If you only want the output of info, you can use the -s flag which silences output from npm:

$ npm run info -s
npm as a build tool

We only used a simple echo so far, but this is a very powerful feature. Everything on the command line is available to us and we can be very creative here. So let’s build on what we’ve covered up to this point and install some packages to create some common workflows.

Common Workflows

The first thing we would like to implement is a linting capability for our JavaScript files. This involves running a program that will analyse our code for potential errors. We are going to use JSHint for this, so the first step is to install the package via npm:

$ npm install jshint --save-dev

After you execute this command, you’ll see a new subfolder named node_modules. This is where JSHint has been downloaded. In addition, we also need to create the following folder structure for our project:

├── assets
│   ├── css
│   │   └── main.css
│   └── scripts
│       └── main.js
├── dist
├── package.json
├── node_modules
└── test
    └── test.js

On a Unix system, this can be done with the following command:

$ mkdir -p assets/css assets/scripts test && touch assets/css/main.css assets/scripts/main.js test/test.js

Linting

Now we’ll force some syntax errors in the main.js file. At the moment the file is empty, so open it and paste in the following content:

"use strict";

var Author = new function(name){
  this.name = name || "Anonymous";
  this.articles = new Array();
}

Author.prototype.writeArticle = function(title){
  this.articles.push(title);
};

Author.prototype.listArticles = function(){
  return this.name + " has written: " + this.articles.join(", ");
};

exports.Author = Author;

var peter = new Author("Peter");
peter.writeArticle("A Beginners Guide to npm");
peter.writeArticle("Using npm as a build tool");
peter.listArticles();

Hopefully the intent of this code is clear — we are declaring a constructor function whose purpose it is to create new Author objects. We also attach a couple of methods to Author’s prototype property which will allow us to store and list the articles an author has written. Notice the exports statement which will make our code available outside of the module in which it is defined. If you’re interested in finding out more about this, be sure to read: Understanding module.exports and exports in Node.js.

Next, we have to add a property to our scripts object in package.json that will trigger jshint. To do that, we’ll create a lint property as follows:

"scripts": {
  "info": "echo 'npm as a build tool'",
  "lint": "echo '=> linting' && jshint assets/scripts/*.js"
}

Here we’re taking advantage of the && operator to chain the commands and file globs (the asterisk) which gets treated as a wildcard, in this case matching any file with a .js ending within the script directory.

Note: the Windows command line does not support globs, but when given a command line argument such as *.js, Windows passes it verbatim to the calling application. This means that vendors can install compatibility libraries to give Windows glob like functionality. JSHint uses the minimatch library for this purpose.

Now let’s lint the code:

npm run lint -s

This produces the following output:

=> linting
assets/scripts/main.js: line 1, col 1, Use the function form of "use strict".
assets/scripts/main.js: line 5, col 28, The array literal notation [] is preferable.
assets/scripts/main.js: line 3, col 14, Weird construction. Is 'new' necessary?
assets/scripts/main.js: line 6, col 1, Missing '()' invoking a constructor.
assets/scripts/main.js: line 6, col 2, Missing semicolon.
assets/scripts/main.js: line 16, col 1, 'exports' is not defined.

6 errors

It works. Let’s clean up those errors, re-run the linter to make sure, then move on to some testing:

(function(){
  "use strict";

  var Author = function(name){
    this.name = name || "Anonymous";
    this.articles = [];
  };

  Author.prototype.writeArticle = function(title){
    this.articles.push(title);
  };

  Author.prototype.listArticles = function(){
    return this.name + " has written: " + this.articles.join(", ");
  };

  exports.Author = Author;

  var peter = new Author("Peter");
  peter.writeArticle("A Beginners Guide to npm");
  peter.writeArticle("Using npm as a build tool");
  peter.listArticles();
})();

Notice how we have wrapped everything in an immediately invoked function expression.

npm run lint -s
=> linting

No errors. We’re good!

Testing

First we need to install the mocha package. Mocha is a simple, yet flexible JavaScript test framework for Node.js and the browser. If you’d like to read more about it, this article is a great place to start: Basic Front End Testing With Mocha & Chai

npm install mocha --save-dev

Next we are going to create some simple tests to test the methods we wrote previously. Open up test.js and add the following content (notice the require statement which makes our code available to mocha):

var assert = require("assert");
var Author = require("../assets/scripts/main.js").Author;

describe("Author", function(){
  describe("constructor", function(){
    it("should have a default name", function(){
      var author = new Author();
      assert.equal("Anonymous", author.name);
    });
  });

  describe("#writeArticle", function(){
    it("should store articles", function(){
      var author = new Author();
      assert.equal(0, author.articles.length);
      author.writeArticle("test article");
      assert.equal(1, author.articles.length);
    });
  });

  describe("#listArticles", function(){
    it("should list articles", function(){
      var author = new Author("Jim");
      author.writeArticle("a great article");
      assert.equal("Jim has written: a great article", author.listArticles());
    });
  });
});

Now let’s add a test task to package.json:

"scripts": {
  "info": "echo 'npm as a build tool'",
  "lint": "echo '=> linting' && jshint assets/scripts/*.js",
  "test": "echo '=> testing' && mocha test/"
}

npm has a few convenient shortcuts, namely npm test, npm start and npm stop. These are are all aliases for their run equivalents, which means we just need to run npm test to kick mocha into action:

$ npm test -s
=> testing

  Author
    constructor
      ✓ should have a default name
    #writeArticle
      ✓ should store articles
    #listArticles
      ✓ should list articles

  3 passing (5ms)

Pre and Post Hooks

It wouldn’t be very efficient if we were to run our test suite and it bailed straight away because of a syntax error. Luckily npm gives us the pre and post hooks, so if you run npm run test it will first execute npm run pretest and npm run posttest when it finishes. In this case we want to run the lint script before the test script. The following pretest script makes this possible.

"scripts": {
  "info": "echo 'npm as a build tool'",
  "lint": "echo '=> linting' && jshint assets/scripts/*.js",
  "test": "echo '=> testing' && mocha test/",
  "pretest": "npm run lint -s"
}

Imagine we hadn’t have corrected the syntax errors in our script previously. In this case, the above pretest script will fail with a non-zero exit code and the test script won’t run. That is exactly the behavior we want.

$ npm test -s
=> linting
assets/scripts/main.js: line 1, col 1, Use the function form of "use strict".
...
6 errors

With the corrected code in main.js:

=> linting
=> testing

  Author
    constructor
      ✓ should have a default name
    #writeArticle
      ✓ should store articles
    #listArticles
      ✓ should list articles

  3 passing (6ms)

We are in the green !

Code Minification

For this section, we’ll need to add a dist directory to our project, as well as several sub directories and files. This is how the folder structure looks:

   ├── dist
   │   └── public
   │       ├── css
   │       ├── index.html
   │       └── js

The command to recreate this on a Unix machine is:

mkdir -p dist/public/css dist/public/js && touch dist/public/index.html

The contents of index.html is simple.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>npm as a build tool</title>
    <link href='css/main.min.css' rel='stylesheet'>
  </head>
  <body>
    <h2>npm as a build tool</h2>
    <script src='js/main.min.js'></script>
  </body>
</html>

Currently main.js is not minified. This is as it should be, because it is the file we are working in and we need to be able to read it. However, before we upload it to the live server, we need to reduce its size and place it in the dist/public/js directory. To do this we can install the uglify-js package and make a new script.

$ npm install uglify-js --save-dev

We can now make a new minify:js script in package.json:

"scripts": {
  "info": "echo 'npm as a build tool'",
  "lint": "echo '=> linting' && jshint assets/scripts/*.js",
  "test": "echo '=> testing' && mocha test/",
  "minify:js": "echo '=> minify:js' && uglifyjs assets/scripts/main.js -o dist/public/js/main.min.js",
  "pretest": "npm run lint -s"
}

Run it:

$ npm run minify:js -s
=> minify:js

And the script creates a minified version of our file in the right destination. We will do the same for our CSS file using the clean-css package.

$ npm install clean-css --save-dev

And create the minify:css script.

"scripts": {
  "info": "echo 'npm as a build tool'",
  "lint": "echo '=> linting' && jshint assets/scripts/*.js",
  "test": "echo '=> testing' && mocha test/",
  "minify:js": "echo '=> minify:js' && uglifyjs assets/scripts/main.js -o dist/public/js/main.min.js",
  "minify:css": "echo '=> minify:css' && cleancss assets/css/main.css -o dist/public/css/main.min.css",
  "pretest": "npm run lint -s"
}

Let’s run the script.

$ npm run minify:css -s
=> minify:css

Watching for Changes

One of the things that Grunt, Gulp and their ilk are great at, is watching a set of files and re-running a specific task whenever one of those files is detected to have changed. This is particularly useful in cases such as this, as it would be a pain in the neck to re-run the minification scripts manually.

The good news is that you can do that in npm, too, using a package such as watch, which is intended to make managing the watching of file and directory trees easier.

$ npm install watch --save-dev

Then in package.json, you need to specify the tasks to run when a change is detected. In this case JavaScript and CSS minification:

"scripts": {
  ...
  "watch": "watch 'npm run minify:js && npm run minify:css' assets/scripts/ assets/css/"
}

Start the script using:

$ npm run watch

Now, whenever any file in assets/scripts/ or assets/css/ changes, the minification scripts will be called automatically.

Build Script

By now we have several scripts that we can chain together to make a build script which should do the following: linting, testing and minifying. It would, after all, be a pain to have to run these tasks individually time after time. To create this build script, alter the script object in package.json, thus:

"scripts": {
  "info": "echo 'npm as a build tool'",
  "lint": "echo '=> linting' && jshint assets/scripts/*.js",
  "test": "echo '=> testing' && mocha test/",
  "minify:js": "echo '=> minify:js' && uglifyjs assets/scripts/main.js -o dist/public/js/jquery.min.js",
  "minify:css": "echo '=> minify:css' && cleancss assets/css/main.css -o dist/public/css/main.min.css",
  "build": "echo '=> building' && npm run test -s && npm run minify:js -s && npm run minify:css -s",
  "pretest": "npm run lint -s"
}

Running the build script gives us the following output.

$ npm run build -s
=> building
=> linting
=> testing

  Author
    constructor
      ✓ should have a default name
    #writeArticle
      ✓ should store articles
    #listArticles
      ✓ should list articles

  3 passing (6ms)

=> minify:js
=> minify:css

Server Script

After we run our build script it would be nice if we could start a server for our content in dist and check it in the browser. We can do this using the http-server package.

$ npm install http-server -save-dev

We make a server script.

"scripts": {
  ...
  "server": "http-server dist/public/",
}

And now we can run our server.

$ npm run server
Starting up http-server, serving dist/public/ on: http://0.0.0.0:8080
Hit CTRL-C to stop the server
_

Of course the server script can be added to the build script, but I leave that as an exercise for the reader.

Conclusion

Hopefully this article has demonstrated how flexible and powerful npm can be as a build tool. The next time you are starting a new project, try not to reach straight for a tool such as Gulp or Grunt — try to solve your needs by using npm only. You might be pleasantly surprised.

If you have any questions or comments, I’d be glad to hear them in the thread below.

Peter Dierx
Meet the author
Peter is a freelance developer from the Netherlands building Ruby on Rails web applications for his clients. He also likes to play with front-end JavaScript frameworks, and is interested in new web technologies in general. In his spare time he rides his bicycle every day and he is also a passionate skydiver.
  • martinczerwi

    Thanks for this article. At first I didn’t know that npm run adds the binaries folder from node_modules to the $PATH variable, but after trying and reading the help page I got it. Would be great, if it was possible to call the local binaries without defining a script, like using a string: npm run “jshint”

    • http://careersreport.com elizabeth_barnes6

      Here~ is how you can make 65 bucks /hr… After being unemployed for half-a-year , I started working over this website and today I am verry happy. After 3 months doing this my income is around 5000 bucksmonth -Check link on MY-PROFILE for more info

    • http://www.joezimjs.com Joe Zimmerman

      If you add “./node_modules/.bin/” to your PATH, you can just use the “jshint” command to run jshint as long as you are in the root directory of a project with jshint installed. This is great for those quick tests of the different arguments with a tool you haven’t used before without having to constantly edit the package.json file for each variation you try.

      I was quite surprised when I first found out that I could use relative PATHs and it’s quite nice.

      • http://www.realchaseadams.com/ Chase Adams

        Just a protip, it’s better to use $(npm bin):PATH rather than the relative path (it’s best practice to use absolute paths in your $PATH).

        • http://www.joezimjs.com Joe Zimmerman

          What’s $(npm bin):PATH? I purposefully want a relative path so it pulls the file relative to the project I’m in.

          • http://www.realchaseadams.com/ Chase Adams

            Did you try it? That’s exactly what it does. `npm bin` will output your current project’s bin path, which is your project’s directory path at the time of running the command.

          • http://www.joezimjs.com Joe Zimmerman

            I’m on a Windows machine, so this isn’t working… I assume the `:PATH` adds it to the PATH? This is nice because `npm bin` will give me the correct directory even if I’m not in the root of the project, which is nice, but I can’t see a way to add $(npm bin) to my PATH in Windows.

          • http://www.realchaseadams.com/ Chase Adams

            I’ve not done any Windows development but my guess is a Google search for something like this will get you in the right direction: “System Properties -> Environmental Variables -> PATH npm bin”

          • http://www.joezimjs.com Joe Zimmerman

            I’ve done a few searches and haven’t been able to find anything, so I assume it’s impossible. I was just hoping someone knew.

        • http://www.joezimjs.com Joe Zimmerman

          Are you adding $(npm bin) to your PATH permanently (`export PATH=$(npm bin):$PATH` in your Bash profile) are you you throwing that in front of the command as you use it (`PATH=$(npm bin):$PATH jshint`)?

          I don’t have a Linux machine, but I’m using Git Bash which emulates Linux in most ways and I’m trying to add it to my Bash profile and it’s not working. So, I want to make sure you can do that on a Linux machine or that I misunderstood.

          If you have to type in `$(npm bin)/jshint` every time, it’s not all that great. I’d rather just use the npm scripts at that point.

          • http://www.realchaseadams.com/ Chase Adams

            Correct, you just need to export PATH as the above in your bash profile, source your bash profile and it’ll pick up your project’s root node_modules/.bin

          • http://www.joezimjs.com Joe Zimmerman

            Interesting… I wonder if the reason it isn’t working for me is because it’s still technically Windows.

          • http://www.joezimjs.com Joe Zimmerman

            Nope. Just tested this on a Linux machine. The moment you source .bash_profile, it’ll generate a value for npm bin based on the folder you’re currently in and it will remain static. It doesn’t dynamically call npm bin every time you try to execute a command in order to have the latest version of what it would output.

          • http://www.realchaseadams.com/ Chase Adams

            if you echo the $PATH with bash It’s “true” but it’s also not true when it comes to npm run. If you create a script in your `package.json` called `whatpath` and set it to `echo $PATH`, you’ll see that `$PATH` is executed with _your_ package’s path.

          • http://www.joezimjs.com Joe Zimmerman

            npm scripts automatically augment your PATH with the current npm bin, regardless of whether or not you add $(npm bin) to the PATH yourself. We’re talking about being able to use local bins without needing to use npm scripts.

            I want to be able to just type which gulp and see that it’s using the one in my nearest npm bin, no matter where I am. That doesn’t happen when adding $(npm bin) to your PATH yourself.

          • http://www.realchaseadams.com/ Chase Adams

            Great point, sorry for creating confusion in that space. Maybe we can blow out this section of the thread so no one else stumbles into it and uses it.

            I agree, it sucks that there isn’t a better way to dynamically infer which `npm bin` to use.

    • http://www.joezimjs.com Joe Zimmerman

      If you add “./node_modules/.bin/” to your PATH, you can just use the “jshint” command to run jshint as long as you are in the root directory of a project with jshint installed. This is great for those quick tests of the different arguments with a tool you haven’t used before without having to constantly edit the package.json file for each variation you try.

      I was quite surprised when I first found out that I could use relative PATHs and it’s quite nice.

  • http://jamessteinbach.com/ James Steinbach

    Have you given node-sass a try yet? https://www.npmjs.com/package/node-sass

    gulp-sass & grunt-sass are basically wrappers around this. And node-sass uses libsass, so there’s no Ruby dependency and you get really fast compile times.

    • http://careersreport.com Kathy Compton

      Allow ~me to show you a genuine way to earn a lot of extra money by finishing basic tasks from your house for few short hours a day — See more info by visiting >MY*&___(DISQUS)*%___ID)

  • Kyle Hall

    I’m having trouble with using wildcards on windows in the linting step. It won’t find “*.js”, but it finds “index.js” just fine. After checking in cmd, powershell, and git bash, all of which have wildcard support, I’m stuck. Any ideas?

    • http://www.joezimjs.com Joe Zimmerman

      Yes, this is a Windows problem. Many (most?) command line packages from npm make this sort of thing work on Windows, but apparently jshint doesn’t. eslint works with the wildcard value.

      • Kyle Hall

        I was able to get it working with the suggestion from the other reply, but thanks for the information about how things work with Windows and suggesting a different library.

    • Jacob

      to make it work on windows you can just pass the entire directory instead of using the glob.
      this:
      jshint js
      instead of this:
      jshint js/*.js

      • Kyle Hall

        Thanks, that did the trick

  • Kevin

    I got the below when I used the wildcard * so I had to specify `main.js` in the package.json

    $ npm run lint -s

    ERROR: Can’t open assets/scripts/*.js

  • Kevin

    I got the below when I used the wildcard * so I had to specify `main.js` in the package.json

    $ npm run lint -s

    ERROR: Can’t open assets/scripts/*.js

  • Kevin

    I also had to write these lines like this in order for it to print `=> linting`
    “`
    “lint”: “echo “=> linting” && jshint assets/scripts/main.js”,
    “test”: “echo “=> testing” && mocha test/”,

    “`

  • pauleveritt

    I think you’ve presented a false choice. Yes, you shouldn’t have ginormous npm scripts. But that doesn’t mean you have to adopt something like gulp as an intermediary, adding complexity to things like Webpack.

    For the latter, most people move all those knobs into a JavaScript (meaning, capable of logic) config file webpack.config.js. Your npm build script is then a manageable call to
    Webpack. Instead of gulp+webpack, it’s just Webpack.

    • http://www.joezimjs.com Joe Zimmerman

      If WebPack could do everything I wanted, I’d just use WebPack with a single config file as you say, but it doesn’t do everything yet so I wrap it in Gulp or plain old JavaScript and take care of the rest in there.

      • pauleveritt

        Sorry, I wasn’t meaning to imply switching to Webpack. I just used it as an example for bundlers.

        All bundlers are meant to be usable standalone, not tied to a particular build tool. But perhaps you have some ultra-edge case that can’t be met by the bundler alone.

  • Kevin

    could not get the minify css to work. the css file is currently empty, may be that has something to do with it?

    $ npm run minify:css -s

    “=> minify:css”

    Error: EINVAL, invalid argument

    at new Socket (net.js:156:18)

    at process.stdin (node.js:664:19)

    at Object. (C:cygwin64homeUser Namenpm_tutbuildtoolnode_modulesclean-cssbincleancss:56:52)

    at Module._compile (module.js:456:26)

    at Object.Module._extensions..js (module.js:474:10)

    at Module.load (module.js:356:32)

    at Function.Module._load (module.js:312:12)

    at Function.Module.runMain (module.js:497:10)

    at startup (node.js:119:16)

    at node.js:906:3

  • davidofkent

    The basic scripts such as compiling jade and stylus and uglifying JS work nicely. My build script works fine. However I cannot get ‘watch’ to work successfully with any of the others. However, I’m no expert. OTOH, I use Gulp with ease.

  • http://www.siteup.ro/international/ Vladi

    There’s also the thing with “watching” for changes. As far as I know, we should use the built-in npm watching functionality because it also provides caching and is faster than a module.

  • Deimyts

    Great article! To the point and easy to follow. I’ve been wanting to start using npm instead of grunt/gulp for a while, but never sat down to do figure it out before. Thanks for making it easy!

    Had a few questions.
    1) Are there any other default-ish tasks besides those listed here that you find yourself adding to a lot of your projects?
    2) Are there any tasks that you find unwieldy or difficult using npm alone, or a point where you feel your scripts get hard to manage? What do you do in that case?
    3) Related to #2: I like being able to use the basic npm versions of tools, instead of the Grunt/Gulp versions. However, Grunt & Gulp both seem to offer more in terms of readability and organization, which gets more important the more complex your build process becomes. Any tips for keeping your npm scripts manageable?

    Also, I noticed a typo in the http-server install line: `$ npm install http-server -save-dev` is missing the second hyphen before save-dev. :)

    Thanks again for the excellent write-up.

  • Ja

    Here is the better tool `npm-watch’,

    It will run test when *js in src/ or in test/ changes.

    {
    “watch”: {
    “test”: “{src,test}/*.js”
    },
    “scripts”: {
    “test”: “tape test/*.js”,
    “watch”: “npm-watch”
    }
    }

  • jose

    I would like to have the functionality of gulp-html-replace as an NPM script. My problem is pretty basic, simply taking the main index.html page and replacing all the script tags with a single bundle.min.js script tag. Maybe the same for the css link tags, too. Any hints?

  • brenotx

    Nice article! I’ll give npm a try. Thank you so much!

Recommended

Learn Coding Online
Learn Web Development

Start learning web development and design for free with SitePoint Premium!

Get the lastest in JavaScript, once a week, for free.