An Introduction to Gulp.js

This article was sponsored by New Relic. Thank you for supporting the sponsors who make SitePoint possible!

We’ve discussed many ways to reduce page weight and increase performance of obese websites. Some are a one-off undertakings, such as enabling gzip compression, switching to an appropriate image format, or removing unnecessary fonts. Other tasks require repeated effort every time you make a change…

  • compressing new or modified images
  • removing console and debugger statements from scripts
  • concatenating and minifying CSS and JavaScript files
  • deploying updates to a production server

You may start with good intentions but the most infallible developer will forget to compress an image or two. Over time, pre-production tasks become increasingly arduous and time-consuming; you’ll dread the inevitable content and template changes. Even if your team has a full-time developer dedicated to running these tasks, it’s mind-numbing and repetitive work. Wouldn’t their time be better spent on more profitable jobs?

What you need is an automated task runner — or build process.

That Sounds Scarily Complicated!

Defining your build process is more complex than performing each task individually but, over time, you’ll save hours of effort and reduce human error. Adopt a pragmatic approach:

  1. concentrate on automating the most time-consuming tasks first
  2. don’t over-engineer your build process; an hour or two is more than enough for the initial set-up
  3. choose task runner software and stick with it for a while. Don’t switch to another system on a whim.

A few of the tools and concepts we’ll discuss may strike fear into your heart, but don’t be afraid of the unknown.

  • Node.js — we’ll be using Node.js, but you don’t need to be a Node.js developer. It’s simply a code runtime built using Chrome’s JavaScript engine, V8. JavaScript knowledge is helpful but there are plenty of cut-and-paste snippets to use and adapt.
  • The command line — you will be typing commands rather than using a GUI, but there’s nothing overly complex.

Grunt vs Gulp

You’ve probably heard about Grunt. Grunt is a Node.js-based task runner.

Gulp is the new kid on the block: it’s a task runner which uses Node.js.

Grunt and Gulp do exactly the same thing. Grunt has been around longer and you’ll find far more help, plug-ins and resources. It’s a great project — if you’re successfully using it now, there’s little reason to switch.

However, nothing is ever perfect and Gulp.js has been developed to solve issues you may have encountered with Grunt:

  • Grunt plug-ins often perform multiple tasks; Gulp plug-ins are designed to do one thing only.
  • Grunt requires plug-ins for basic functionality such as file watching; Gulp has them built-in.
  • Grunt uses JSON-like data configuration files; Gulp uses leaner, simpler JavaScript code.

Not everyone will agree with this last point, but I suggest you view the Gulp presentation slides and decide for yourself.

The most important Gulp concept is streams. Think of your files passing through a pipe; at one or more points along that pipe, an action is taken. For example, we could insert all our JavaScript files into a scripts pipe which:

  1. concatenates files into one
  2. removes console and debugger statements
  3. minifies the code
  4. puts the resulting file in a specific location.

Data is input into one method. That method outputs new data — which is used as input for the next method. It’s reminiscent of jQuery chaining which applies different actions in sequential order, e.g.

$("#element").text("hello world!").addClass("myclass").fadeIn();

Enough theory — let’s start using Gulp.

Step 1: Install Node.js

Node.js can be downloaded for Windows, Mac and Linux at nodejs.org/download/. You can also install it from the command line using a package manager.

Once installed, open a command prompt and enter:

node -v

The installed Node.js version number will be displayed. You can do the same for npm — the Node.js package manager which is used to install modules.

npm -v

If either command fails, check you’ve typed that command in lowercase. No good? Verify you installed Node.js correctly.

Step 2: Install Gulp

We can now install Gulp using npm. We’ll add a -g flag to ensure Gulp is available globally for any project:

npm install gulp -g

If you’re using Mac or Linux, you may need to prepend sudo to the start of this command to grant administrator access, i.e.

sudo npm install gulp -g

If any commands in this tutorial fail, add sudo to the start and you should be fine.

Verify that Gulp has installed the following command:

gulp -v

Step 3: Set Up Your Project

For this example, our project files will be contained in a folder named test. Navigate to this folder from the command line, e.g.

cd test

The actual location of the test folder will differ across systems — ensure you’re in the correct location by typing dir on Windows or ls on Mac/Linux.

Our test folder contains the following sub-folders:

  • src — the location of pre-processed HTML source files and folders:
    • images — uncompressed images
    • scripts — multiple pre-processed script files
    • styles — multiple pre-processed CSS files
  • build — the location of production files for upload including:
    • images — compressed images
    • scripts — a single minified script file
    • styles — a single minified CSS file

(Note that build folders will be created automatically by our tasks.)

First, we must install Gulp locally:

npm install gulp --save-dev

This will create a node_modules folder within test where Gulp and plug-in code resides.

Finally, create an empty gulpfile.js configuration file within the test folder. This is used to define our tasks.

Step 4: Install Your First Plug-in

Gulp won’t do much on it’s own — we need to install and configure plug-ins to perform specific tasks. First, let’s check the quality of our JavaScript source files using jshint; install the plug-in from the command-line:

npm install gulp-jshint --save-dev

Open your gulpfile.js configuration file in a text editor and add the following JavaScript code:

// include gulp
var gulp = require('gulp'); 

// include plug-ins
var jshint = require('gulp-jshint');

// JS hint task
gulp.task('jshint', function() {
  gulp.src('./src/scripts/*.js')
    .pipe(jshint())
    .pipe(jshint.reporter('default'));
});

This performs the following operations:

  1. Includes Gulp.
  2. The gulp-jshint plug-in is included as an object named jshint.
  3. A new Gulp task named jshint is defined. This pipes all scripts in the src/scripts folder to the jshint object and outputs errors to the console.

Save gulpfile.js and run this task from the command line using:

gulp jshint

You’ll see any errors in the command console. Example output containing an error is shown below.

[gulp] Using file D:\test\gulpfile.js
[gulp] Working directory changed to D:\test
[gulp] Running 'jshint'...
[gulp] Finished 'jshint' in 8.24 ms
D:\test\src\scripts\lib.js: line 2, col 20, Missing semicolon.

1 error

Step 5: Add Other Tasks

Let’s be a little more adventurous. We’ll look for new or changed image files in src/images, compress them and output to build/images. For this, we’ll need to install the gulp-changed and gulp-imagemin plug-ins:

npm install gulp-changed --save-dev
npm install gulp-imagemin --save-dev

Next, add them as dependencies to the top of our gulpfile.js configuration file:

var changed = require('gulp-changed');
var imagemin = require('gulp-imagemin');

And, create a new Gulp task to do the hard work:

// minify new images
gulp.task('imagemin', function() {
  var imgSrc = './src/images/**/*',
      imgDst = './build/images';

  gulp.src(imgSrc)
    .pipe(changed(imgDst))
    .pipe(imagemin())
    .pipe(gulp.dest(imgDst));
});

Save gulpfile.js, then enter the following command:

gulp imagemin

The images are compressed, copied to the build folder, and savings are reported, e.g.

[gulp] Using file D:\test\gulpfile.js
[gulp] Working directory changed to D:\test
[gulp] Running 'imagemin'...
[gulp] Finished 'imagemin' in 5.71 ms
[gulp] gulp-imagemin: ? battery.png (saved 2.7 kB)
[gulp] gulp-imagemin: ? app.png (saved 3.2 kB)
[gulp] gulp-imagemin: ? tick.png (saved 2.8 kB)

Similarly, we can minify all HTML files in the root of src using the gulp-minify-html plug-in:

npm install gulp-minify-html --save-dev 

Then, add a htmlpage task to gulpfile.js:

// include plug-ins
var minifyHTML = require('gulp-minify-html');

// minify new or changed HTML pages
gulp.task('htmlpage', function() {
  var htmlSrc = './src/*.html',
      htmlDst = './build';

  gulp.src(htmlSrc)
    .pipe(changed(htmlDst))
    .pipe(minifyHTML())
    .pipe(gulp.dest(htmlDst));
});

Save gulpfile.js, then test HTML compression using:

gulp htmlpage

Too easy? Let’s build our production JavaScript by concatenating all source files, stripping console and debugger statements, and ripping out whitespace using the plug-ins:

npm install gulp-concat --save-dev 
npm install gulp-strip-debug --save-dev 
npm install gulp-uglify --save-dev

A scripts task can then be added to gulpfile.js:

// include plug-ins
var concat = require('gulp-concat');
var stripDebug = require('gulp-strip-debug');
var uglify = require('gulp-uglify');

// JS concat, strip debugging and minify
gulp.task('scripts', function() {
  gulp.src(['./src/scripts/lib.js','./src/scripts/*.js'])
    .pipe(concat('script.js'))
    .pipe(stripDebug())
    .pipe(uglify())
    .pipe(gulp.dest('./build/scripts/'));
});

This example passes an array of filenames to gulp.src(); I want lib.js to appear at the top of the production file followed by all other JavaScript files in any order. As usual, we can run this task using:

gulp scripts

Finally, let’s complete our operations by concatenating the CSS files, adding any required vendor prefixes, and minifying with the following plug-ins:

npm install gulp-autoprefixer --save-dev 
npm install gulp-minify-css --save-dev 

Update gulpfile.js:

// include plug-ins
var autoprefix = require('gulp-autoprefixer');
var minifyCSS = require('gulp-minify-css');

// CSS concat, auto-prefix and minify
gulp.task('styles', function() {
  gulp.src(['./src/styles/*.css'])
    .pipe(concat('styles.css'))
    .pipe(autoprefix('last 2 versions'))
    .pipe(minifyCSS())
    .pipe(gulp.dest('./build/styles/'));
});

And running the task:

gulp styles

The autoprefixer plug-in is passed a string or array indicating the level of browser support — in this case, we want the current and previous versions of all mainstream browsers. It looks up each property at caniuse.com and adds additional vendor-prefixed properties when necessary. Very clever — I challenge you to do that by hand every time you make a CSS change!

I’ve used a small number of useful plug-ins in these examples, but you can find many more at npmjs.org. Others of interest include:

Step 6: Automate Tasks

Until now we’ve been running one task at a time from the command line. Fortunately, Gulp allows us to create a default task and run any number of dependent sub-tasks. Add this function to the end of gulpfile.js:

// default gulp task
gulp.task('default', ['imagemin', 'htmlpage', 'scripts', 'styles'], function() {
});

Then, type the following command at the command prompt:

gulp

All four tasks are now run in sequence.

But that’s still too much hard work! Gulp can monitor your source files using the watch method, then run an appropriate task when a file change is made. We can update the default task to check our HTML, CSS and JavaScript files:

// default gulp task
gulp.task('default', ['imagemin', 'htmlpage', 'scripts', 'styles'], function() {
  // watch for HTML changes
  gulp.watch('./src/*.html', function() {
    gulp.run('htmlpage');
  });

  // watch for JS changes
  gulp.watch('./src/scripts/*.js', function() {
    gulp.run('jshint', 'scripts');
  });

  // watch for CSS changes
  gulp.watch('./src/styles/*.css', function() {
    gulp.run('styles');
  });
});

When we now run:

gulp

The process will remain active and react to your file changes. You won’t need to type it again — press Ctrl+C to abort monitoring and return to the command line.

Step 7: Profit!

Applying the processes above to a simple website reduced the total weight by more than 50%. You can test your own results using page weight analysis tools or a service such as New Relic which provides a range of sophisticated application performance monitoring tools.

While you’ll need to invest a couple of hours getting to grips with Gulp, the learning curve seems shallower than Grunt, and it’s easy to reuse a configuration file in other projects. I hope you found this tutorial useful and consider Gulp as part of your production process.

Useful links:

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • Craig Buckler

    Thanks Jed. You can duplicate the node_modules folder and your gulpfile.js configuration. However, I suggest you write a batch/shell script with all npm install commands (or a single command) so the latest versions are downloaded when you start a new project.

    Note also you can run “npm update” to update all dependencies at any point.

    • Jed Lehmann

      Thanks Craig, sounds like I’ll have to dive in when I find some time.

  • adimauro

    Just as I start getting the hang of Grunt, I’m now reading about Gulp everywhere! The syntax is nice. It feels more like coding, whereas the Grunt configuration is only slightly better than having to manage XML config files. Gulp syntax reads much nicer.

    Of course, I prefer the Grunt logo…if that counts for anything! But, after going through this article, I think you’ve convinced me to give Gulp a serious try. Thanks.

    Too many tools, too little time.

    • Aurelio De Rosa

      If you’re happy with Grunt stick with it, there’s no sense in just changing tools because of fashion. Otherwise you’ll find yourself *next week* dealing with the next shining tool that we’ll released.

      • adimauro

        Yes, I agree, but, I also like trying out new technologies. There is always something to learn from new approaches. I’m in the process of researching for a new project, but, I certainly wouldn’t suddenly switch tools mid-project!

        You bring up a good point, though. The number of libraries being released seems to be accelerating, especially with JavaScript. Keeping up is almost impossible. At some point you do have to focus on your core group of libraries that fit best with your workflow/programming style and not worry about every new library that comes out.

        • Craig Buckler

          I agree with Aurelio: stick with Grunt if you’re happy using it. Since writing this I’ve also discovered Brunch — yet another task runner.

          Personally, I prefer Gulp but I wouldn’t encourage anyone to switch for the sake of it. Something better will almost certainly appear within a matter of weeks!

  • vsync

    on the HTML tast, why do you need “changed(htmlDst)” if you are already using the gult “WATCH” method?

    • Craig Buckler

      The watch process will fire if it spots any new or updated files. Every file is then processed regardless of whether it’s been done before. Sometimes that’s necessary, e.g. for Sass compilation. Sometimes it’s not, e.g. image processing. The change() pipe only passes files which have changed to ensure they’re not done again.

  • Kurt Meredith

    Craig, thanks for this tutorial on Gulp. I am embarrassed to say I got stymied at step 3 where you start talking about navigating to the “test” directory. Could you explain this a little bit more? Do we have to create our own “test” directory and sub-directories, or is there something that I am missing? I tried creating my own “test” directory structure manually, where I usually create my development files, but then the following steps in the tutorial don’t work. That leads me to think I have missed a step somewhere. I appreciate any pointers. [currently running Linux 10.04; node, npm and gulp are all successfully installed]

    • Craig Buckler

      Hi Kurt. First, create your own directory/folder. In Linux, you use “mkdir test” to create a test folder. You then use “cd test” to move into that folder. I hope that helps?

      • Kurt Meredith

        Thanks, Craig. I must have something set incorrectly. In the terminal, I created and moved to the “test” directory at ~/Development/gulp/test and then ran:

        kurt@kurt-linux:~/Development/gulp/test$ npm install gulp –save-dev

        But instead of the “node_modules” directory and sub-directories installing under the “test” directory, they installed under my home (~/) directory. I need to figure out why the files are not being installed to the directory that I run the command from.

        [SOLUTION]
        I found that, at least in my setup, if the “node_modules” directory does not already exist in the current working directory, then npm will start traveling up the directory heirarchy until it finds one. Once I made a “node_modules” directory within the “test” working directory, the “npm install gulp –save-dev” command worked properly.

        • Craig Buckler

          Ahh. Try running “npm init” in your work folder first. That doesn’t appear to be necessary for everyone, but it won’t do any harm.

  • http://notrobotic.com/ Jason Rhodes

    Craig, nice intro. Just a few things that I’ve been seeing lots of people be confused about:

    gulp.run is out, dependencies are in. A task’s dependencies are passed as its second argument, as you do in the default task in your example above.

    gulp.task("default", ["scripts", "styles"]);

    But later you have your watch tasks use gulp.run, which is deprecated as of version 3.5. In most cases, dependencies are your friend. For example, you could rewrite your watch tasks:

    // default gulp task
    gulp.task("default", ["imagemin", "htmlpage", "scripts", "styles"], function() {
    // watch for HTML changes
    gulp.watch('./src/*.html', ["htmlpage"]);
    // watch for JS changes
    gulp.watch('./src/scripts/*.js', ["jshint", "scripts"]);
    // watch for CSS changes
    gulp.watch('./src/styles/*.css', ["styles"]);
    });

    gulp is all about async. Remember that when you list dependencies, they won’t run in order—gulp tries to run tasks in parallel by default, to make things finish faster. If you need “jshint” to run before “scripts”, don’t depend on ["jshint", "scripts"]. Instead, make the “scripts” task depend on “jshint”, etc.

    Streams are the main way gulp handles all these dependencies, so if you don’t return the stream (or accept and run the callback in the task function), gulp won’t know when a task has finished. To make sure your dependencies run the way you expect, make sure your tasks return the stream, like this:

    // JS concat, strip debugging and minify
    gulp.task('scripts', function() {
    return gulp.src(['./src/scripts/lib.js','./src/scripts/*.js'])
    .pipe(concat('script.js'))
    .pipe(stripDebug())
    .pipe(uglify())
    .pipe(gulp.dest('./build/scripts/'));
    });

    Just a few simple clarifications, hope they help. Happy gulping!

    • Craig Buckler

      Thanks Jason – appreciate the feedback.

      • http://notrobotic.com/ Jason Rhodes

        It’s hard to keep up. I just see so many confused questions come into the gulp issue queue, I figure it’s a good idea to try to get good information out there where people can find it. Thanks for writing up the article.

  • Peter Januarius

    Hi Craig – very well written & easy to follow. Helped a lot.

    Thanks

    Pete…

  • Gemma W.

    Thanks for this tutorial. I’m new to Gulp and you made it so much easier to understand how to do this. I found this much simpler to understand than the other articles I found previously.

    I have this set up, and I’ve even added a couple of other plugins and added them to the default task at the end. Works well. :)