Web
Article

Supercharge Continuous Delivery in Jenkins with Workflow

By Peter Nijssen

This article was sponsored by CloudBees. Thank you for supporting the sponsors who make SitePoint possible!

Recently, I wrote a series of articles about Jenkins, an easy-to-use continuous integration (CI) technology for enterprise IT. In these articles, we took a quick look at how we can assure the quality of our PHP projects using the tool. In this article, we’ll dive into the Jenkins plugin “Workflow”, created and maintained by both CloudBees and members of the Jenkins community. This open source plugin builds upon the industry standard CI technology and expands it to a full continuous delivery (CD) platform with sophisticated pipeline management.

As enterprises scale up their use of Jenkins and bring more and more workflows into their systems, automating them can become a challenge. The Workflow plugin helps to build and automate complex workflows. By scripting, you can mix multiple actions like parameterized triggers, copied artifacts, promoted builds and conditional build steps. Through the Workflow plugin, it becomes relatively easy to write such a script.

Installing and creating a workflow

Before you start, you might want to make sure you have basic knowledge of Jenkins by reading my previous articles. If you haven’t done so, I suggest keeping those articles open as a reference.

Let’s get started by installing the Workflow plugin to get a good view on this particular plugin. Within the plugin page, search for Workflow: Aggregator and install it. All dependencies will be installed automatically for you. When done, make sure you restart Jenkins.

Now it’s time to create a new job by clicking new item in the left menu. Fill in the form and make sure you pick workflow as your type. When done, submit the form and head straight to the configuration page for this job.

workflow_1

The configuration looks very basic from the outside, but it’s very powerful. The most important part is the script field, in which you’ll script your workflow. Scripting is done in the groovy language. Let’s start with the old fashioned helloworld example, so we get the very basics of this mighty plugin. Add the following content to your script field and click save.

echo 'Hello SitePoint!'

If you start a build and you take a look at the console output, you’ll see the following content.

Started by user Peter Nijssen
Running: Print Message
Hello SitePoint!
Running: End of Workflow
Finished: SUCCESS

Of course, it’s nice to see we can echo a message to the console, but we want to script our whole build process. Let’s say we want to build our project Jumph, from the previous series of articles.

If you’re new to the groovy language like I am, you might wonder how to start. There’s a snippet generator for the plugin, which will help you start quite fast. Below the script field, you’ll see a snippet generator.
Mark the checkbox in front of snippet generator and select build from job. Fill in Jumph and click generate groovy. You will get the following output, which you can insert in your script text area.

build 'Jumph'

workflow_2

If you now run a build, you’ll notice Jumph is actually being kicked off. Nice! However, we could have started the build process of the project ourselves. This is a simple example of what is called a step in Workflow terminology. A step can also be just archiving the artifacts of your project, or sending an email when it’s done. Let’s take a deep dive into Workflow, to get a better understanding of it’s power.

Let’s say we want to build the project for deployment. We want to create a tar.gzfile which later can be used for deployment. With Workflow, we can script this process.

def src = 'https://github.com/peternijssen/Jumph.git'

stage 'Build'
node {
    git url: src
    sh 'export SYMFONY_ENV=prod && composer install --no-dev --optimize-autoloader'
    sh 'bower install'
}

In the above script, we first define our URL for the Git repository. Within the current node, we pull in this Git repository and run composer install while we set the Symfony environment to production. Next we run Bower to install our front-end dependencies. If we were to run this script, our project would be prepared to be deployed.

Perhaps you have a tool like Gulp or Grunt running along to prepare your front-end. You can easily expand this script to run multiple scripts to prepare your application for deployment.

The next step we want to do is pack the application in a tar.gz file and move it to our public root, so it can be downloaded.

def timestamp = new Date().format(“yyyyMMddHHmmss”)
def fileName = ‘jumph-’ + timestamp + ‘.tar.gz’

stage 'Package', concurrency: 1
node {
    sh 'tar -zcvf ' + fileName + ' app/ bin/ src/ vendor/ web/'
    sh 'cp ' + fileName + ' /srv/builds/'
   mail body: 'A new build is awaiting: ' + fileName, subject: 'New build available', to: 'example@sitepoint.com'
}

We’re also sending an email and making sure our file name includes a timestamp, so we’ll always be aware of when the file was actually created.

If we think we’re going to use this part more than once, we can wrap it in a ‘function’.

node {
    packageApp(fileName)
   mail body: ‘A new build is awaiting: ’ + fileName, subject: ‘New build available’, to: ‘example@sitepoint.com’
}

/**
 * Package the app in tar.gz file
 *
 * @param fileName name of the file
 */
def packageApp(fileName) {
    sh ‘tar -zcvf ’ + fileName + ’ app/ bin/ src/ vendor/ web/’
    sh ‘cp ’ + fileName + ’ /srv/builds/’
}

In the end, your complete script will look like this:

def src = ‘<a href="https://github.com/peternijssen/Jumph.git">https://github.com/peternijssen/Jumph.git</a>’
def timestamp = new Date().format(“yyyyMMddHHmmss”)
def fileName = ‘jumph-’ + timestamp + ‘.tar.gz’

<p>stage ‘Build’
node {
    git url: src
    sh ‘export SYMFONY_ENV=prod &amp;&amp; composer install –no-dev –optimize-autoloader’
    sh ‘bower install’
}

stage ‘Package’, concurrency: 1
node {
    packageApp(fileName)
    mail body: ‘A new build is awaiting: ’ + fileName, subject: ‘New build available’, to: ‘example@sitepoint.com’
}

/* FUNCTIONS */

/**
 * Package the app in tar.gz file
 * 
 * @param fileName name of the file
 */
def packageApp(fileName) {
    sh ‘tar -zcvf ’ + fileName + ’ app/ bin/ src/ vendor/ web/’
    sh ‘cp ’ + fileName + ’ /srv/builds/’
}

We still have a pretty simple script compared with what we can actually do with Workflow. For example, we could also start by running some simple tests to make sure our build is in the correct shape before we actually package the app. Instead of packaging, we could also deploy the app straight to our staging environment. After checking our staging environment, we could also let Workflow ask you for permission to deploy it to the production environment.

If you’re interested in these kinds of flows, you can check out the Java examples which you can find here and here.

Perhaps you also noticed we divided the script into two stages. A stage named build and a stage named package. By setting the concurrency to one on the package stage, we make sure that only the most recent build, that passed all the previous stages, will be performed. Next to that, a stage will also divide the script into sections.

As your whole build process grows, you can imagine it can take a lot of time for it to be completed. If your server then experiences problems or gets shut down, your whole process could get disrupted. However, Workflow is suspendable. This means Jenkins will remember at which part of the script it stopped, and will continue executing as soon as it’s back up. Through Workflow, you can make sure your process keeps on running where it left just before the shutdown.

Conclusion

As seen in the article, Workflow is a major step in the evolution of Jenkins which takes it from CI to full dev-to-production continuous delivery, which is the key enabling technology in DevOps transformations. The plugin allows you to write a simple script to create a continuous delivery pipeline from a simple job.

If you’re interested in what Workflow can do, check out this tutorial to get a better idea. If you’re an author of a Jenkins plugin, you might be interested in getting it to work in combination with Workflow through DSL.

If you need an even more advanced package then the Workflow plugin, you might want to take a look at the CloudBees Jenkins Enterprise solution on the CloudBees website.

How would Workflow improve your build process? Have you given it a go yet?

Recommended

Learn Coding Online
Learn Web Development

Start learning web development and design for free with SitePoint Premium!

Instant Website Review

Use Woorank to analyze and optimize your website to improve your website to improve your ranking!

Run a review to see how your site can improve across 70+ metrics!

Get the latest in Front-end, once a week, for free.