JavaScript
Article

Measuring JavaScript Functions’ Performance

By Peter Bengtsson

Performance has always played a crucial part in software. On the web, performance is even more important as our users can easily change website and visit one of our competitors if we offer them slow pages. As professional web developers, we have to take this issue into account. A lot of old web performance optimization best practices, such as minimizing requests, using a CDN and not writing rendering blocking code, still apply today. However, as more and more web apps are using JavaScript, it’s important to verify that our code is fast.

Suppose that you have a working function but you suspect it’s not as fast as it could be, and you have a plan to improve it. How do you prove this assumption? What’s the best practice for testing the performance of JavaScript functions today? Generally, the best way to achieve this task is to use the built-in performance.now() function and measure the time before and after your function executes.

In this article we’ll discuss how to measure code execution time and techniques to avoid some common pitfalls.

Performance.now()

The High Resolution Time API offers a function, named now() that returns a DOMHighResTimeStamp object. It’s a floating point number that reflects the current time in milliseconds accurate to a thousandth of a millisecond. Individually, the number doesn’t add much value to your analysis, but a difference between two such numbers gives an accurate description of how much time has passed.

In addition to the fact that it is more accurate than the built-in Date object, it’s also “monotonic”. That means, in simple terms, that it’s not affected by the system (e.g. your laptop OS) periodically correcting the system time. In even simpler terms, defining two instances of Date and calculating the difference isn’t representative of the time that has passed.

The mathematical definition of “monotonic” is (of a function or quantity) varying in such a way that it either never decreases or never increases.

Another way of explaining it, is by trying to imagine using it around the times of the year when the clocks go forward or go back. For example, when the clocks in your country all agree to skip an hour for the sake of maximizing daytime sunshine. If you were to make a Date instance before clocks go back an hour, and another Date instance afterwards, looking at the difference it would say something like “1 hour and 3 seconds and 123 milliseconds”. With two instances of performance.now() the difference would be “3 seconds 123 milliseconds and 456789 thousands of a millisecond”.

In this section, I won’t cover this API in detail. So if you want to learn more about it and see some example of its use, I suggest you to read the article Discovering the High Resolution Time API.

Now that you know what the High Resolution Time API is and how to use it, let’s delve into some potential pitfalls. But before doing so, let’s define a function called makeHash() that we’ll use for the remainder of the article.

function makeHash(source) {
  var hash = 0;
  if (source.length === 0) return hash;
  for (var i = 0; i < source.length; i++) {
    var char = source.charCodeAt(i);
    hash = ((hash<<5)-hash)+char;
    hash = hash & hash; // Convert to 32bit integer
  }
  return hash;
}

The execution of such function can be measured as shown below:

var t0 = performance.now();
var result = makeHash('Peter');
var t1 = performance.now();
console.log('Took', (t1 - t0).toFixed(4), 'milliseconds to generate:', result);

If you run this code in a browser, you should see something like this:

Took 0.2730 milliseconds to generate: 77005292

live demo of this code is shown below:

See the Pen YXmdNJ by SitePoint (@SitePoint) on CodePen.

With this example in mind, let’s start our discussion.

Pitfall #1 – Accidentally Measuring Unimportant Things

In the example above, you can note that the only thing that we do between one performance.now() and the other is calling the function makeHash() and assigning its value to a variable result. This gives us the time it takes to execute that function and nothing else. This measurement could also be made as detailed below:

var t0 = performance.now();
console.log(makeHash('Peter'));  // bad idea!
var t1 = performance.now();
console.log('Took', (t1 - t0).toFixed(4), 'milliseconds');

A live demo of this snippet is shown below:

See the Pen PqMXWv by SitePoint (@SitePoint) on CodePen.

But in this case, we would be measuring the time it takes to call the function makeHash('Peter') and how long it takes to send and print that output on the console. We don’t know how long each of those two operations took. You only know the combined time. Also, the time it takes to send and print the output will vary greatly depending on the browser and even on what’s going on in it at that time.

Perhaps you’re perfectly aware that console.log is unpredictably slow. But it would be equally wrong to execute more than one function, even if each function does not involve any I/O. For example:

var t0 = performance.now();
var name = 'Peter';
var result = makeHash(name.toLowerCase()).toString();
var t1 = performance.now();
console.log('Took', (t1 - t0).toFixed(4), 'milliseconds to generate:', result);

Again, we won’t know how the execution time was distributed. Was it the variable assignment, the toLowerCase() call, or the toString() call?

Pitfall #2 – Measuring only Once

Another common mistake is to make just one measurement, summarize the time taken and draw conclusions based on that. It’s likely to be totally different at different times. The execution time greatly depends on various factors:

  • Time for the compiler to warm up (e.g. time to compile the code into byte code)
  • The main thread being busy doing other things we didn’t realize were going on
  • Your computer’s CPU(s) being busy with something that slows down your whole browser

An incremental improvement is to execute the function repeatedly, like this:

var t0 = performance.now();
for (var i = 0; i < 10; i++) {
  makeHash('Peter');
}
var t1 = performance.now();
console.log('Took', ((t1 - t0) / 10).toFixed(4), 'milliseconds to generate');

A live demo of this example is shown below:

See the Pen Qbezpj by SitePoint (@SitePoint) on CodePen.

The risk with this approach is that our browser’s JavaScript engine might make sub-optimizations which means that the second time the function is called with the same input, it can benefit from remembering the first output and simply use that again. To solve this issue, you can use many different input strings instead of repeatedly sending in the same input string (e.g. 'Peter'). Obviously, the problem with testing with different inputs is that naturally the function we’re measuring takes different amounts of time. Perhaps some of the inputs cause longer execution time than others.

Pitfall #3 – Relying too Much on the Average

In the last section we learned that it’s a good practice to run something repeatedly, ideally with different inputs. However, we have to remember that the problem with different inputs is that the execution might take much longer than all the other inputs. So let’s take a step back and send in the same input. Suppose that we send in the same input ten times and for each, print out how long that took. The output might look something like this:

Took 0.2730 milliseconds to generate: 77005292
Took 0.0234 milliseconds to generate: 77005292
Took 0.0200 milliseconds to generate: 77005292
Took 0.0281 milliseconds to generate: 77005292
Took 0.0162 milliseconds to generate: 77005292
Took 0.0245 milliseconds to generate: 77005292
Took 0.0677 milliseconds to generate: 77005292
Took 0.0289 milliseconds to generate: 77005292
Took 0.0240 milliseconds to generate: 77005292
Took 0.0311 milliseconds to generate: 77005292

Note how the very first time, the number is totally different from the other nine times. Most likely, that’s because the JavaScript engine in our browser makes some sub-optimizations and needs some warm-up. There’s little we can do to avoid that but there are some good remedies we can consider to prevent a faulty conclusion.

One way is to calculate the average of the last nine times. Another more practical way is to collect all results and calculate a median. Basically, it’s all the results lined up, sorted in order and picking the middle one. This is where performance.now() is so useful, because you get a number you can do whatever with.

Let’s try again but this time we’ll use a median function:

var numbers = [];
for (var i=0; i < 10; i++) {
  var t0 = performance.now();
  makeHash('Peter');
  var t1 = performance.now();
  numbers.push(t1 - t0);
}

function median(sequence) {
  sequence.sort();  // note that direction doesn't matter
  return sequence[Math.ceil(sequence.length / 2)];
}

console.log('Median time', median(numbers).toFixed(4), 'milliseconds');

Pitfall #4 – Comparing Functions in a Predictable Order

We’ve understood that it’s always a good idea to measure something many times and take the average. Moreover, the last example taught us that it’s preferable to use the median instead of the average.

Now, realistically, a good use of measuring function execution time is to learn which of several functions is faster. Suppose we have two functions that take the same type of input and yield the same result but internally they work differently.

Let’s say we want to have a function that returns true or false if a certain string is in an array of other strings, but does this case insensitively. In other words we can’t use Array.prototype.indexOf because it’s not case insensitive. Here’s one such implementation:

function isIn(haystack, needle) {
  var found = false;
  haystack.forEach(function(element) {
    if (element.toLowerCase() === needle.toLowerCase()) {
      found = true;
    }
  });
  return found;
}

console.log(isIn(['a','b','c'], 'B'));  // true
console.log(isIn(['a','b','c'], 'd'));  // false

Immediately we notice that this can be improved because the haystack.forEach loop always goes through all the elements even if we have an early match. Let’s try to write a better version using a good old for loop.

function isIn(haystack, needle) {
  for (var i = 0, len = haystack.length; i < len; i++) {
    if (haystack[i].toLowerCase() === needle.toLowerCase()) {
      return true;
    }
  }
  return false;
}

console.log(isIn(['a','b','c'], 'B'));  // true
console.log(isIn(['a','b','c'], 'd'));  // false

Now let’s see which one is the fastest. We do this by running each function 10 times and collecting all the measurements:

function isIn1(haystack, needle) {
  var found = false;
  haystack.forEach(function(element) {
    if (element.toLowerCase() === needle.toLowerCase()) {
      found = true;
    }
  });
  return found;
}

function isIn2(haystack, needle) {
  for (var i = 0, len = haystack.length; i < len; i++) {
    if (haystack[i].toLowerCase() === needle.toLowerCase()) {
      return true;
    }
  }
  return false;
}

console.log(isIn1(['a','b','c'], 'B'));  // true
console.log(isIn1(['a','b','c'], 'd'));  // false
console.log(isIn2(['a','b','c'], 'B'));  // true
console.log(isIn2(['a','b','c'], 'd'));  // false

function median(sequence) {
  sequence.sort();  // note that direction doesn't matter
  return sequence[Math.ceil(sequence.length / 2)];
}

function measureFunction(func) {
  var letters = 'a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z'.split(',');
  var numbers = [];
  for (var i = 0; i < letters.length; i++) {
    var t0 = performance.now();
    func(letters, letters[i]);
    var t1 = performance.now();
    numbers.push(t1 - t0);
  }
  console.log(func.name, 'took', median(numbers).toFixed(4));
}

measureFunction(isIn1);
measureFunction(isIn2);

We run that and get following output:

true
false
true
false
isIn1 took 0.0050
isIn2 took 0.0150

A live demo of this example is shown below:

See the Pen YXmdZJ by SitePoint (@SitePoint) on CodePen.

What the heck has just happened? The first function was three times faster. That was not supposed to happen!

The explanation is simple but subtle. The first function which uses haystack.forEach benefits from some low-level optimizations in the browser’s JavaScript engine that we don’t get when we use an array index technique. It proves our point: you never know until you measure it!

Conclusions

In our attempt to demonstrate how to use performance.now() to get an accurate execution time in JavaScript, we stumbled across a benchmarking scenario where our intuition turned out to be quite the opposite of what our empirical results conclude. The point is that, if you want to write faster web apps your JavaScript code needs to be optimized. Because computers are (almost) living breathing things, they are unpredictable and surprising. The most reliable way to know that our code improvements yield faster execution, is to measure and compare.

The other reason we never know which code is faster, if we have multiple ways of doing the same thing, is because context matters. In the previous section we perform a case insensitive string search looking for one string among 26 other strings. It’s likely that the conclusion would be totally different if we instead had to look for one string among 100,000 other strings.

The list above isn’t exhaustive as there are more pitfalls to be aware of. For example, measuring unrealistic scenarios or only measuring on one JavaScript engine. But the sure thing is that a great asset for JavaScript developers who want to write faster and better web apps is performance.now(). Last but not least, remember that measuring execution time only yields one dimension of “better code”. There’s also memory and code complexity considerations to bear in mind.

What about you? Have you ever used this function to test your code’s performance? If not, how do you proceed in this stage? Please share your thoughts in the comments below. Let’s start a discussion!

  • http://careersreport.com Margaret Lan

    I will show excellent internet job opportunity… three-five hrs of work /a day… Payment at the end of each week… Bonuses…Payment of 6-9 thousand dollars /a month… Merely several hrs of spare time, desktop or laptop, most basic knowing of web$ and dependable internet connection is what is needed…Get more information by visiting my page

  • http://eliperelman.com Eli Perelman

    Another good API for measuring performance would be the User Timing APIs. You can make many calls to `performance.mark(‘name’)` and `performance.measure(‘markA’, ‘markB’)`, and then process all the results later via `performance.getEntries`.

    • http://CareersReport.com peggy Larson2

      I have to share$ this great internet freelancing opportunity… 3 to 5 hours of work /a day… Payment each week… Performance depending bonuses…Payscale of $6k-$9k /month… Just few hours of free time, desktop or laptop, most basic understanding of web and trusted internet connection is what is required…See more on my disqus–page

  • nobo

    If you really need to measure the performance of your function, why not assign it on a single js file and run it on a single html template, in chrome debugger, open the network tab and refresh the page. The console.log will also taken into account so remove it and append the output value to actual DOM, you actually measuring the performance of your function

    • Chris Stephens

      You could but that seems like more work. Also, I would prefer to run the function in the context of my application and its data. Lets use the foreach vs the for loop example. You would think the for loop would be faster but it is not in the scenario used in the article. Now, will the foreach always be faster? What if I have to iterate over 1,000 or 10,000 different objects? The foreach would be forced, unless hacked, to iterate over every index. The for loop could break early. Seems to me in order to get the best sense of a functions performance on your site you need to take into account the data of your site. I could be wrong though. Great article.

  • http://mrale.ph/ Vyacheslav Egorov

    The explanation is simple but subtle. The first function which uses haystack.forEach benefits from some low-level optimizations in the browser’s JavaScript engine that we don’t get when we use an array index technique. It proves our point: you never know until you measure it!

    This explanation is too subtle — it really lacks details on which optimizations magically happened to a function with forEach to make it faster and thus must always raise suspicion. Function with forEach is actually harder to optimize than a for-loop – in a sense turning forEach+closure into a loop is a removal of abstraction, essentially a holy grail of compiler engineering (conversely: less abstraction means simpler to optimize), but I digress.

    A more obvious explanation is that something went wrong with an experiment… And indeed it did: if you put a debugger statement into for loop you will discover that CodePen instruments loops with preemption checks (to prevent infinite loops from killing UI, I guess). These checks look like this:

    if (window.CP.shouldStopExecution(1)) {
    break;
    }

    forEach is obviously not instrumented in the same way, because its implementation hides inside the VM.

    If you hide your code from CodePen instrumentation by rewriting it like this:

    var isIn2 = new Function(“return function isIn2(haystack, needle) {
    for (var i = 0, len = haystack.length; i < len; i++) {
    if (haystack[i].toLowerCase() === needle.toLowerCase()) {
    return true;
    }
    }
    return false;
    }")();

    Then suddenly the numbers change and isIn2 becomes significantly faster and faster than a forEach – as it quite obviously should.

    This proves one very important point: measuring does not generate knowledge. You never know, until you actually know.

    That’s why it is utterly important to doubt every single number you get and don’t accept simple-subtle explanations – until things are actually explained down to the last minuscule detail.

    • http://www.peterbe.com Peter Bengtsson

      Wow! That’s a really interesting comment. I didn’t know that CodePen got so involved. I knew the forEach is heavily optimized in browsers but this CodePen secure-execution hack changes a lot.

      Regarding “measuring does not generate knowledge”, I’m not convinced. Ideally you should have knowledge before you measure but oftentimes the intricacies are too complex that theoretical knowledge doesn’t cut it.

      For example, the mathematics that goes into designing the curvature of an airplane wing is near perfect in the field of fluid dynamcs, but in the field of physical airplane wing production they use numerical analysis from actually bombarding wings with fluids and measure, measure and measure.

    • Kenneth Davila

      Very insightful!!

  • Marsup

    FWIW you’re using Array.sort() w/o a compare function, it’s known to sort elements as strings, you might end up with the wrong order.

    • http://www.peterbe.com Peter Bengtsson

      How so? That’s interesting. Do you know of an example where it goes wrong?

      • Marsup

        Try [1,2,3,10].sort(). To sort numbers you need to do [1,2,3,10].sort(function (a, b) { return a – b; }).

        • http://www.peterbe.com Peter Bengtsson

          THANK YOU!

Recommended

Learn Coding Online
Learn Web Development

Start learning web development and design for free with SitePoint Premium!

Get the latest in JavaScript, once a week, for free.