WebAssembly Is Overdue: Thoughts on JavaScript for Large Projects

At Auth0, most of our software is developed using JavaScript. We make heavy use of the language both on the front and the back-end.

In this article, we’ll take a look at JavaScript’s usefulness as a general purpose language and give a brief run down of its development, from conception to the present day. I’ll also interview some senior Auth0 developers on the ups and downs of using JavaScript at scale, and finally look at how WebAssembly has the potential to complete the picture and transform the language into a full-blown development platform.

JavaScript as a General Purpose Language

What may seem obvious to young developers today was not so clear in the past: can JavaScript be considered a general purpose language? I think we can safely agree the answer to this question today is “yes”. But JavaScript is not exactly young: it was born in 1995, more than 20 years ago!

For over 15 years, JavaScript gained little traction outside the web, where it was mainly used for front-end development. Many developers considered JavaScript little more than the necessary tool to realize their dreams of ever more interactive and responsive websites. It should come as no surprise that even today JavaScript has no portable module system across all common browsers (although import/export statements are part of the latest spec). So, in a sense, JavaScript development slowly picked up as more and more developers found ways to expand its use.

Some people would argue that being able to do something does not mean it should be done. When it comes to programming languages, I find this a bit harsh. As developers, we tend to acquire certain tastes and style. Some developers favor classic, procedural languages and some fall in love with the functional paradigm, while others find middle-ground or kitchen-sink languages fit them like a glove. Who’s to say JavaScript, even in its past forms, was not the right tool for them?

A Short Look at JavaScript Progress throughout the Years

JavaScript began its life as a glue language for the web. The creators of Netscape Navigator (a major web browser in the 90s) thought a language that designers and part-time programmers could use would make the web much more dynamic. So in 1995 they brought Brendan Eich on board. Eich’s task was to create a Scheme-like language for the browser. If you’re not familiar with Scheme, it’s a very simple language from the Lisp family. As with all Lisps, Scheme has very little syntax, making it easy to pick up.

However, things were not so smooth. At the same time, Sun Microsystems was pushing for Java to become integrated into web browsers. Competition from Microsoft and their own technologies was not helping either. So, JavaScript had to be developed hastily. What’s more, the rise of Java made Netscape want their new language to act as a complement to it.

Eich was forced to come up with a prototype as soon as possible; some claim it was done in a matter of weeks. The result was a dynamic language with syntax similar to Java but with a very different philosophy. For starters, the object model in this new language was entirely different from the Simula-derived Java object model. This initial prototype of a language was known as Mocha, and later as LiveScript.

LiveScript was quickly renamed JavaScript just as it was launched, for marketing reasons. Java was on the rise, and having “Java” in the name could spark additional interest in the language.

This initial release was the first version of JavaScript and a surprising amount of what is known as JavaScript today was available in it. In particular, the object model—prototype based—and many of the functional aspects of the language—semantics of closures, asynchronous nature of the API—were set in stone. Unfortunately, so were many of the quirks resulting from its rushed development.

This version, although powerful in many aspects, was missing notable features that are helpful when developing ever greater systems. Exceptions are one example.

The next few versions of JavaScript were concerned with making it widely available. One of the first steps taken to achieve this was to make it into a standard. Thus a standardization effort began through ECMA, and later through ISO. ECMAScript, which was the name adopted after standardization, was very similar to the first versions of JavaScript included in Netscape Navigator. It was not until ECMAScript 3 or JavaScript 1.5 in 1999 that most of JavaScript as we know and use it today was finalized. This version included exception handling, instanceof, all common control mechanisms (do/while, switch), eval and most built-in functions and objects (Array, Object, etc.).

A dark period began after that for JavaScript. Competing groups had different ideas for JavaScript’s development. Some advocated for advanced features such as modules, a kind of static typing, and class-based object-oriented programming. Others thought this was too much. A proposal for ECMAScript 4 was made and implementers started integrating some features in their engines. Unfortunately, the community never settled on which features to include. Microsoft was also working on JScript, an implementation of JavaScript with extensions. As a result, ECMAScript 4 was abandoned.

It was not until 2005 that JavaScript development started to pick up. Refinements to ECMAScript 3 were made. Several other features (let, generators, iterators) were developed outside the standard. The turmoil caused by the failed ECMAScript 4 specification settled and in 2009 it was agreed that the refinements to ECMAScript 3 were to be renamed ECMAScript 5. A path for future development was defined and many of the features proposed for version 4 started being reevaluated.

The current version of the standard, ECMAScript 7 (a.k.a 2016) includes some features that were slated for version 4 such as classes and import/export statements. These features are intended to make JavaScript more palatable for medium and large system development. This was the rationale behind ECMAScript 4 after all. But is JavaScript living up to this promise?

Let’s take a look at a not-so-objective rundown of JavaScript features.

Language Features: The Good

Syntactic familiarity

The C family of languages share vast mindshare. C, C++, Java, C# and JavaScript combined probably outnumber all other languages in use. Although it probably is the cause of many of JavaScript quirks, making JavaScript a C-like language in syntax made it simpler for existing developers to pick up. This helps even today, as C-like languages still dominate the development landscape.

An inexperienced developer can easily start writing JavaScript code after taking a look or two at common examples:

function test(a, b, c) {
  a.doStuff(b.property, c);
  return a.property;
}

Asynchronous nature

Perhaps the biggest shock for new developers coming into JavaScript is the way everything is asynchronous by nature. This takes some time getting used to but makes complete sense if you consider how JavaScript was conceived: as a simple way to integrate programmable logic into web-pages. And when it comes to this, two things need to be considered: non-blocking behavior is essential, and shared memory is too complex.

The solution: callbacks and closures.

const consumer = new Consumer();

$.ajax({
  method: "GET",
  url: "http://test.com/resource"
}).done(function(data) {
  consumer.push(data);
});

Arguably, JavaScript saw an uptake for server side development due to the benefits of this approach. Features in the works such as async/await will make asynchronous development even easier.

Functional features and closures

JavaScript’s multi-paradigm approach has paid off. Many languages deeply entrenched in one paradigm, such as Java, have started to implement other paradigms. JavaScript has had this from the beginning. Prototypal inheritance is powerful enough to implement all OOP semantics. Closures allow for functions to be treated as first-class objects and be passed around as such. Objects and Arrays with a convenient notation (JSON) combined with these features make JavaScript inherently powerful.

The following is an example taken from RxJS docs:

const source = getAsyncStockData();

const subscription = source
 .filter(quote => quote.price > 30)
 .map(quote => quote.price)
 .forEach(price => console.log(`Prices higher than $30: ${price}`);

Language Features: The Bad

Quirks

JavaScript was developed in haste, and it shows. For example, automatic semicolon insertion, a feature meant to ease development by non-developers, can generate unexpected results:

function test() {
  functionCall();
  obj.operation();

  // Other code

  return  //<-- semicolon inserted here, returns undefined
    {
      key: "This object should be returned instead"
    }
}

These quirks make JavaScript non-intuitive and can result in lost productivity. Experienced developers know how to avoid these pitfalls, so medium and big systems do require experienced developers, perhaps more when using JavaScript rather than other languages. JavaScript Garden lists a couple of these quirks.

Weak typing and automatic conversions

Although tests are an essential part of JavaScript development, not all tests always catch simple casting mistakes. What’s more, JavaScript performs many implicit casts. Expert developers are well aware of the semantics of these casts and try to avoid them when possible.

An extreme example of how JavaScript casting works can be seen below:

console.log((![]+[])[+!![]]); 
//This prints the character “a”

This is possible due to being able to cast any value to a boolean value. The first implicit cast results in the value “false” being cast to an integer and then indexed to its second value. Crafty, and crazy.

Module system

ECMAScript 6 (2015) has finally defined the syntax of a potential module system. However, no browser currently implements this in a usable way. In other words, even today external module loaders are required.

Modules are essential for proper software development. A standard way to split and reuse code is perhaps one of the most fundamental aspects of it. When it comes to JavaScript Modules, we are still using competing solutions: require (Node.js modules), import/export plus a module loader or transpiler (Babel, System.js, Webpack), or even plain old immediately invoked functions or UMD.

Globals and hoisting

JavaScript variables are always defined at function scope (unless let is used to declare them, a recent addition). This can result in unexpected changes to variables. It is easy to imagine how unexpected changes to variables can be problematic for big-scale development.

function test() {
  if (true) {
    var a = 1;
    console.log(a);
  }

  var a;
  if (a) {
    // This code runs, a === 1
    console.log("Here");
  }
}

As JavaScript was expected to be used by non-developers, it is non-strict with some basic checks. All variables, even when they are not defined, are created in some context. When no context is specified, they are created in the global context. In other words, if for some reason you forget to specify the right context for a variable, it will silently get created and updated in the wrong place.

function test() {
  variable = "test";
}
test();
console.log(window.variable);

Fortunately, stricter checks for globals are available in JavaScript strict mode.

Lack of proper integer types

All numeric variables in JavaScript are of floating point type (except in very specific cases). This is usually enough. Unfortunately, many algorithms expect well-defined integer semantics to be available. It is usually possible to implement these semantics on top of JavaScript numeric types, but this results in sub-optimal code.

For 32-bit integer types, optimal semantics are available using bitwise operators (the only case where 32-bit integers are available in JavaScript). Unfortunately, there is no native alternative for 64-bit integers (which are natively available in many platforms nowadays).

Current versions of JavaScript include typed arrays in part for this reason. These are not enough, though.

Opinions on Developing a Big System Using JavaScript

At Auth0, most of our software is developed using JavaScript. We invested heavily on Node.js early on. So far it has paid off. But some of our most senior developers have many stories from the trenches.

We have asked Damian Schenkelman, Director of Engineering, and Jose Romaniello, Head of Engineering, to share their thoughts on this matter.

Q: What is your opinion of JavaScript as a general purpose language?

D. Schenkelman: I like the language as it has a very small core set of concepts, and closures are a very powerful feature on which you can build on.

There are drawbacks, obviously: implicit type conversions and a weak type system. I find that if you stick to the good parts, JavaScript can be a nice language. Of course, tests are an essential part of development, as well.

J. Romaniello: I think it is fine. You can use it for pretty much everything nowadays but it is not ideal for many scenarios.
The things that at some point seem a benefit to achieve some goal can easily turn against you.

The language itself is quite simple to understand, as well as the platforms where it runs, namely the browser or Node.js. The real power of JavaSacript comes from the ecosystem of tools, libraries, and its vast community.

I think Node.js got its philosophy quite right (maybe from Unix?) small core and vast userland.

Most modules for Node.js expose just a single function doing something very specific and there are modules for everything. These are just well documented small building blocks that developers understand and use to solve their problems.

I think it can’t be attributed to the package manager or the language but this is more like a de-facto way of getting things done. Other technologies have similar tools but instead of having libraries they have all-or-nothing “frameworks” like Spring, WCF, etc.

Q: In your years developing a top performing service using JavaScript, can you think of any stories from the trenches where JavaScript has completely borked things or totally saved the day?

D. Schenkelman: Actually, I find most of the big mistakes we have made were related to missing properties or objects with the wrong type. These are mistakes that would be easily avoided by an implicit checking of types and that require more discipline to write thorough tests in JavaScript. I think progressive typing can help a great deal in these cases. Unfortunately, we have not drafted a new set of guidelines to start doing this today, but this is something we are strongly considering. I think TypeScript is a step in the right direction, in particular when it comes to inter-module contracts. Of course, this is not to say TypeScript should replace tests: not at all, but it could help catch silly mistakes. Linters help a great deal as well.

J. Romaniello: Matias, Eugenio, Iaco and I come from a very different world before Auth0. We were .NET developers for many years. Starting Auth0 with Node.js allowed us to develop at an incredible pace compared to other languages because we had JavaScript in the database (Mongo), JavaScript in the back-end (Node) and JavaScript in the browser. Using a strongly typed language with a schema-based database usually requires writing adapters and mapping from one model to another. In JavaScript, you use “hash-maps” all the way down.

I can’t attribute any specific failure to the language itself. We have made a lot of mistakes, for instance, we learned the hard way that looping over 100k objects to render something blocks the event loop.
Regarding Node.js specifically, we sometimes wish to have more detailed errors. There are cases where you just get a “ECONNRESET” exception without any other detail. Fortunately, the Node.js codebase is easy to understand and it allowed me to fix those things.

Q: If you could pick any language or framework to develop a back-end such as Auth0’s from the ground up nowadays, which language or framework would it be? Would you pick Node.js and JavaScript again?

D. Schenkelman: I think this is not as important as it may seem. I mean, developing a platform, especially when it comes to startups, is so much more than just coding. Code is just the means to achieve your product. As long as the set of tools can be reasonably applied to the problem domain in question, coding is just one more piece of the puzzle. You will achieve results whether you pick Java, JavaScript, C# or many other of the tried and tested platforms.

Engineering must also consider the business side of things. As long as you can be reasonably productive as a team, the language is just not as important as shipping, or keeping up with your client’s demands, or making a profit.

JavaScript, in general, has been easy to pick up for most developers in our team. When you are growing fast, this is very important. In my experience, the big platforms are all good at this. So, it is very hard to say what would have happened had we picked something else, but I also think this is not too important either.

J. Romaniello: The Auth0 back-end is evolving into small services. This allows us to auto-scale on different types of load, gives us improved fault tolerance, better monitoring etc. We are using Node.js but in a different way to when we started. I think I would pick either Node.js again or something like Erlang/Elixir.

---

In general, our most experienced developers think JavaScript has a great ecosystem, and it pays off, even if the language sometimes does not quite fit the problem. But what if we could open that ecosystem to more tools?

Enter WebAssembly

On the back-end, you have a world of choices. Finding the right tool for the job is a non-issue. But when it comes to front-end development or client-side applications you are stuck with JavaScript. And, as we have seen above, JavaScript is a perfectly valid tool for many applications. Its ever bigger use for big systems is a testament to that, but it is disingenuous to think it is the right tool for all cases.

WebAssembly has the potential to change all of this. Imagine the possibility of choosing tried and tested libraries inside your company for your new project. Do you have an internal algorithms library implemented in C? No problem, compile it to WASM and load it in your application. Then develop whatever parts are reasonable in JavaScript. This is the kind of power the web has been missing for years, and it is finally right around the corner. And not just for the front-end. Node.js is expected to allow loading WASM modules as well. In a sense, WebAssembly is the metamorphosis of JavaScript virtual machines from language-centric to generic VMs.

Since the .NET platform was released in 2002, general purpose virtual machines have soared. Java, for example, has become a platform for new and existing languages. Scala and Clojure are perhaps the biggest exponents of this trend. Entirely new platforms have been developed on the premise of the benefits of having a set of tried tools available in combination with the right language for a problem. And JavaScript has become a rich platform.

The last few months in WebAssembly have been exciting: Binaryen, a new compiler infrastructure to generate WASM files has started working; Firefox, Chrome, and Edge have working WebAssembly implementations behind experimental flags; the spec and design documents have grown in size. Even a full-blown, runnable demo with an ASM.js fallback of a Unity example is available for you to try. WebAssembly is right around the corner, but it is still not ready.

Meanwhile, huge applications are being developed in JavaScript out of need or lack of flexibility. The bigger the app, the bigger the chance you will hit the limits: big integer math, SIMD, threading, etc. WebAssembly is the complement the JavaScript ecosystem has been missing for years.

Conclusion

JavaScript is the right tool for many applications. Functional features, syntactic familiarity, its asynchronous nature, a huge number of libraries, and a strong community make it one of the best development platforms out there.

However, lack of flexibility in its integration with other solutions forces JavaScript into places where it is not the right tool for the job. If all you have is a hammer, everything looks like a nail.

WebAssembly will completely change this situation, turning JavaScript into a full-blown development platform. WebAssembly is the final push JavaScript needs, and it can’t come soon enough.