Eliminating async Javascript callbacks by preprocessing

The Catch 22 of AJAX is, for the sake of an easy life, most of the time we want to write “synchronous code” but asynchronous is the only way to avoid some rather nasty usability issues. This means rather than being able to write simple code, as we’d like to, such as;


function doClick() {
    var xmlhttp = new XMLHttpRequest();
    xmlhttp.open("GET","http://example.com/products",false);

    # Execution blocks here, waiting for the response to complete...
    xmlhttp.send(null);
    alert(xmlhttp.responseText);
}

…we’re required instead to handle this via callbacks, the simplest example being…


var xmlhttp = new XMLHttpRequest();

function doClick() {
    xmlhttp.open("GET","http://example.com/products",true);

    // Set the callback
    xmlhttp.onreadystatechange = handleResponse;
    xmlhttp.send(null);
}

function handleResponse() {
    if ( xmlhttp.readyState == 4 ) {
        alert (xmlhttp.responseText);
    }
}

…but that’s now introduced a whole load more potential issues. The callback now relies on the global xmlhttp object being available (and globals for any significant size project are generally evil). And what if the user keeps firing that doClick() function? What about async requests that pause for a coffee break then return unexpected much later (timeouts required)? And that’s just for starters.

Anyway – a couple of interesting projects are working on giving us the best of both worlds – asynchronous requests but (what looks like) blocking code. Both work on the basis of extending Javascript itself, so that what before was a massive effort in manual coding becomes neatly hidden behind a new Javascript operator or keyword.

Narrative Javascript

The first is Narrative Javascript which adds a new “blocking operator” -> so that your code becomes something like;


function doClick() {
    # Note the blocking operator...
    var response = doHttpRequest->("http://example.com/products");
    alert(response);
}

The Narrative JS overview makes a good starting point. Perhaps an advantage of Narrative JS is it’s pure Javascript – although the docs advise preprocessing offline using something like Rhino, you probably could (in theory) preprocess your code on demand (at a potentially significant performance cost) in the browser, the Javascript parser being narcissus (also pure JS). At the very least, being all JS is likely to make people more confident about using it.

jwacs

The second is jwacs – Javascript With Advanced Continuation Support. This actually goes a fair bit further than just being able to simulate blocking code, adding four new keywords (and an import statement). The earlier example (with aide of a bundled jwacs utility API) becomes;


function doClick() {
    var response = JwacsLib.fetchData("GET", "http://example.com/products"))
    alert(response);
}

To see the extended Javascript, you need to look at the above fetchData definition;


  fetchData: function(method, url)
  {
    var http = JwacsLib.getHttpObj();
    var k = function_continuation;

    http.onreadystatechange = function()
    {
      try
      {
        // Report results to the continuation on completion
        if(http.readyState == 4)
        {
          // Check for errors
          if(!(http.status == undefined ||
               http.status == 0 ||
               (http.status >= 200 && http.status < 300)))
          {
            var err = new Error("Server returned " + http.status);
            throw err;
          }

          // No errors, so resume the continuation with the raw results
          http.onreadystatechange = JwacsLib.emptyFunction;
          resume k <- http.responseText;
        }
      }
      catch(e)
      {
        // Errors are thrown as exceptions into the continuation
        http.onreadystatechange = null;
        throw e -> k;
      }
    };

    http.open(method, url);
    http.send(null);
    suspend;
  }


Note the function_continuation, suspend, resume and extended throw: throw e -> k; above. The jwacs preprocessor is written in LISP

So what’s the general feeling here? Would you consider using these?

The very notion of extending Javascript with new syntax may be, to many, offensive. You’ve also introduced some significant dependencies – a later change of plan could lead to significant re-writes (and of course they’re both still very much prototype).

At the same time, writing anything non-trivial in Javascript involving asynchronous processing and callbacks can quickly become a nightmare – why not eliminate human effort with some smart syntax? The general approach here seems good to me.

webtuesday

While I’m here – a quick ad for webtuesday tonight with Patrice talking about his experiences web testing with Selenium (it’s not just “more Java” you know ;) at tilllate’s HQ.

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • Ghandi

    here is something I had done because of the globals and the need for possibly having multiple requests at once. This is pretty basic, but it does the job for my small things.

    
    function request(post, callback) { //Standard XMLHttpRequest
    	if(isMSIE) { // Internet Explorer
    		var requester = new ActiveXObject('Microsoft.XMLHTTP');
    	}
    	else if(isFire) { // Mozilla Firefox
     		var requester = new XMLHttpRequest();
    	}
    	else { // Opera & others
    		var requester = new XMLHttpRequest();
    	}
    
    	requester.onreadystatechange = function() { callback(requester); };
    	requester.open("POST", "./info_req.php");
    	requester.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded; charset=UTF-8');
    	requester.send(post);
    }
    
    function status(requester) { //Basic status method
    	if(requester.readyState == 4) {
    		if(requester.status == 200) {
    			alert(requester.responseText);
    		}
    		else {
    			errors.innerHTML = "" + requester.status + ": " + requester.statusText + "";
    		}
    	}
    	else {
    		//do whatever
    	}
    }
    
  • Ghandi

    Sadly it doesn’t look to nice. Perhaps you can copy and paste into an IDE.

  • Joseph Scott

    Having a clean way to write JavaScript as if things were sync while actually being async would be terrific. Creating more and more layers of triggered events when async calls complete is very ugly.

    I’ve put off features in JavaScript projects for exactly this reason.

  • Pingback: Ajaxian » Eliminating async Javascript callbacks by preprocessing

  • Brent Ashley

    I’ve been grappling with this for years. My first Microsoft Remote Scripting app used synchronous calls, with the attendant browser freezups. XMLHttp will do the same if you ask it to and you’re still not given much control to handle timeouts, error trapping etc. With a single scripting thread, a non-blocking sleep() function is moot, so it’s not been practical to do it in Javascript. I’ll be interested to understand how jwacs solves the problem.

  • Anonymous

    This doesn’t require globals. Why not use closures that have access to the local state? If javascript didn’t have closures then yes this would be a problem, but since it does it’s really not an issue. The first solution with -> operator is the equivalent of using a global because it doesn’t unwind the stack on calls that block. The continuation support is really the only solution to the asynchronous vs. synchronous problem. Coroutines give you the synchronous semantics without the hassle of synchronizing threads between multiple calls.

  • Pingback: Eliminating async Javascript callbacks by preprocessing > Archives > Web 2.0 Stores

  • http://www.phppatterns.com HarryF

    This doesn’t require globals. Why not use closures that have access to the local state? If javascript didn’t have closures then yes this would be a problem, but since it does it’s really not an issue.

    That much I’d worked out. Think there are a two problems there – first that closures are to an extent orthogonal to most programmers way of thinking and specific to Javascript, a few closures in an app that’s otherwise objects and permanent state can be a route to memory leaks.

    The first solution with -> operator is the equivalent of using a global because it doesn’t unwind the stack on calls that block.

    Interesting point I hadn’t considered.

    The continuation support is really the only solution to the asynchronous vs. synchronous problem. Coroutines give you the synchronous semantics without the hassle of synchronizing threads between multiple calls.

    Makes me think Javascript 2.0 has missed the point.

  • Chris Double

    I use Narrative Javascript in the manner you mention, running the comiler from the browser, transforming the code on the fly. I even allow embedding of Narrative Javascript in script tags with a type of “text/x-njs”, and the code looks for those, transforms them and evals them. It works quite well.

    Another thing I built on top of it is a lightweight threading system with erlang style communication primitives. This lets you run independant processes on the browser. You do need to call a yield type function to release the timeslice (or it does it itself when using XHR, etc). More here if interested:

    http://www.bluishcoder.co.nz/2006/06/more-concurrency-in-narrative.html

  • Kris Zyp

    I think Chris and I are the the two biggest Narrative JS users right now. Chris has taken the approach of compiling as the scripts are loaded into the browser. On the otherhand I am developing a web content management system, where functions can be attached to objects within in the system and the functions are compiled using the (modified) Narrative JS compiler before they are entered into the WCM’s database and then they are ready for use whenever a user makes use of their functions as preprocessed code. On of the advantages of this approach is the code is already processed before it is needed at runtime. My system is not ready for demos yet, but it will be soon at http://www.authenteo.com

  • kyberfabrikken

    Because of lexical scope, globals are really a non-issue in javascript. You can always wrap the code in a scope.
    It has already been mentioned, but a closure would solve the problem at hand. There are minor issues with memory leaks, but ultimately that’s just shortcommings of the implementation of a specific interpreter. I also think that the actual consequences of the problem are seriously overrated. A few bytes of memory are lost, but in the big picture it’s rather negligible. Most leaked memory will be returned when the page is navigated away from, and even ajax’ed pages are not going to be open on users machines for days.

    closures are to an extent orthogonal to most programmers way of thinking and specific to Javascript

    That may be true, but I also think it’s fair to expect the programmers to understand the language they are using.

  • chubbard

    Closures I think is the easiest way to deal with the problem at hand. Memory leaks are something you have to consider when you do this sort of thing, but I think that’s something that can be overcome with education. The draw back to closures is that the end user has to use them which means the above mentioned education is important.

    Continuations is a much hard thing to understand, and if someone can’t understand closures then they don’t have a prayer to understand continutations. Primarily because no mainstream language has included them in a very long time. Continuations are really forms of coroutines which left the computer science world in the 60s/70s when processes were deemed the best form to express concurrency. Unfortunately, threads/processes really only work nicely for server side concurrency, not for UI concurrency, or event driven concurrency. The big plus to continuations is that you can encapsulate all of that advanced logic inside a framework and offer it to the end programmer as magic. Downside is when it breaks or they have to debug it you run into the problem mentioned above.

    I realy haven’t looked at the Javascript 2.0 spec, but if they’re planning on adding threads I’d be very afraid. Threads and UIs don’t mix as mentioned above. Adding continuations would be a much nicer addition over threads. Overall, I’m in favor of Javascript adding continuations to the language. But, closures are the next best thing when you don’t have that power. The cool thing is that closures are almost equivalent to coroutines without the overhead of a full stack.

  • chubbard

    Oh and the other big drawback to closures is that you can’t encapsulate them very easily. So if I have a routine that does some AJAX with the server and then executes some code on the client like update some field. I can’t encapsulate that in a class and expect to reuse it. Say these a member methods of a class called ProductsTable:


    function refresh() {
    var xmlhttp = new XMLHttpRequest();
    xmlhttp.open("GET","http://example.com/products",true);
    // Set the callback
    xmlhttp.onreadystatechange = function() {
    handleResponse(xmlhttp);
    };
    xmlhttp.send(null);
    }

    function handleResponse(xmlhttp) {
    if ( xmlhttp.readyState == 4 ) {
    $('products').innerHTML = xmlhttp.responseText;
    }
    }

    I can’t add more functionality after the AJAX call. The closure is closed to modifications from the outside. So I can’t add new behavior after to the handleResponse method. Say I wanted to reuse this:


    var products = new ProductsTable();

    function doClick() {
    products.refresh(); // method from above.
    updateTotal();
    }

    function updateTotal() {
    $('total').innerHTML = '$' + products.total;
    }

    This has serious impact on reuse and code structure. You’d like the updateTotal method to occur after the call the back end, but actually this won’t happen because the closure inside the products object will get the callback. The updateTotal method will run before the call from the server returns, or worse at the same time.

    This is where contiuations really shine over closures because it allows us to use the language mechanisms for reuse without have to change the structure of our programs. All of our reuse constructs in languages are based on synchronous coding styles.

  • kyberfabrikken

    You’d like the updateTotal method to occur after the call the back end, but actually this won’t happen because the closure inside the products object will get the callback. The updateTotal method will run before the call from the server returns, or worse at the same time.

    You should have a look at Mochikit’s Deferred api, which offers a convenient way to chain async events together.
    Implementing your example could look like this :


    products = {};
    products.refresh = function() {
    var request = getXMLHttpRequest("GET", "http://example.com/products");
    var d = sendXMLHttpRequest(request);
    d.addCallback(bind('handleResponse', this));
    return d;
    }

    products.handleResponse = function(xmlhttp) {
    $('products').innerHTML = xmlhttp.responseText;
    }

    function updateTotal() {
    $('total').innerHTML = '$' + products.total;
    }

    function doClick() {
    var d = products.refresh();
    d.addCallback(updateTotal);
    return d;
    }

  • Pingback: brentashley » Blog Archive » Simplicity begets Stability

  • Pingback: MondoBlog » Blog » Fast AJAX Links Collection - Num 8

  • Pingback: Eliminating async Javascript callbacks by preprocessing - News

  • Pingback: Fashion Magazine Online

  • http://givotov.ru jakjarlyfreld

    Автор талантлив

  • http://pogzna.ru SeneourermKem

    Ура!, тот кто писал убого опубликовал!

  • http://obostol.ru Galthiele

    В принципе, афтар зачетно написал.

  • http://gamfle.ru joubjermomb

    Однако, хозяин сайта недурно накреативил!

  • http://velreki.ru Zeksbomeott

    Писака следит за сайтом

  • http://flashigr.org InfonsoloJulk

    Аноним давай зачетку

  • http://pasdiab.ru PeecePhinna

    Камрад прострели себя коленку

  • http://sadnadache.ru Prorceanown

    Думаю, неплохая статья

  • http://cardioset.ru/ Aleksander

    Что то новенькое, пишите есче очень нравится.

  • http://perupl.ru GagoEssesttal

    Чего и следовало ожидать, хозяин сайта голимо написал!