Crouching Javascript, Hidden PHP [2]

Continuing yesterday’s rant, this installment should (hopefully) get a bit more juicy, if I can work out where it’s going.

First up, already some good feedback to the last part (thanks!), most to-the-point being Alan;

I can tell you having worked with SOAP, various RPC layers, and simple GET/POST / XML parses, that most of the time RPC tends to suck big time, It’s great for the first 3 test calls, but as a project grows, there is a point where it doesnt work as expected. You then enter one of the worst nightmares in hell to debug.

When javascript (especially if you are risking life and limb trying to get a MS implementation to work) comes in, it’s alot easier to stick with the basics.. KISS / POST/GET/ strings and XML as needed..

Also interesting is Danne Lundqvist’s similar but alternative approach to what Javascript Serializer does with script src attributes; manipulating the attribute using the DOM which he describes here.

What’s up with Javascript

Returning to the thought from last time: “It’s 2004. Someone must have solved this once-and-for-all by now.” (the problem being how to we pull fresh data from the server into a page that’s already loaded), think it’s worth trying to understand (guess) why “things” haven’t progressed further.

Think the starting point there is asking “what’s up with Javascript?” which really is a puzzle.

If you really take a close look at the ECMAScript spec (long story short: ECMAScript == Javascript) and do some serious delving with a decent implementation like Mozilla’s you might come to a stunning conclusion – as a dynamic language, the design is excellent.

Although Javascript is generally applied only to the very specific purpose of browser scripting (so there’s a ton of problems it’s design has never really had to incorporate), you could reasonably ask why Perl or PHP aren’t built so cleanly. Try this in PHP, for example;


try {
eval( someJsInThisVar );
} catch ( e ) {
alert ( "Syntax error "+e+" in "+someJsInThisVar);
}

…or, as I was talking about here, attaching new functionality to fundamental objects like;


String.prototype.toPHP=function() {
var s = this;
return 's:'+s.length+':"'+s+'";';
};

So if we say Javascript is actually a pretty good programming language, why isn’t put to more extensive use. Today it still seems relegated to adding the occasional simple gimmick to a web page, while being dubbed as the worst invention ever by members of the W3 committee (nice response to that here BTW).

Douglas Crockford pretty much sums it up in The World’s Most Misunderstood Programming Language. I’d also add XML parsing to the causes Douglas mentions – DOM often isn’t the easiest way to do things – Javascript needs a native SAX parser (although you can find a pure Javascript implementation here), a pull parser and something like Simple XML IMO.

Ultimately Javascript uptake boils down to developer good sense – it was, in the past, too easy to waste countless hours writing and debugging Javascript, across multiple browsers. Hours which could have been avoided by shifting a more to the server side, where you have an environment you could trust. Only a few have been bold (or insane) enough to do heavy loading with Javascript.

I say in the past because I think Mozilla has changed that – we’ve now got an implementation we can trust plus tools like Venkman to help out (side note – these days I find unit testing eliminates much of the need for debugging – jsunit seems to be about the best choice for Javascript).

Today, if you write for Mozilla first, Microsoft’s recent JScript implementations turn out to be close enough to only need minor tweaking to get stuff right. There’s the occasional aaaarrggh! like this;


Foo.prototype = {
x: 1,
y: 2,
}

IE (last time I looked) doesn’t like that second comma (had to get if off my chest).

Hooking up the Server and Client Side

My fundamental view of the client side has always been “I rather do this server side”. Part of my motivation there comes down the points Douglas Crockford makes; a historical mistrust of Javascript. But that’s begun to change to Mozilla.

But there’s another, more critical point which constains doing really funky stuff with Javascript, something I haven’t seen discussed in detail.

If I’m building a dynamic application which is primarily server-side, that’s already enough complexity to deal with – adding another layer of logic in Javascript, on the client side, is asking for nightmares (as Alan points out).

The big issue IMO is breakability – if I make changes to the server side, it’s too easy to break the client side inadvertently. That’s because I’ll typically be evolving the two more or less independently.

Although there’s obviously a relationship between them, there’s no built-in connection so changes to the server side aren’t guaranteed to be reflected on the client side.

Breakability

Returning to earlier points, in assessing the various techniques to pull further information from the server side, without reloading the entire page, think a key point in assessing their relative merit is breakability (to abuse the English language some more). If you make changes over here, what are the chances you broke something over there?

Taking the GMails frames + intensive Javascript approach to task (overnight I now have an account – many thanks for the offers!), I see it as highly breakable. It may not be a problem for Google, who have the staff to get it right but much of the relationship between server side elements, from the point of view of the user, is deeply embedded in the client side. A naive question perhaps but what’s the impact of, say, changing the name of a GET variable server-side?

Similar thoughts cross my mind when thinking about Livesearch. How easy would it be to build and manage a complex application based on the same approach? That’s not to say it can’t be done but I assume Christian maintains the client seperately from the server. And the server uses CSS classes which are defined here. On the one hand Livesearch is the epitomy of KISS. On the other hand, do the dependencies between various independent entities (Javascript, CSS and PHP) really amount to inherent complexity?

So I guess the question boils down to how do get changes to the server side to propogate cleanly to the client side?

A Lesson from SOAP

Think SOAP has got one (but not more) redeeming feature. And in fact it’s not defined by SOAP itself but rather WSDL. WSDL means accessing a service like Dietrich Ayala’s WhoIsGoingToBePresident service with PEAR::SOAP can be reduced to just;


require_once 'SOAP/Client.php';

$WSDL = & new SOAP_WSDL('http://dietrich.ganx4.com/president/server.php?wsdl');

$Predictor = $WSDL->getProxy();

$Prediction = $Predictor->whoisgoingtobepresident();

echo 'Kerry: '.$Prediction->kerry.'
';
echo 'Bush: '.$Prediction->bush.'
';

WSDL provides a precise definition of the service to the point where clients for the service can be generated and used simply by knowing the URL to the WSDL document. You don’t need to care about the underlying network or the data encoding required to allow it to be passed over the net.

In terms of yesterdays rant, the mechanism and payload have been taken care of without you needing to think about them. All that’s left is population – displaying the data in a visually appealing manner. If there remote service changes, of course it’s going to break your UI, but you’ll only need to think about making the right RPC call and tweaking the way you display the result.

In my opinion, hooking up the server side and client side of a web application needs to be this easy. At the same time, there’s only the Mozilla SOAP implementation out there and last time I tried it wasn’t a pleasant experience. Also SOAP fundamentally means bloat, IMO – it’s just too much complexity for this problem. We’re not concerned with giving the world access to a web service – only interested in getting Javascript talking to the server side for a single application.

Javascript Generation

The other way to go, as I see it, is to cut out the middle man (WSDL) and have the server side generate Javascript primed to talk back to the server, leaving only the population to think about (where we probably want some level human invention for the sake of a good looking design).

In general, generating Javascript which reflects something happening server side can be made to work well.

PEAR::HTML_QuickForm does a nice job when it comes to validation, for example. If you specify a server side form validation rule, such as a regex to require alpha-numeric characters, QuickForm can also embed the same rule as Javascript in the output HTML, saving the user a page reload before finding out they made a mistake. There’s no need for you, the developer, to think about the Javascript – it’s generated automatically. Change the regex in PHP and the Javascript changes as well to reflect it.

We’ve used a similar technique in (obligatory plug) WACT with the
tag, which performs client side autocompletion (try typing in the name of a country). There’s no need to mess around with Javascript by hand. In your HTML template you might have;


Then in PHP you can load the tag with a list of items to autocomplete against like;


$Page = & new Template('/form.html');
$CountryInput = & $Page->getChild('countries');
$CountryInput->setAutoCompleteList($listOfCountries);
$Page->display();

At not point are you confronted with Javascript.

So how do we apply code generation, in a useful way, to pulling data from the server side into a page which is already loaded?

Bearing in mind SOAP/WSDL, my view is the way to go is generating a Javascript client which takes care of the mechanism and payload and needs only to be called by a developer.

To avoid a long discussion, here’s how I think it should work, more or less, from the point of view of a developer working with it.

If I have a PHP class like;


class UserAdmin {
function & listUsers() {
return new DBC::NewRecordSet("SELECT * FROM users");
}

function deleteUser($username) {
$sql = "DELETE FROM users WHERE username='".DBC::escape($username)."'";
return DBC::execute($sql);
}
}

You should be able to register it with the Javascript script generator, perhaps something like;


$Generator = & new Javascript_Generator();

if ( isset($_GET['class']) && $_GET['UserAdmin'] ) {
echo $Generator->generate('UserAdmin');
}

Whatever this generates I can then call directly in Javascript e.g.;



It’s not quite as simple as that – there’s a need to define callbacks if we want to use async calls to the remote server and I haven’t discussed the PHP “listener” which responds to the requests like “listUsers”.

But that’s the basic idea. The generated Javascript client is built, using reflection, from the class I defined in PHP. In other words I can work with an object in Javascript which behaves according to a PHP class definition. The mechanism and payload are “invisible” to the code I write in PHP and Javascript – it’s just a question of populating the UI with data. The potential “breakability” is, hopefully, much lower and changes can be quickly propogated to Javascript.

Anyway – that’s where I’m hopefully headed with this thing – time is not permitting right now. Could it needs to be shot before it becomes a lengthy waste of time. I’d certainly love to hear better ideas (or best yet; that you’ve already done it).

More Remote Javascript
Finishing up, to give Jason yet more stuff to look at;

- XML for Script Server-Side Proxies; more tricks with the script src attribute and a very clean implementation (part of a very cool Javascript library).

- JSRS; uses Javascript generated iframes. If I understand it right, very cool as the iframes never get added to the document, just used to fetch remote data.

- Remote Scripting with Javascript, IFrames and PHP (article)

- Blueshoes JSRS – seems to be a fork from JSRS linked above which uses WDDX.

Anything else?

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • John Lim

    Harry. I can highly recommend JSRS. I have been using it since 2002. Works very reliably.

    http://www.ashleyit.com/rs/

    For a demo of what we are using it for, see the cascading menus that auto-update at:

    http://phplens.com/lens/ex/ex960.php

    PS: The iframe does get added to the document, but is invisible, unless you enable debugging.

  • arborint

    One of the problems with Javascript is the total lack of standard libraries to do even the simplest of things. Button rollovers and forms validation is a classic examples. I’ve coded various versions of each or used the Macromedia or Adobe ones.

    Why there are not standard libraries or at least standard interfaces for these things has always baffled me. There are just not that many ways to solve these problems in Javascript. Standard interfaces would give us a standard to code to.

    I think if the PHP community came up with some basic interfaces for Javascript libraries the problems you mention would be reduced. Then there would be the target for interoperability. Libraries would then compete on performance, stability and error handling.

  • patrikG

    For something akin to a library there was DynAPI (http://www.dansteinman.com/dynduo/), but unfortunately, this projects has lapsed.
    It was specifically geared towards DHTML – whic does leave out probably the largest share of what Javascript is used for these days.

    Simply to make the point that there were efforts to make a common API for Javascript to all browsers.

  • http://www.realityedge.com.au mrsmiley

    A comment on JSRS for John Lim, I was (note the past tense) all for JSRS in past, but the issues are with cross browser compatability. It only works properly in IE. Its flaky at best under Mozilla with the results not always what you expect. I also found that in terms of application scalability, the code base becomes hard to maintain long term as you have to maintain both client and server side code bases in tandem to make sure they both work all the time.

    For DynAPI, as far as I know, its still under development, but has slowed right down. My biggest thing for creating a rich client was a drag drop interface, but couldn’t get it working the way that I wanted.

    I have yet to find a nice unobtrusive DHTML api that doesn’t require me to create interfaces “their way”. The key for me is that I still want to have to API’s at my disposal without have to create the interface in a way dictated by that API. For example, to get drag drop running under DynAPI, I cant enable it for relatively positioned elements, only absolute.

    Maybe we should be posting some of these concerns in the DHTML blog to see if those guys can shed any light on the matter?

  • arborint

    I haven’t used JSRS, but I have used DynAPI for fairly complex drag and drop stuff that worked like Flash but could be easily generated out of a database. It was pretty easy to get thing working using DynAPI. If there were standard libraries like DynAPI there would be much less need for Flash, especially if you add SVG to the mix. I’d like a more modular set of libraries than DynAPI, as nice as it is, because as I recall it doesn’t deal with the RPC stiff Harry is tackling.

    I am interested in the direction of the examples shown above. I don’t think we need classes built by reflection as much as just solid classes to do the things we need on the client side. The problem with the Javascript sample above is that it magically assumes that there is solid code to do the communication and to display the values. I think we need to start with those libraries. Then it will be easy enough to wrapper them to reflect our PHP framework.

    I think a set of coherent libraries that dealt with things like:

    - graphic positions and display
    - events (timer, keyboard, etc.)
    - inter-frame communication
    - window/document control (open, close, set, etc.)
    - layer management
    - forms data access and validation

    On top of those you could build things like drop down menus whose items could change based on a remote data call.

    I think Harry is more interested in the layer above these, but it would be interesting to do a survey of available Javascript libs to see of a coherent client side framework could be pieced together. Something specific to PHP that PHP applications could target just like Dreamweaver or ColdFusion target their own Javascript libs.

  • http://blog.casey-sweat.us/ sweatje

    I have found the x lib from http://www.cross-browser.com/ to be very good for cross browser supported drop and drag operations. However, I agree with arborint, the intent here is to allow for easy communication between javascript and php without refreshing the entire page. js doing xmlrpc was what I tested in my origial blog that prompted this, along with noodling using i?frames to do the communications. The slickest response I have seen so fare is from dotvoid how posted this little gem: http://www.dotvoid.com/view.php?id=13
    In my mind, this is jsrpc :)

  • arborint

    Excellent Jason. If you added the dotvoid.com method and also straight frame/iFrame communication to the X lib you may be close to what I was asking for. The real question is: what does a Javascript library that is PHP centric look like. Maybe you can shed some light.

    One of the problems I see is that javascript tends to spread out. Your wactjavascript_autoComplete example demonstrates this. It has the Javascript function in the , but the data array and the specific form field onKeyUp= code is down in the HTML.

    My experience is that the stuff in the should be done first, all the while building a list of the javascript code that needs to be included in the . Then after the is finished you build the required javascript for the . That also eliminates multiple declarations of the same javascript functions by the generator when several things in the use the same functions.

    I do like the way wactjavascript_autoComplete accepts a data array. Receiving most/all of the data via parameters is one key requirement for a PHP centric library.

  • http://www.phppatterns.com HarryF

    I have seen projects that have tried to link the server and client side for the purpose of building rich clients, such as Tika:TK and WABAWT (Java / Servlets). Aside from that fact they tie you to a particular look and feel, as mentioned above, think they stuggle with the basic request / response nature of HTTP (vs. desktop event models). A solid layer of “networking” might solve that…

    The problem with the Javascript sample above is that it magically assumes that there is solid code to do the communication and to display the values. I think we need to start with those libraries.

    Agreed. All this has got me thinking I need to release what I’ve done so far and see where it goes.

  • http://www.oscarm.org/ omerida

    I’ve started looking more and more at javascript and definitely agree that it’s worth a second look from those of us who may have dismissed a few years ago. I’m a bit frustrated that standard libraries to do common stuff aren’t easier to find or available? Any good places to look besides hotscripts and the like?

  • http://www.sample.com Widow Maker

    Only starting to briefly reading this second instalment though very interesting all the same.

    Thx to Harry for bringing the subject up, please continue.

  • arborint

    Here’s a thought. Imagine a PHP framwork around using Javascript within PHP. I’m imaging something that would do things like:

    - Treat Javascript like a template so you could fill in varable values, build arrays, lightweight code generation (e.g. multiple ifs for form validation functions), fill in parameters, etc.

    - Associate function calls with the Javascript lib they are in, so when you’re done generating code it could give you a list of which libs needs to be included in the .

    - Support for (somehow) connecting the client side Javascript with server side PHP. More important for RPC or forms, less for graphic stuff.

    - Something that worked standalone, but also worked with template systems.

    Perhaps if we gathered up examples of the most common ways that Javascript works together with PHP (e.g. wactjavascript_autoComplete), we could find some patterns in what seems to be chaos.

  • Marc

    Ever took a look at http://www.domapi.com??

  • soma
  • Anonymous