Crouching Javascript, Hidden PHP [3]

It’s alive! http://xmlrpccom.sourceforge.net/scriptserver/.

Here’s the short explaination…

Let’s say you have a PHP class that looks something like;


class Math {
function add ($x, $y) {
return $x + $y;
}
}

You can now use this from Javascript like;


var m = new math();
alert (m.add(2,2));

No need to think about the underlying mechanism / encoding (as discussed earlier).

A complete example can be seen by looking at the following;

- PHP Source for a “server” – this is what you’d write server-side

- The Javascript Client Code generated automatically from the PHP server (note that can be “compressed” with the switch of a PHP constant)

- The Client (View > Source for Javascript) – this is all you’d code by hand client side.

(Update)
- output from server – example of what the Javascript client get’s back from PHP; a string (for eval) containing an anonymous function which itself contains an array. Much more compact than XML with no issues in parsing (although not very human friendly. The tail of the URL corresponds to the /class/method/ names.

Download all this in the first (alpha) release from here.

Phew! Now that’s out of my system, can catch up with email etc (before Kev murders me ;))

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • John Serris

    I just checked this out *really* quickly and noticed something. When I do the sync tests my browser uses up 100% cpu usage until it gets a response back (tested in ie5+6 and fx 0.9 on XP). The async tests dont hang at all (obviously). If the server doesnt respond back (which happened once while checking this) I had to kill the browser in task manager.

    Can anything be done to avoid this?

    Really excellent work though mate! Looking forward to seeing this grow.

  • Jordan

    This is nothing short of revelatory. I’ve always been very impressed, but very intimidated by slick JavaScript interfaces (GMail comes to mind, of course), as I’m not a JavaScript wizard, but this makes it seem entirely approachable. I can’t wait til this gets a bit more mature so I can implement it in some of my projects. Great work!

  • http://www.phppatterns.com HarryF

    Can anything be done to avoid this?

    Possibly. XMLHttpRequest has an abort() method. Might need to implement some kind of sleep routine in Javascript for a timeout – no sure yet.

    Whether this can be done and preserve compatibility with IE’s implementation of XMLHttpRequest I don’t know – Mozilla’s implementation allows you to do more right now.

  • http://www.phppatterns.com HarryF

    Hmmm – from where I’m am response from the SourceForge server is insanely slow. In this case that may actualy be a good thing as it highlights a bunch of problems I obviously don’t get running locally.

    Technically ScriptServer isn’t tied to XMLHttpRequest – other approaches like using a script src attribute and DOM manipulation might work better. Something to explore.

  • John Serris

    Yeah Harry, sourceforge was unbelievably slow for me too. Thought it was just my end. As you say though, this could be a good thing :)
    You could probably try and simulate slow responses locally to help you build some sort of abort/timeout mechanism.

    I’m really looking forward to using this in a project too!

  • orixilus

    no hanging (tested in ie and firefox), just perfect :) can’t wait for a beta version!

  • http://www.realityedge.com.au mrsmiley

    Wow!!! Pretty much sums it up really. All of your previous blogs on annonymous js functions, data type serialisation, rpc, etc, now make complete sense in light of this project.

    I must say that this is a very slick project and whenever you can stabilise it, I would seriously consider using this in a commericial environment. Assuming of course that the security issues are sorted out as well.

    Might I suggest you examine the work done with the Sarissa project in XMLHttpRequest compatability between browsers. They have implemented a whole host of things in the opposite browser that the other doesn’t have. You might be able to draw on some techniques to handle the request timeouts and such.

    All in all, my response times were excellent, and I didn’t find anything wrong with it using IE6 other than the fact the XUL demo didn’t work, but that’s to be expected.

  • http://www.realityedge.com.au mrsmiley

    Just occured to me, that you haven’t addressed the issue of thin rich clients outside of the xul examples.

    How do you see this project interworking with a cross browser interface? The main strength that I can see from this project is that you can do all your heavy data crunching on the server side, and then use the client side to render the results.

    But if you were to say take this into context with WACT or any other component driven architecture, how would you handle something like a Calendar or a Table of data? Return a string of html from the server and output to the browser using something like DOM?

    Obviously the requirement here is to reduce the number of remote requests to a minimum in order to maximise the speed of the application. For example, balancing the used resources of an arbitrary Table.ToString() versus “foreach TableRow foreach RowCell output “ celldata

    “;

    I know the code isn’t real, but the scenario is. Where do you see this project fitting in with n-tier architectures or MVC frameworks? Does the browser become the controller and the viewport in one? Or server renders the viewport ready for the client to display natively?

  • http://www.sitepoint.com/ mmj

    This client-side/server-side interaction would help solve a number of problems relating to usability of forms.

    1. Validation against existing data: A form could check if a value entered conflicts with an existing value in the database without the user needing to submit the form and get a ‘This username is already taken’ message.

    2. Heirarchical select: Where the values available in one select input depend on the value selected entered in another select input. The select could be updated in real time without the need to have all of the possible combinations (possibly thousands) downloaded already.

    3. “Insert Link:” An insert link feature on a form could present you with a list of links to choose from, and let you navigate within those links without reloading the page if, for example, you have thousands of links in many categories.

  • Evan Broder

    What about dynamically creating an iframe and running dynamic “stuff” through that in situations where you can’t initialize the XmlHttpRequest object?

  • http://www.rideontwo.com z0s0

    Absolutely fantastic Harry. It make a complete mockery of my framework, but it’s today been deployed in SitePoint’s backend CRM system.

    Security holes and all ;-)

    Do the methods (both client and server side) need to be lowercase?

  • http://www.rideontwo.com z0s0

    Little bugfix: I was getting XMLHttpRequest permission problem once I went live with this. Turned out it was because serverURL had “sitepoint.com” instead of the “www.sitepoint.com” that the parent app used.

    Quick fix was in ScriptServer/Server.php, line 55

    changed

    $this->serverUrl = ‘http://’.$_SERVER['SERVER_NAME'].$_SERVER['SCRIPT_NAME'];

    to

    $this->serverUrl = ‘http://’.$_SERVER['HTTP_HOST'].$_SERVER['SCRIPT_NAME'];

  • http://www.phppatterns.com HarryF

    You might be able to draw on some techniques to handle the request timeouts and such.

    Thanks. Will take another look at Sarissa.

    All in all, my response times were excellent, and I didn’t find anything wrong with it using IE6 other than the fact the XUL demo didn’t work, but that’s to be expected.

    The XUL demo should probably be disabled on the SF site – the mailresponder.php (which acts as it’s server) is supposed to be configured to point at an IMAP server, to read someones email. Any volunteers? ;)

    Just occured to me, that you haven’t addressed the issue of thin rich clients outside of the xul examples.

    Very good point to which the general answer is I’m not sure.

    Re: XUL – once you’re able to fetch data from a server this way, suddenly DHTML and forms looks attactive again. Right now think XUL needs to evolve some more to be really useful for “web launching” – the Mozilla crew seem to be more focused on Firefox (understandably) right now. But that’s another long discussion…

    The main strength that I can see from this project is that you can do all your heavy data crunching on the server side, and then use the client side to render the results.

    You also have, to an extent, access to all the servers resources, such as a database or the filesystem. One example I’ll hopefully add soonish is a server side “clip board” – a CTRL+C in Javascript stores the selected text in a PHP session server-side, allowing it to be retrieved later in the session. Who knows what else is possible?

    The main mission here is to build a solid / reliable / predictable / easy-to-use layer between the client and server. With that in place, what might normally be “crazy ideas” become realistic, as you can build on top of what’s already there.

    To that end the unit tests play an essential part in establishing trust in this implementation. Side note – does anyone know of / want to implement Javascript MockObjects? It’s starting to get essential – I need a Mock for the XmlHttpClient in particular (guess I could hand code it)

    But if you were to say take this into context with WACT or any other component driven architecture, how would you handle something like a Calendar or a Table of data? Return a string of html from the server and output to the browser using something like DOM?

    That question hits exactly the spot ;) If we’re stictly following the Rule of Seperation, the server should only return data, leaving presentation to Javascript.In reality, that could lead to some very heavy (performance-wise) clients plus it can be painful to manipulate the UI in Javascript from a developer point of view e.g. how to you View > Source when the HTML you’ve created with Javascript only exists in memory?

    Some kind of compromise is likely. One thing the examples don’t yet really show is how you could represent the implementation of a PHP class directly in Javascript. The returned data doesn’t have to be just primitive Javascript types – you could put Javascript functions and classes in there as well. You’d need to write a PHP class for generating the code, by extending ScriptServer_SerializedElement but the option would be there to return a result set as a Javascript function which excepts, say, a HTMLTableElement and attaches the rows to it. Right now it’s easiest to see this “mapping” with the ScriptServer_Error class, which gets translated into a Javascript exception.

    Re: WACT, I guess it would be a case of writing a tag which generates the Javascript code you would have hand-coded here plus somehow generating the server script. Will have to experiment with trying to make the inputautocomplete tag fetch data from the server-side.

    Where do you see this project fitting in with n-tier architectures or MVC frameworks? Does the browser become the controller and the viewport in one?

    No idea! ;) I’m primarily focused on putting something together that makes this specific problem easy to solve. What the implications are for MVC etc are things I’d rather not think about. It could be that at some fundamental level this approach simply doesn’t fit well with web app design so no-ones going to use it.

    This client-side/server-side interaction would help solve a number of problems relating to usability of forms.

    Absolutely. At work what’s been holding us back from putting an existing, heavily forms-based, GUI online is entirely the usability issues. The web interface has to be at least as good as the existing desktop UI to “compete”.

    What about dynamically creating an iframe and running dynamic “stuff” through that in situations where you can’t initialize the XmlHttpRequest object?

    That’s on the todo list. In this case may actually be better to dynamically create a script tag and use it’s src attribute. I’ve tried to design the code so that switching the “network client” (currently XMLHttpRequest) is easy. Just got to do the work now (anyone interested in helping out [you need to be very self-starting though], say the word).

    Do the methods (both client and server side) need to be lowercase?

    Technically I guess not but the reason I opted for this was because I’m using PHP reflection functions like get_class_methods() to allow easy “registration” of user PHP classes with the “server”. With PHP4 these return all the method names in lower case. With PHP5 the case it preserved. Meanwhile, in all PHP versions, when you call a method (or create an object) it’s case-insensitive while Javascript is case-sensitive. The bottom line is it allows ScriptServer to work without problems under PHP4 and 5.

    The happy side effect though is it reduces the chances of conflicting with native Javascript classes, which always use studly caps naming. In the example here, we’ve got a class callled ‘math’ – if case sensitivity was preserved it would clash with the in-build Math class in Javascript.

    Some other notes on registering a PHP class with the server. The constructor and methods beginning with an underscore will be ignored.

    You can also use Jeff’s “handle” protocol that I was talking about here, so you can load a whole bunch of classes into the server without creating objects until they’re used or having PHP include them (you’d need to pass you’re own ScriptServer_HandleDescription when calling ScriptServer_Server::addHandler() though).

    $this->serverUrl = ‘http://’.$_SERVER['HTTP_HOST'].$_SERVER['SCRIPT_NAME'];

    Thanks for the tip off!

  • MiiJaySung

    Very interesting article.

    Ties in nicely as I might be researching something similar for some stuff at work. Basically I will need a system where things like only the sidebar or only a form needs to be refreshed on a page, so pages are split into zones/sections. The big issue for me is what to do if JS is turned off. This is where WACT can play a big part, as someone already pointed out. If JS is not there, the JS events aren’t called for things like links and form buttons. This is where WACt can come in handy as there could be a server side form and anchor tag to deal with making URL’s that are adapted so the whole page can be reloaded in case JS is not there. This would also mean you might want to make some custom controllers to deal with different sections of pages that have dynamic content, such as side bars, forms, etc.

    –Jason

  • http://www.phppatterns.com HarryF

    The big issue for me is what to do if JS is turned off.

    That’s always the turn off with Javascript. Ideally, as you mention, if Javascript is unavailable the application should be able to switch to “request / response” mode where the entire page has to be reloaded on every user click. Gonna be alot of work to make that seemless though.

    I’m interested in a “subclass” of application where accessibility is not an issue and you can reasonably demand users conform to requirements. By happy coincidence the type of web application that really has a need for “rich components” tends to the sort of thing you limit to a smaller group of users (rather than exposing to the entire net). The classic example is the admin interface behind a website which tends to only have one or two users but is very “click / reload” intensive and could really benefit from this approach.

  • Julian Tarkhanov

    Is the escaping properly handled in the JS layer?
    I noticed multibyte chars are not passing through (which is sad because this library rocks).

    Maybe you can take a look at this
    http://www.zend.com/codex.php?id=839&single=1

  • http://www.phppatterns.com HarryF

    Is the escaping properly handled in the JS layer? I noticed multibyte chars are not passing through

    More stuff on the TODO list – doesn’t surpised me the multibyte strings are a problem. That’s likely to be an area where I need help. Any chance of a failing unit test case that describes the problem? Alternatively example data that fails

  • http://www.entechnevision.com Dwarth

    one of the biggest concern I have now is that IE for mac doesn’t support XmlHttpRequest yet and not all mac user use Safari or a good browser like Firefox or Camino !!!

    When the migration to new browser from mac user will be more present those solution will be more possible (of course for intranet or high end site it’s marvelous)

  • julik

    You are not obliged to use this. The beauty of this solution emerges from the fact that you can have the same logic being calle through Javascript almost in realtime, and by PHP scripts with traditional request-response loop.

    For example – a traditional ‘sitewide bookmarking’ – you add an ‘add bookmark’ link, which at the same time calls the JS to do it in the background, and if JS doesn’t work – the link redirects to the page where a bookmark is added from the referrer. The user is then bounced back to the page he was at. Accessibiliy at it’s best, fancy and coll, and will work on a PDA. And because you have the same logic accessible in both places it is very easy to implement :-)

  • http://www.phppatterns.com HarryF

    one of the biggest concern I have now is that IE for mac doesn’t support XmlHttpRequest yet

    Will be looking at alternatives to XmlHttpRequest soon (such as tricks with script src or frame src). I think it should be possible plug those in instead of XmlHttpRequest but won’t know until I’ve tried it.

    Julik has a good point as well.

  • Davey

    It should be noted that Opera 7.6 Preview 1[1,2] is said to include an xmlHttpRequest implementation, would be good to test this in Opera :)

    - Davey
    [1] http://my.opera.com/forums/showthread.php?threadid=65041 – For Win32
    [2] http://my.opera.com/forums/showthread.php?&threadid=65170 – For *nix

  • http://maetl.coretxt.net.nz/ maetl

    Excellent work Harry! I really like it. As you say, it would be great for implementing admin clients and potentially things like rich text editors too.

    Some time ago, I used JSRS to make a ‘command line’ search form, where the results from the server slide dynamically into the page with motion, rather than just a dull page refresh…

    It would be interesting to explore ways this scriptserver could liven up search results.

  • http://www.phppatterns.com HarryF

    Is the escaping properly handled in the JS layer? I noticed multibyte chars are not passing through (which is sad because this library rocks).

    Think I’ve fixed at least some of this problem now. Seems the problem is PHP’s unserialize().

    If you have a string like “Joe Bl

  • http://www.jolttheweb.com night_prowler

    I Don’t understand, I have been mixing PHP and JavaScript for easily a year now and using similarly to how you showed in your sample…and very often have JavaScript query the server for a variety of reasons.

  • ostri

    Harry I like the idea of marshalling you presented above. It’s so
    obvious, but on the other side so simple. All good ideas are simple. :-)
    This pushed me into into detailed investigation of your example.
    I have few suggestions for improvements:

    1. Some JS code is fixed (e.g. marshalling code) and some variable
    (e.g. JS code that depends on class/method to be called). I’d rather
    send fixed JS code as is, while variable code can be generated through
    some engine.

    2. IMO there are three types of js code:
    - type A – fixed code that does not change (e.g. marshalling
    code). There is no need to generate this code at all.

    - type B – code that is generated to reflect the classes that are
    (virtually) invoked remotely. This code depends on class end method
    signatures and it can (must) be generated. But this generation
    should occur only when the definition of the php class changes. That
    is once per session at most.

    - type C – js code which is only a medium for data exchange, and it changes upon each remote call.

    type A and B should be defined at develompmet time (we used to say
    compile time, but there is no compiler :-) ). While type C should
    be generated on the fly. This way we can lower the burden on CPU
    and also on network.

    3. I think we should learn something from CORBA. With a litle help of
    reflection (php 5) we could generate js code that needs to be
    generated (Type B). There are several ways to assure that client and
    server stub are in sync. I am investigating a time stamp on generated
    JS code and a corresponding HEAD call inside js class
    constructor. Another way should be just plan cvs $Id$ string.

    Question: Different type of communication without page reload is
    adding a script to head during the JS code execution. I was able to
    find html realisation of this idea. Do you have an idea how to do it
    in XUL ?

  • http://www.phppatterns.com HarryF

    1. Some JS code is fixed (e.g. marshalling code) and some variable (e.g. JS code that depends on class/method to be called). I’d rather send fixed JS code as is, while variable code can be generated through some engine.

    Understood. The ScriptServer_Renderer class is pretty dodgy in fact – makes life easy right now (allows fixed JavaScripts to be stored outside the web root) but will probably need rethinking later.

    Question: Different type of communication without page reload is adding a script to head during the JS code execution. I was able to find html realisation of this idea. Do you have an idea how to do it in XUL ?

    Haven’t seen it done yet. There’s an interesting user note at the bottom here which is suggestive – perhaps it’s overcomplicating things but may be the overlay can be changed at runtime.