Jason has been pondering the options of getting Javascript talking to PHP or, put another way, how to get a web page to load data without reloading (or at least without appearing to reload).
Why do this? Because you can give the impression of a rich client (a desktop GUI) vs. a typical web-based thin client where simple user interaction often requires extensive page reloads and waiting around.
By way of example, consider a simple web-based administration app where the administrator has a list of users and can edit each user in the list. How can we avoid reloading the list of users between each edit to a single user?
At first glance it’s a simple problem to solve – or perhaps it should be a simple problem to solve. Look a little further and you discover it’s one of those grey areas of web development that leaves you thinking “It’s 2004. Someone must have solved this once-and-for-all by now.”
Anyway, this is a long and not-particularly-focused rant which also follows on from Seperating Browser from Resource and Serializing PHP data structures for Javascript.
Feel free to flame / correct. The mission is to try to define the “problem” as fully as possible. Perhaps this needs to be transplanted to a WIKI in due course.
Mechanism / Payload / Population
Think there’s three main technical areas to consider when thinking about how to get Javascript talking to PHP;
– the mechanism you use to transport information between Javascript and a remote server
– the payload you actually exchange over that mechanism
– how you populate the user interface with the payload you’ve received
Note deliberately I’ve avoided words like “transport” and “protocol” because I think they cloud the issue, particularly when dealing with stuff like XML-RPC which amounts to a protocol sitting on a protocol.
Mechanism
First a quick word on terminology;
Synchronous vs. Asynchronous Calls
In the context of this subject, when you call some function in Javascript, which instructs the browser to fetch something from from a remote server, does code which executed the call sit and wait for the function to return (a synchronous call) or can execution continue, allowing the function to execute in the background (an asynchronous call)?
From the point of view of an end user, when they click on that “Get Data” button, does the user interface freeze until the data fetching operation is complete (which will be the case with a synchronous call) or can they continue messing around while the browser is busy fetching stuff from the remote server (asynchronous)?
Side note – related to this in the notion of blocking or non-blocking sockets. See also stream_set_blocking() in PHP.
Available Mechanisms
Considering the options you have to get Javascript talking to a remote server, realistically (i.e. ignore stuff like Mozilla’s SOAP implementation) it seems to boil down to one of two choices;
– Tricks with HTML src attributes
Given a tag with which refers to an external resource and can return something parse-able, you can vary the src attribute using Javascript, allowing the main page to remain resident in the browser.
The most common way to do this is with a frame or iframe – see Remote Scripting with IFRAME (as the first obvious link from Google).
Another approach is varying the src attribute of a script tag, which is illustrated nicely by Javascript Serializer, a library to get PHP and Javascript talking, which implies loading executable Javascript from the remote server.
In general I guess this is probably the sanest mechanism, if you want something you can make work quickly, with minimum “risk”. It is fundamentally “clunky” though, IMO. You have little means to identify and handle errors such as the remote server being down at the time the request was made plus you’ve got to embed stuff in your HTML which really shouldn’t be there (and is just waiting from someone to break it while modifying the page). You’re also forced into asynchronous calls which isn’t a disaster but sometimes you want your app to make the user wait until the response to an action is complete.
– XMLHttpRequest
XMLHttpRequest, available with more-or-less similar APIs in recent versions of Mozilla / Firefox, IE and Safari gives you a Javascript object you can treat as an HTTP client.
XMLHttpRequest gives you a lot of control, allowing your code to be aware of server responses (like 404) for error handling as well as providing synchronous and asynchronous modes e.g. (Mozilla example);
var XmlHttp = new XMLHttpRequest();
// Async - execution continues after this
XmlHttp.open('GET', 'http://localhost/slowscript.php', true);
// Sync - execution waits until HTTP request is complete
XmlHttp.open('GET', 'http://localhost/slowscript.php', false);
There’s no need to embed dubious tags in your HTML
The obvious point against XMLHttpRequest is the browsers which don’t support it but, as in Jason’s case, the type of application where you want rich-client behaviour is often deployed to a known user group (e.g. an Intranet app) where you can reasonably demand they get with the program. Otherwise documentation and experience with XMLHttpRequest is still relatively thin on the ground (but getting better) – if you’re not confident in Javascript that may be a problem.
Payload
The next question is what do you actually exchange? Much of the discussion has been thrashed out with SOAP and the ideas of RPC vs. REST and encoded vs. literal. Avoiding long discussions of semantics, the choices involved looked something like this;
RPC vs. REST
– RPC (Remote Procedure Calls)
Suits programmers; instead of using a some library locally, you know a network address for that library and call it remotely.
For the web, “network address”, typically means a single URL, data exchanged using the HTTP POST method.
To see web based RPC in it’s rawest sense (avoiding clouding the issue with XML), take a look at MIME-RPC, particularly the examples ( head to Kev’s Advanced Email in PHP tutorial as a starting point for understanding MIME plus scan over Methods GET and POST in HTML forms – what’s the difference? if you’re not sure ).
From a PHP point of view you might have a URL like;
http://www.site.com/rpc-server.php
In Javascript you’d POST stuff to this URL (either using the form hidden in a frame like Jason suggests or with XMLHttpRequest).
The simplest view of the problems with web-based RPC are “can you interact with the server using just your browser?” (easy to debug?). The implication is the “src” based mechanism above won’t work as it’s restricted purely to GET based requests.
– REST
The “REST way” (see here for a good summary), in essence, gives preference to the HTTP GET method.
Put another way, it should be possible to surf your remote server with a browser e.g. http://www.tekool.net/php/js_serializer/exemple_calculator.php?1,+,1 – you can modify the URL manually.
REST suits the src attribute based approach nicely and is generally easy to debug. If your server generates something like XML, it’s also easy to re-use for different types of operations (e.g. throw in some XSLT).
The problem (or the place where smart decisions need to be made) is how you map a function call (plus it’s arguments) in Javascript to a given URL. And what about sending a complex (or large) data structure, like a multidimensional array, to the server (you probably need POST at this point).
Encoded vs. Literal
The next question is what form should the data being exchanged actually take? And will a request contain data in the same form as the response?
– Encoded
Encoding the data means representing native variables (in Javascript or PHP) in some intermediate form which both the client and server can understand. XML-RPC is a well known approach to encoding both client requests and server responses in an XML format that describes the data types directly e.g. the server response;
< ?xml version="1.0"?>
…might correspond to the following in PHP;
$response = array (
12,
"Egypt",
false,
-31,
);
…and in Javascript;
var response = [
12,
"Egypt",
false,
-31
];
XML-RPC also pretty much decides that you’re going to use RPC (vs. REST) but still represents one of the “sanest” approaches, given numerous and mature implementations in PHP and Javascript (see list below).
There are alternative ways to encode the data, including WDDX and SOAP encoding. WDDX is probably the next-sanest way to go, after XML-RPC, when considering PHP and Javascript ( PHP WDDX extension and a Javascript implementation).
You might also encode a request in a different manner to the response, perhaps using URL encoding (variables in the GET query string) for the request while even generating Javascript for the response.
Also, a nice trick for PHP (I think), for dealing with more complex data types in a client request, is to serialize in Javascript variables into a string ready for PHP’s unserialize() function (see growing user contributed notes plus my attempt here).
Meanwhile the response could contain literal Javascript ready for eval() – my attempt at a code generator for this is here while Javascript Serializer has a more straightforward implementation.
The main problem with encoding is you can get bogged down, both in terms of development and performance, in translating between native data and encoded data.
– Literal
The literal approach means moving information around without the information itself containing a description of how it translates to native language data types. You might have something extra, like an XML schema document, which describes how the document should be “translated” but it’s up to you whether you use it.
Perhaps the most well known form of this is Amazon’s XML service. If consider a URL like this, it contains information as to what you should translate the document into in your chosen language / platform. That said it’s pretty clear that this;
< ?xml version="1.0" encoding="ISO-8859-1"?>
Would look something like this in PHP;
$catalog = array (
'keyword' => 'php',
'product_group' => 'books',
'products' => array (
array (
'title'=>'PHP and MySQL for Dynamic Web Sites',
'asin'=>'0321186486',
'author'=>'Ullman, Larry',
),
array (
'title'=>'PHP and MySQL Web Development',
'asin'=>'067232525X',
'author'=>'Welling, Luke',
),
),
);
This fits nicely with something like PEAR::XML_Serializer, as I’ve covered before here. Unfortunately have yet to find a ready-to-roll equivalent to XML_Serializer in Javascript (although it wouldn’t be too hard to write one). Do we want to the overhead of parsing XML?
Another, very literal approach is what Christian Stocker has done with LiveSearch, the output being a chunk of HTML you can insert directly into a page. So far no-one’s asked questions like “where’s the doctype?” ;)
One general problem (or key decision point) is how client requests should be sent – how does a client request all the books from three different categories, for example – should requests be encoded while responses are literal?
Also, especially if the server side developers work independently (remotely) from those building clients, how do they agree on a common format? And is it easy to parse? All the joys of screen shaping…
Population
Once we’ve reached the point of having received something back from the server, whatever it is, the next question is how to “blend” it with the user interface?
The choices seems to be between manipulation with DOM, XSLT or somewhat dodgy (but probably easier to hack) visual tricks with frames.
GMail seems to have gone for the later, using frames (I don’t have an account so I’m guessing from what Mark Pilgrim says here). In other words there’s no effort client side at all – the response is delivered in ready-to-view form.
XSLT could be a slicker way to do things. Then again…
Or using DOM you can manipulate anything you like… at a price – you need to write the code which populates that table with the result set you just received from the server. This can get pretty painful and may take you into the realm of templating (already a disaster area on the server side). And life gets even more interesting when you want to do something like update only a single row in that list of users, after editing a single user.
The Way to Go?
No idea ;)
Probably the safest is what GMail is doing – let’s not be clever. That said, reckon it can lead to it’s own development nightmares. More on that another time.
Rant Over
There’s a second part to this focused more on practical development issues e.g. should building an XML-RPC server in PHP look like this;
< ?php
include 'xmlrpc.inc';
include 'xmlrpcs.inc';
function sumAndDifference ($params) {
// Parse our parameters.
$xval = $params->getParam(0);
$x = $xval->scalarval();
$yval = $params->getParam(1);
$y = $yval->scalarval();
// Build our response.
$struct = array(
'sum' => new xmlrpcval($x + $y, 'int'),
'difference' => new xmlrpcval($x - $y, 'int')
);
$return_val = & new xmlrpcval($struct, 'struct');
return new xmlrpcresp($return_val);
}
new xmlrpc_server(
array(
'sample.sumAndDifference' => array(
'function' => 'sumAndDifference'
)
)
);
?>
or like this;
< ?php
include('IXR_Library.inc.php');
function sumAndDifference ($params) {
$x = $args[0];
$y = $args[1];
return array(
'sum' => $x + $y,
'difference' => $x - $y,
);
}
$server = new IXR_Server(
array(
'sample.sumAndDifference' => 'sumAndDifference',
),
);
>>
Another point – how easy is it to break the client-side when the server-side changes? What can be done with generated Javascript?
XML-RPC in PHP and Javascript
A choice few;
– PHP
+ XML-RPC – the ever-experimental extension. Nice if you’ve got it compiled on your server although painful to work with when in comes to introspection and awkward to use with classes defined in PHP.
+ Incutio XML-RPC – excellent library from Simon. Simon’s given up on it sadly. Meanwhile I’m slowly hacking a (now somewhat bloated) fork here. Note: I highly recommend pulling Incutio XML-RPC apart and seeing how it works.
+ PEAR::XML-RPC – traces it’s roots back to the original usefulinc library. Personally not a big fan (e.g. count the globals). If you use it, make sure you find out what the XML_RPC_encode() function does.
+ Keith’s XML-RPC implementation – it doesn’t get any more straightforward.
– Javascript
+ JavaScript o lait – primed for use with XMLHttpRequest. Haven’t tried it but code looks good and it’s what makes this possible. Getting pretty big but also actively developed.
+ Scott Andrew’s XML-RPC message builder – gives you XML-RPC messages – up to you what mechanism you use. Lightweight, which is nice.
+ vcXMLRPC – pretty complete implementation although development now dead.
+ nsXmlRpcClient – Mozilla only over XMLHttpRequest – watch out for the permissions minefield (see here).
Harry FuecksHarry Fuecks is the Engineering Project Lead at Tamedia and formerly the Head of Engineering at Squirro. He is a data-driven facilitator, leader, coach and specializes in line management, hiring software engineers, analytics, mobile, and marketing. Harry also enjoys writing and you can read his articles on SitePoint and Medium.
