It’s rant time again (apologies—move along unless you’re feeling fiesty). It’s that REST vs. SOAP thing.

I forgive you this time :) … this subject needs to be debated

In today’s frameworks, in JAVA/.NET and even Python and PHP … it is a super-simple task to create a web service … you just define some methods and annotate them as being web-services.
The point is that … the underlying technology (SOAP in this case) is transparent and could be replaced, if needed, with CORBA / XML-RPC / whatever new technology comes along … that’s why abstractions *ARE* usefull.

At that point SOAP’s basic type encoding doesn’t help you—you need to add your own XSD to register this rule.

There is no way that I would ever delegate it to the transfer protocol itself to validate the data it transfers. The data should be validated either at the source or at the receiving end; in between it’s just transferred.

Web services are really nothing more than glorified RPC’s, and should be viewed as nothing more than remote routines, and treated that way.

Harry, if I understood you correctly, you have listed several technologies and standards (REST, custom XML, XSchema, Relax-NG, XSLT…) as a “better” replacement for a single standard (SOAP, with or without WSDL).

Well first you’re taking bits of what I said out of context but I understand the point.

Guess there’s a short view and a long view here;

The short view… the technologies I’m talking about and generally well known and most people will be working with some or all in a typical web application. SOAP doesn’t mean you can forget XML and certainly not XML schema. In fact there’s a good chance if you’d be using XSLT somewhere as well – many of the web service development platforms encourage you to work with “XML views” on your database and use XSLT between that and your “web service layer”.

So really you’re often talking about using all these technologies anyway and the decision is then with or without SOAP.

The long view… generalizing (and probably badly) there’s basically two mindsets to development these days.

One is the “plug and play” way – putting together components from different vendors, with help from a mega-IDE, and staying well away from the details – if you’re doing J2EE or .NET, this is probably what you’re doing. Perhaps the key point of view here is “one size fits all”.

The other is the “get intimate” way, where you get deeply involved in the details and regard most development tasks as a special case that needs you personal attention – one size cannot fit all – this is more the dynamic language way and the primary tool is a text editor of some sort.

I’d argue REST fits the latter view of development and SOAP the former.

And I think it’s safe to say Yahoo, Google, flickr, del.icio.us and most any of the big online names use the latter approach.

For one, I absolutely agree with you that the principles behid REST are valuable, and I would like to see it well implemented. However, the current state of the affair is that classic Web services have a more stable foothold in the development environments.

Yes and no. That REST has a label is almost a problem in itself. Really we’re talking about stuff people already use, in particular HTTP and perhaps XML. You can find HTTP clients and server for most any language and platform you care to name, plus tools for logging, debugging, authentication etc. XML in general is also well done.

But there are some areas where best practice and strategy is not well defined. For example how do you publish all your URLs in a manner which is easy to discover – perhaps this is the nearest thing to “WSDL for REST” (although you could use WSDL itself to REST).

But turning the guns of SOAP / WS-*, although there’s plenty of tools and implementations, my experience is it either “just works” and everyone breathes a sigh of relief or it doesn’t work, and you’re in for a debugging nightmare. Because SOAP et al adds extra layers on top of HTTP, it gets very hard to see what’s really happening. Add to that that SOAP is trying to be everything to everyone means that what one person does with SOAP (e.g. document / literal style) may not be well supported by an implementation on another platform.

Fow two, REST defines only half of the story, i.e. the way how to transfer your data and RPC. However, there is still no standard for the actual format of the transferred information—only in these comments we have mentioned several, from custom XML to YAML to JSON to PHP serialization. The advantage of SOAP is that it uses a standard, even if not technically ideal, part of the HTTP protocol for information transfer, and has a standard, well-defined format of that information—SOAP.

OK here lies part of SOAPs illusion – there’s basically two styles of encoding with SOAP – encoded and literal and it seems you’re talking about the encoded style (nice summary here BTW).

In encoded style, you have a one-to-one correspondance between native data types (string, int) and tags in SOAP (xsd:string, xsd:int).

Now that’s fine, but what, as stepz seems to require up there, do you do if you want a rule to apply to a particular string (e.g. a username) that it’s between, say, 6 and 15 characters and that the characters are alpha-numeric?

At that point SOAP’s basic type encoding doesn’t help you – you need to add your own XSD to register this rule. At the point you need to start doing this, encoded SOAP loses some it’s value – generally if you want to validate some tags, you want to validate all tags – at that point you way as well have gone with document style SOAP and you’re basically in the same position as you’d be if you defined your XML format from scratch, and you start wondering what the value of SOAP was in all this. And remind me what was the benefit of SOAP vs. XML-RPC again?

And for three, as I argued above, SOAP has been implemented in virtually every development tool and platform out there, making use of Web services as simple as (in PHP, from a recent real-life example I had):

I’d still say you can’t beat the example above for simplicity. Similar can be done with “plain old XML” like this.

“URLs and HTTP is why the web is working. HTTP and URLs work so well they’re even capable of being useful in spite of complexities in media-types based on XML.”

(From lesscode.org)

For one, I absolutely agree with you that the principles behid REST are valuable, and I would like to see it well implemented. However, the current state of the affair is that classic Web services have a more stable foothold in the development environments.

Fow two, REST defines only half of the story, i.e. the way how to transfer your data and RPC. However, there is still no standard for the actual format of the transferred information — only in these comments we have mentioned several, from custom XML to YAML to JSON to PHP serialization. The advantage of SOAP is that it uses a standard, even if not technically ideal, part of the HTTP protocol for information transfer, and has a standard, well-defined format of that information — SOAP.

And for three, as I argued above, SOAP has been implemented in virtually every development tool and platform out there, making use of Web services as simple as (in PHP, from a recent real-life example I had):

The lesson here is that all this complexity being pushed by so-called enterprise architects, software vendors and big 5 consulting companies is bullshit. If you are building distributed applications for your business, you really need to ask yourself what is so complex about the problems that you have to solve that makes it require more complex solutions than those that are working on a global scale on the World Wide Web today.

Anyway, if anyone can point out any pragmatic reasons, why should I use REST, I’d be happy to hear them.

Think the main pragmatic reasons are general: stuff like performance / scaling given that HTTP comes with a bunch of stuff to help, easier to debug (”just” HTTP rather than HTTP + extra layers / extra libraries), simple APIs (urls, GET, POST etc.) and simple libraries.

But none of those are particularily convincing. Also looking at your requirements, seems like there’s experience with SOAP already there which means you’re already able to visualize this project from start to end. REST has a bunch of grey areas where best practice seems to be missing, so it’s entering a “riskier” zone.

But some general thoughts against your list, which are somewhat like “you don’t want to start from here”.

* Current system interconnects are anything from NFS transfered xml files to somewhat RESTy HTTP GET access to an imagedatabase to more and less strictly defined SOAP interfaces. Reuse is pretty limited and won’t be one of the top concerns.

WebDAV might be interesting as an alternative to NFS. If you haven’t tried it, see HTTP_WebDav_Server and try a webdrive evulation as client.

* Software components are for now written PHP and Python, but I’d hesitate to shut out Java or .NET forever.

If you’re using XML, in general, Java and .NET have a way in. Of course it depends on exactly how you’re using XML but…

* All exchanged messages need to carry transaction ID’s, logging metadata, authentication info and some other similar metadata.
* Almost all messages are required to be logged (by law actually). Logging is to be taken care of by the communicating peers themselves.

I’m not sure SOAP gives you any particular advantage there – if this information is getting packed into messages, it almost makes more sense to do the work of designing an XML format carefully (vs. say allowing encoded SOAP to take care of it for you). Perhaps there’s some WS-* specs that could help out but seems the more you use of the WS-*, the less chance you have to broad support on many platforms.

Regarding logging, it sounds like you’re really wanting to store copies of documents (vs. encoded data structures) – there’s SOAP document literal but I think you’ll be headed into trouble there with PHP and Python. Thinking instead of exchange of “simple” XML documents and files makes this a little easier, plus you’ve get access to URLs logged by Apache (or otherwise) which may be nice to have (vs. all access tunnelled through POST to one URL).

Should also point out that HTTP already comes with fields for some basic meta data (which may or may not be useful), such as date/time stamps, file types. You’d also be able to apply mod_security easily to a REST based service.

* I’d like the ability to automatically validate the exchanged messages for correctnes.

That’s doable again, without SOAP. XML Schema or Relax NG if you want to do this against XML. That said, using Python or PHP it’s in some ways easier to do it in code (that’s if validation will only be happening locally). That advantage of XML schema there is (sadly almost), it’s well known so if you publish it to someone else, there’s a better chance they’ll understand it. Validating that a string is between 10 and 15 characters though, effectively nullifies the advantage of encoded SOAP and means you have to make the effort of designing an XML format.

* Some services, about 1/3 or so, will need to be exported to the outside with SOAP+WSDL using a bit different set of metadata than internal use requires.

The question there is is SOAP+WSDL actually a requirement in itself? What you say there suggests XSLT but again, the effort of repackaging some data in “raw” Python or PHP is probably minimal.

* The data will have XMLSchema definitions probably anyway for other architectual reasons. Atleast base datatypes that are combined into messages in realtively straightforward ways will anyway be defined in XMLSchema.

That’s again pointing at “document / literal” but doesn’t have to mean SOAP. In general my experiences of XML Schema + SOAP is the relationship between them isn’t broadly understood and is subject to interpretation (rightly or wrongly) and varying implementations – publishing an XML schema alongside a “REST interface” strikes me as something that the “other end” is more likely to pay attention to. In PHP that’s an explicit call to schemavalidate.

* A usable API for RPC is required, together with mapping to native datatypes. Once the messages are defined, I don’t want to know anything about wire encoding, wire protocol or anything. I just want a RPC with all the behind the scenes including logging, routing, etc. taken care for me.

Here’s the fine detail part – HTTP is itself an RPC protocol with a small collection of methods (GET, POST, PUT etc.) – SOAP (rpc/encoded) and XML-RPC “tunnel” through existing HTTP methods (normally POST). Anyway – can understand what you mean by “not caring” but think this is an elusive goal. With RPC/encoded SOAP you can get to a “hello world” quickly without having to care about what goes over the wire but given earlier validation requirements, the effort required to define schemas for your data structures could be equally spent defining your own XML formats which, once done, means you can stop caring ;)

Overall I guess the immediacy you (sometimes) get with SOAP or XML-RPC tails off once your API grows or things like performance become a concern.

Perhaps one argument for HTTP which is strong and easy to grasp is conditional GETs – this get’s very hard / ugly to do with SOAP/RPC or XML-RPC IMO – you have a return value which is either the result you want to something saying “unchanged” and that translates into ugly hacks, to the point where is no one does it. Where I work we used XML-RPC at one point for moving documents to a web server for display – at some point have to re-write it – when an unchanged document takes 5 seconds to load and people are saying “man this is slow”, having an efficient / easy way to report changes, update caches seems like a really good idea.

So all said and done, even considering all the talk about demise of SOAP, I’d still go with SOAP+WSDL for now. The reasoning from the PHP side (haven’t had the time to thoroughly study relevant Python libraries):

Actually should point out that the demise is really for online use on the real Internet – for purely “internal” use, where you control the endpoints, SOAP is just as valid (or invalid) as similar older technologies (like CORBA or DCOM).

SOAP has ext/soap on PHP side, that makes RPC braindead simple. I don’t know of anything that elegant for REST.

Although it doesn’t match all your requirements, you need to look at that Amazon example up there again – it’s pretty braindead. Also PHP’s serialized format is pretty well supported in other languages – see the list at the end of this post (but note also the risks). JSON or YAML are also well supported in most languages.

SOAP has atleast some tool support, REST has pretty much none if you don’t count the fact that in simpler cases the format is simple enough to use by hand.

Well it could well be argued that REST doesn’t need any extra tools because everything is already there. For debugging your browser plus livehttpheaders might often be enough. There’s a ton of HTTP testing / automation tools, beginning with WWW::Mechanize. Of your requirements only the use of XML schema really needs something toolwise (as in XMLSpy).

Anyway, if anyone can point out any pragmatic reasons, why should I use REST, I’d be happy to hear them.

Returning hear again, if you’re willing to donate some project hours, why not just try one REST web service.

I still can’t believe how slow YAML uptake has been by the PHP community.

Think there are some specific functions stalling interest, namely serialize() and parse_ini_file().

I still can’t believe how slow YAML uptake has been by the PHP community.

Oh dear god, that’s awful. If I didn’t read the yaml.org About page, I’d swear that was invented by a Python programmer. Levels of indentation to denote structure? I can’t imagine working with that in a productive manner.