Rediscovering the obvious

(Rant alert: you have been warned)

Been plowing through The Art of Unix Programming, which, in general, is excellent – should be required reading for anyone developing software.

Joel Spolsky wrote a good critique under the title Biculturalism, at the end of last year, and agree that Eric S. Raymond does seem to be occasionally (but not always) unreasonably blinded / biased against Windows and the notion of GUIs.

One particular example crops up in his discussion of documentation, where I feel he underrates the value of WYSIWYG editors, even when used to edit structural markup (vs. presentational), where it serves as a form of code generator. Having used Epic Editor (Reviewed by Kevin here), which is a commercial docbook editor, found it worked well – many times better than hand coding, even if you are using emacs. I’d argue that the lack of decent WYSIWYG editor for docbook is why PEAR is largely undocumented, by way of example.

Despite that (and it’s interesting insight in itself to see where he get’s it “wrong”), it’s an excellent book.

What stuck me as perhaps the remark of the whole book was this;

In retrospect, it is rather startling how oblivious we all were to the implications of our own behaviour.

He’s talking here in the context of Open Source development and The Cathedral and the Bazaar but I think that remark can be applied more broadly to software development as a whole – we’re too blind to the human factor. That’s where development methodologies like extreme programming come in. If you read between the lines of hype and jargon surrounding XP (which is par for the course, if you want your ideas taken seriously), it’s really about adopting an approach well suited to normal human beings, bearing in mind our limitations.

Returning to Raymond, he goes on to say;

This experience should make any thoughtful and humble person wonder what other important unifying concepts are still implicit in our behavior and lurking right under our collective noses, hidden not by their complexity but by their very simplicity.

Having originally studied Physics (perhaps the original form of systems analysis, excepting Philosophy) my fundamental view is any system that has observable rules can (and should) be understood by anyone. Once understood that system becomes predictable. Of course the more complex the system, the harder it can be to determine the rules which govern it and experience plays a part in that process, while an individual’s conditioning can lead to bias that obscures the rules. That said, the first impressions of a “naive” observer often carry real insight, and should be valued.

The reverse situation is when intelligent individuals sacrifice what their “gut feeling” and buy into the latest vision of the “right way” to do things, as proscribed by the expert of the day.

An example outside of software development (but relevant to all the PHP fathers out there – congrats to George who’s the latest) is the debate which revolved around violence in entertainment, and it’s effect on children. Back in the 80’s, as Hollywood upped the level of violence in films, there was huge discussion as to whether watching Arnie blow away another legion of minor actors was good for children. The entertainment industry managed to silence the critics by asking “Where’s your proof?” (or: where’s your expert?). Years later the Joint Statement on the Impact of Entertainment Violence on Children, having conducted countless studies, confirmed that screen violence can be damaging to children. Wow – there’s a surprise! Who would have thought it?

Returning to software development, what’s gets my goat is seeing experts (of the flavor that never got past “Hello World!”, if they even got that far) holding forth on the future of software development (as well as software houses riding the wave).

All time classic probably goes to Professor M. Eric Johnson (Professor of Operations Management and possessor of smile that needs wiping) who said, back in December 2003, in an interview on outsourcing with the NYT (registration required);

Out in the Bay Area there are plenty of folks who would love to create a little bit of protectionism around their I.T. jobs, but we are far better off letting a lot of those jobs go. Low-skill jobs like coding are moving offshore and what’s left in their place are more advanced project management jobs.

Nuff said. (there’s further discussion here and here).

Meanwhile, enter Software Factories, the next revolution. I sincerely do not mean this as MS bashing – the heart of the idea seems to the same as the advice Eric S. Raymond has to offer – take advantage of code generation and domain specific languages (which you might argue PHP is an example of). The problem I have with “Software Factories” is the tone used – more “follow us, we know better, sacrifice your reason to the revolution”. Software developers may not buy it but history tells us those in those “advanced project management jobs” most likely will.

Again it seems the human element is being lost. By contrast, ActiveState published a whitepaper in July 2004: Dynamic Languages-ready for the next challenges, by design. If anything can stake a claim to “revolution” (in an ironic sort of way), I think it’s languages like Perl, Python, Ruby and PHP, having succeeded in reducing the gap between human intent and machine instruction, resulting in a real impact on software development effort and costs.

Sadly no one with marketing dollar is really pushing them, although someone at Microsoft is, thankfully, on the case, having hired Jim Hugunin, author of Jython (Python for Java) and Iron Python (Python for .NET).

Anyway – not entirely sure what I’m driving at here but the general feeling I get is those paying for software and it’s development are getting increasingly frustrated by big promises followed by poor delivery. Yet more hype and “abstract practice” does more to damage what’s already a poorly understood “industry”, providing fuel to those who believe software development is a low skilled job. KISS.

Win an Annual Membership to Learnable,

SitePoint's Learning Platform

  • Paul M Jones

    Harry, you left out a quote right after the lead-in “What stuck me as perhaps the remark of the whole book was this;”. Am interested to see the excerpt. :-)

    – pmj

  • arborint

    A little too much to comment Harry. Wow.

    I was interested in you comments on: “follow us, we know better, sacrifice your reason to the revolution” I think this is true (and maybe necessary) that these groups hype their method. It was true of structured programming, OOP, extreme programming, now AOP and Software Factories. The problem is there is no meta layer on top to match the methodologies to the programmers. Maybe someday someone will come up with a questionaire for programmers that then recommends which tool chains might work best for them.

    I think it is pretty funny that Visual Studio 2005 Team System has now invented many of the paradigms that open source programming has been using for years. Good for Microsoft!

    Final thought regarding code generation. I think PHP is a domain specific language in many ways. It certainly has a reduced problem space. I think code generation is possible when there are good foundation code. That is why things like your ScriptServer are so interesting and important. They become a target for code generation. This is especially useful for the nasty problems like keeping PHP and Javascript talking together where a code generator can make the programmer’s life much easier.

  • culley

    I liked to book as well. I found his “emacs is clearly the best editor in the world” attitude blind to the fact that editor choice is a *preference*, but other than that it was mostly a good read.

    culley

  • James

    I find the phrase “tool chains” to be interesting – do they really bind the programmer?

    Right now, software programming is a job of craftsmanship, not production. I don’t know that it will ever have a Henry Ford to automate the production. How can you turn intent into code? You can explain (pseudo-code) the intent, then turn the pseudo-code into real code – but that’s what interpretive languages are supposed to do anyway.

    I think, that in the end, Software coding will be more along the lines of the medical profession (we learn a lot of basic rules, memorize a TON of facts, but in the end are free to excercise our good judgement) than a ‘Software Factory’.

    Project Management will help define the intent, but coders will always have to turn the pseudo-code (read: specs) into real code.

  • http://www.phppatterns.com HarryF

    Not sure I’d underestimate Microsoft – I’m sure the TEAM version of Visual Studio will be excellent. Just would rather see them sell it on it’s own merits, rather than trying to invent a new “theology”.

    Back to The Art of Unix Programming, another part which echoes in PHP is here;

    The wrong way to get to a minilanguage design is to extend your way to it, one patch and crufty added feature at a time. On this path, your specification file keeps sprouting more implied control flow and more tangled special-purpose structures until it has become an ad-hoc language without your noticing it.

    Wonder where we’ve seen that before in PHP? ;)