An Interview With Jim Weirich

I’ve been familiar with Jim Weirich’s name for a while; among other things he wrote the “rake” tool which most of us use on a daily basis. Then I was lucky enough to see Jim do a presentation at GoRuCo this June, when he explained some of the more advanced features of rake. I was immediately struck by how Jim was able to explain a very complex topic in a natural, straightforward way. Later this Fall at RubyConf I saw Jim gave an amazing keynote address that derived the Y-Combinator from first principles, explaining some of the basic ideas behind Lambda Calculus along the way. This time he not only clearly explained an even more difficult topic, but was able to make what would normally be a dry, mathematical subject very entertaining.

This month I was thrilled had the chance to interview Jim for RubySource; it was a great opportunity for me to learn more about him and how he approaches public speaking. We also had a chance to talk about how he got started as a computer programmer, why he learned Ruby, functional programming, Ruby’s threading model and also his new RSpec-Given framework. I’ve typed in the interesting parts of our conversation here – take a few minutes to learn more about one of our industry’s most innovative and charismatic thought leaders!

The job description of a computer programmer

Q: How did you get started as a computer programmer? Why did you choose this profession?

My very first introduction to computer programming was back in high school, back in the early 1970s. I got a book from the library about computer programming that described a BCD style computer. You programmed it in more or less assembly language. It was kind of a bizarre thing to program a BCD computer in assembly language, but I got into that.

Q: What does BCD mean?

BCD, “Binary Coded Decimal,” means all the numbers were
stored as decimal numbers, not binary numbers.

Q: I’ve heard of that before, now that you explain it.

It was a very unusual machine architecture, but it was appropriate for a beginner, and appropriate to learn a little bit of programming with it. I thought: “This is kind of fun!” So when the career counselor came around I thought this might be a career I was interested in. And then I read the description of a computer programmer, and it sounded horrible!

Q: What was the description?

It was something like: “… You’re all alone, sitting in the dark! …You never talk to anybody during hours of hard work!” I thought that this programming stuff might be cool, but a real job – I’m never going to do that. So I forgot about it.

Later in college my advisor said: “Why don’t you sign up for this Introduction to FORTRAN course; it sounds interesting and who knows you might like it.” Then I went to the class, and the instructor went to the blackboard and started writing down code, explaining it as he went. I remember that very first function he put up there was called “member.” He was going through it, explaining how it worked, etc., when I thought: “This must be some strange dialect of FORTRAN… it has way too many parentheses!” It was Lisp! It turned out I got into the one and only programming course taught by Dan Friedman, who wrote the Little Lisper.

Q: And what about the job description of a programmer? Were you afraid of working alone in the dark all the time?

Actually, I find programming to be a very social type of activity. The programmer sitting alone in the dark, I think, is not necessarily true. Today if you’re working alone in your basement, you’re in contact with people via Twitter, over IRC, over all kinds of things – github issue lists, mailing lists – you’re always interacting with people. It’s not the lone wolf kind of thing that it was portrayed to be in the 1970s.

A happy discovery

Q: How did you get started with Ruby?

I was working in C++ and Java at the time, and I also did a lot of Perl to do tooling. You know, all those little scripts that every programmer writes to make his job easier. This would have been in 2000 I think. I was really getting frustrated with Perl. Although Perl was really great to do your quick hacks, it did not seem to grow very elegantly. You could not do nice abstractions with Perl, at least not easily. If you needed a list, Perl was awesome – but if you needed a list of lists, then Perl was kind of clunky.

I was looking for something else. I wanted something that would express abstractions a little more, and since I was an object oriented programmer, I wanted my scripting language to support that kind of thing. I looked really, really hard at Python. This was the third time I made an attempt to learn Python. But it wasn’t like “Bam,” switch over and I’m ready to run. I had to stop and I had to look up everything I was doing in Python. I never stuck with it long enough to make it second nature to me.

In the middle of this third attempt to learn Python, I saw an email from Dave Thomas – I don’t remember exactly where it was – and he said: “we discovered this little language called Ruby and we really like it. And you might like it too.” I had just finished reading The Pragmatic Programmer and thought: well, if Dave Thomas says this is a good thing then I have to check it out. So I downloaded Ruby and got it working and tried it out for a few minutes, and thought: “This is it! This is exactly what I want.” And within 3 days I totally stopped writing Perl scripts and starting using Ruby instead.

Q: What version of Ruby was this?

It was Ruby 1.6. This would have been in the Summer of 2000.

Q: And what was it about Ruby that made you decide this was exactly what you wanted?

It did things in the same way that Perl did, but in an object oriented way. So I would sit down and think: “Well, I would do it like this in Perl, but if I had objects I’d want it to work like this.” And I would write it in Ruby and it would just work! I was immediately able to use it for all those quick and dirty things that I used Perl for, and then I discovered that Ruby has great abstractions, and grows very nicely into a real language as well. It was a happy discovery for me.

Is Ruby a functional programming language?

Q: So Ruby is an object oriented language. Is it also a functional programming language? And what does that mean?

That’s a good question, because there’s a lot of attention on functional languages these days. In fact, we’re having the Cincinnati functional programmers meeting tonight. I told my wife I was going to stay late for that and she goes: “Ok – and tomorrow night you’re staying for the non-functional programming meeting?”

There are two things that make a language a functional language. The first thing is the ability to manipulate functions as first class citizens. You can pass functions around; you can bind them as variables; by using closures you can create new functions of them, etc. That’s the primary requirement for being a functional language. In that regard, Ruby certainly meets that requirement: we have lambdas and closures; we can bind them to variables; we can create new ones dynamically. It is a functional language in that sense.

There’s a second requirement, though, that I think makes functional programming really interesting. This is that functional programming tends not to use state modifying semantics. And Ruby fails in this regard, because we modify state all over the place!

What makes functional programming interesting to me today is the idea of concurrency. What makes concurrency hard is that if you have two things happening at the same time, and they are working on the same piece of data, you get into what’s called a race condition. You don’t know which one is successful. For example, take something as simple as incrementing an integer – two tasks doing that, one task goes in and fetches the value, increments it in a register and stores it back. At the same time somebody else is touching that value, incrementing it in a register and storing it back. If they interleave in exactly the right way, if you start with 10 and two tasks increment it by 1, then you can end up with 11 rather than 12, just by the order of events. So you have to be very, very careful when you have shared data that is mutable.

There are two ways out of this problem. One is not to have shared data. The other is not to mutate shared data. And functional programming takes that second approach. You don’t mutate data; you only generate new data, in essence. Therefore concurrency is trivial in a pure functional language like that. I think in today’s technology environment, as you have more and more CPU’s on your laptop, threaded code, concurrent code, is going to become more and more important. That’s why functional programming is so interesting. A language like Clojure has really, really good constructs for easily managing multiple threads, and not having the threads clobber each other and cause race conditions, because you don’t modify data. At least in Clojure when you do modify data you do it in very, very controlled circumstances, when it’s safe to do so.

I think Ruby is suffering from that, because its threading model is out of the 1960s and 1970s.

Ruby’s Threading Model

Q: Right; especially MRI Ruby. I think the interpreter itself is not thread safe. Is that correct?

The interpreter is thread safe, but they made it thread safe by introducing the Global Interpreter Lock (GIL). At RubyConf they talked a lot about the GIL. Essentially what that means is when you enter the interpreter, it sets a lock so that other threads cannot also enter the interpreter at the same time. It’s makes it thread safe, but it also makes it difficult to get useful threading out of a situation like that.

Q: However, other Ruby interpreters, like JRuby and Rubinius, are thread safe, right?

Yes! They do their locking at a much smaller level. The thing is that if you design from scratch to be thread safe at a smaller level, then it’s not hard to do. Adding in thread safety after the fact is what was really hard, and that’s what the MRI developers are dealing with. They have to add in thread safety after the fact, which is why GIL is an easy solution.

I did hear Matz say they did a version that uses “microlocking” – that is, locking when you’re actually modifying tables that are shared. He said it ran much slower than the GIL version. So do you want a slower Ruby where you have better threading, or a faster Ruby where you have the GIL. That’s a tradeoff you’ve got to make. So if you’re really looking at threading environments, then JRuby and possibly Rubinius are things that are worthy to look at. Even so, in both of those you’re still looking at shared mutable state, and Ruby just does not have, at this time, a really good construct for dealing with that.

10 Papers Every Programmer Should Read

Q: It’s interesting how sometimes what seems to be a new, modern idea is actually based on research and thinking done a long time ago. For example, the functional programming ideas we just discussed, or Lambda Calculus which you talked about in your keynote presentation at RubyConf.

That’s 80 year old stuff!

Q: Do you have other examples of that sort of thing – old ideas that were pulled into something new?

There’s an excellent blog article by Michael Feathers from 2009
called 10 Papers Every Programmer Should Read (At Least Twice) [author’s note: this is a link to a list of the 10 papers; Michael’s actual article, originally posted on the Object Mentor web site, is no longer available online]. Let me just point out a couple of the papers from that.

The first one is: On the criteria to be used in decomposing systems into modules by David Parnas. He took a very simple program called “keyword in context” where you go through a text file, find all the words, find out where they are used and record the context of where they are used so you can display them in an index, with the words surrounding them. This was actually an experiment. He wrote the system in two different manners, and compared what happened to them when he introduced changes to the requirements. What he found was that the best way to break down the program into modules is to have each module hide a secret.

Q: Is that one of the ideas behind object oriented programming?

It is one of the ideas – that every object contains information, implementation details, that does not matter to the outside world. You can change those details without effecting the outside world. This very basic research really highlighted that this is what we really want to do. That’s one of the papers I really recommend reading. What was interesting to me was that he actually performed the experiment. He actually wrote the code both ways and then measured how much code he had to change to get this effect.

Another good paper is: Can Programming Be Liberated from the von Neumann Style? by John Backus. This is one of the very early papers on what he called “ML,” which is a functional programming language. ML is the foundation for things like Haskell and OCaml.

There’s also an awesome paper by Ken Thompson that just blows my mind. It was done in 1984, and is called: Reflections on Trusting Trust. He describes how to hack a compiler so that he can insert a bug into the Unix system that is not in source code anywhere.

Public Speaking

Q: I think you have a natural way of making very difficult topics understandable. How do you boil complex topics down into something that makes sense to a general audience?

I have read a good number of books on presentations, on how to do presentations and how to prepare presentations. And the books tell you to make outlines and things like that – that just does not work for me!

What I like to do when I’m preparing a talk is to use a whiteboard, or maybe something like Omnigraffle on the Mac. I’ll start drawing little boxes with ideas in them. Then I’ll start drawing lines between all the ideas. So if I’m doing a talk about Ruby, I’ll list all the things about Ruby that I want to talk about. And I’ll draw lines between them – kind of like “mind mapping.” Except my mind map is more like a directed graph.

So I’ll get all the concepts on a piece of paper and then I’ll try to find a story that will walk through this graph and hit all the topics. Not necessarily in the order I’ve drawn arrows relating the ideas, but that covers them and makes an interesting story out of them.

Q: So you’re sort of a technical storyteller? Is that the right way to describe it?

I would like that description, actually!

Q: To make what sometimes are very dry topics, like Lambda Calculus, into an interesting story to tell is quite an art. I think most of our eyes would glaze over if we tried to read the original academic papers on Lambda Calculus from 80 years ago.

I think the key to doing a good presentation is to be excited about the topic. If you are excited about that and you let that excitement show through to your audience, then they will get excited as well. I can’t tell you how many professors in college I had that would speak in a monotone voice. It drove me crazy. I said if I ever get up and speak I’m not going to do it that way. I’m going to be excited and I’m going to let me excitement shine through. I think that is the real key. All the preparation beforehand will aid that, but if that excitement is there then you’re going to show it.

rspec-given

Q: Do you have any new presentations coming up in the near future?

In January I’m going to be talking about the Given/When/Then framework that I did. I’m really excited about that. Every time I go to do testing now I start using that framework. It expresses tests so elegantly. I’ll be speaking at CodeMash about that.

Q: Does it have anything to do with Cucumber and the BDD philosophy? Or is it just coincidence that you are using the same three keywords?

It’s a coincidence; there’s nothing really directly related to Cucumber. When I use Cucumber I really like the way they did the Given/When/Then thing.

With RSpec-Given you say: here are all the things to setup. This is the code you’re testing and this is what’s true after you run your test. That is, in my mind, an excellent way of specifying tests. Even when I was using Test::Unit I would often break up my test methods into three sections: here are my givens, here’s my code under test, and here’s my assertions. But I really wanted a way of expressing that very clearly, so that it was very clear what was given, that it was very clear what part of this code was being tested. And I think that’s missing a lot in some of our very procedural test frameworks. Test::Unit and even RSpec doesn’t break that out very clearly in my opinion.

About two or three years ago I was thinking about this, and I started jotting ideas down on paper, and I was at the Ruby Hoedown in Nashville at the time, and a bunch of us EdgeCasers were sitting around at the back and I passed this paper around the table, getting comments on it, and Joe O’Brien turns and says to me: “You’re not really writing another test framework, are you?”

I played for these ideas for about a year or maybe two years before stumbling upon a really trivial implementation in RSpec that worked out really, really nicely. I think that says a lot for RSpec that had the right abstractions and supported exactly what i wanted to do, very, very closely. What we end up with is just a very, very thin library that sits on top of RSpec. You can use all your RSpec knowledge you’ve gained beforehand but can then add this stuff in for just a little better clarification on your code.