Common Lisp and the Towers of Blub

So I’ve been teaching myself Common Lisp. It’s a slow process because I don’t get a lot of time to do it, and also I’m a bit slow. The reason I’ve been teaching myself? Well I’m wholly dissatisfied with C++, Java, C#, and to a much lesser extent Python. I still love C though. She’s my lady. We connect.

The dissatisfaction started after I read Paul Graham’s “Hackers and Painters” book. I wrote about it too. Whilst reading the book I came across this paragraph and identied myself as a card-holding member of team “Blub”:

As long as our hypothetical Blub programmer is looking down the power continuum, he knows he’s looking down. Languages less powerful than Blub are obviously less powerful, because they’re missing some feature he’s used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn’t realize he’s looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.

Since then I’ve been on a journey. It’s taken me through books, well a book. I’ve discovered that Emacs is like 1 million times more powerful and sexy than her slightly musty smelling older sister Vi, with whom I’ve had an almost 15 year relationship and am trying to get out of my head. :x!

If you don’t believe me about Emacs (and why should you) consider how you’d write a video editing suite in whatever code editor you use or can extend. I already knew long ago that Emacs is at least partly written in Emacs Lisp but I should have guessed that arguably the most productive Common Lisp development environment, or IDE, is also an Emacs extension called SLIME.

Soon after that I discovered some fantastic screencasts in the Ruby On Rails mould for stupid people (like me): SLIME, Hunchentoot, and Selenium. BTW, I know Selenium is not Common Lisp but the screencast presents a Common Lisp binding to the Selenium Remote Control server that makes it possible to use Selenium from Common Lisp as a testing framework.

So it feels like I’ve come along way. But today, your humble programmer-of-Blub discovered something on the HyperSpec that scared me a little. Through the PlanetLisp RSS feed I discovered that I am also really crap at mathematics. I did this by trying some of these puzzles. In attempting to solve puzzle #2 in Lisp I found I needed to brush up on some Lisp basics. After some digging, and through a series of HyperLinks, I ended up here and before that here:

;; This illustrates the use of SETQ on a symbol macro.
 (let ((x (list 10 20 30)))
   (symbol-macrolet ((y (car x)) (z (cadr x)))
     (setq y (1+ z) z (1+ y))
     (list x y z)))
=>  ((21 22 30) 21 22)

I think I stared at this code snippet for about 2 minutes before it finally made sense when I at last remembered that setf works on a place and not a symbol. Now, I’m guessing an experienced Lisp-er would look at the sample and say ‘so-what’? I’m guessing that non-Lispers would say, anything that looks that complicated must be bad.

And this is when it struck me again. And this time it hurt. I may have lost a little “Blub” to get this far but I still have way much more “Blub” to lose.

Continuations and Exceptions

I very often find myself tripping over ‘old’ ideas in software technology and discovering a rich topic that I had no idea even existed. This weeks elephant in the room is the ‘Continuation’ language/compiler feature.

Let’s take a quick look at a simplified version of an example of a continuation that I found to illustrate the point.

irb(main):041:0> def ab
irb(main):042:1>    puts "a"
irb(main):043:1>    callcc { |continuation| return continuation }
irb(main):044:1>    puts "b"
irb(main):045:1> end
=> nil
irb(main):046:0> x = ab()
=> #<Continuation:0xb7c5ee84>
=> nil

The callcc function accepts a closure as an argument. This closure accepts the ‘continuation’ and we immediately return it from the function. The continuation contains all of the state that got us to the point before we exit, variables, stack-frames, the lot. The key is that the continuation is itself a closure and when it is called the code will continue from the next line. I think it’s easy to see that this feature has pitfalls. It’s like a GOTO but without the benefit of having a line number to lookup! This is because on the outside every continuation looks pretty much the same. But this approach does have arguably real tangible benefits in Web servers like Seaside.

So what?

Well it turns out that exceptions in C++ derivative languages are a stupider half-cousin of continuations called: escape continuations. Thinking about it the similarity is striking and not all that surprising. The reason I say they’re stupid though is because exception objects are not re-entrant in the same way as continuations are, and the context that is captured from the exception is defined by the programmer. However, it’s plain to see that the exception handling we have in most modern programming languages like C++, Java, C#, etc are a diluted form of continuation.

So what?

Well I’m not totally convinced of the practicality of continuations in web programming environments, partly because I’m a bit old-school when it comes to performance and those languages that have web-frameworks and support continuations (Smalltalk, Ruby, PHP) are slow-as-molasses. The other reason I’m not keen on the idea of continuations is the practicality of working with them, i.e. they are a variation on the GOTO in that it’s very hard to follow the flow of the program by just looking at it and reading it. But I know that one day I will discover the need to make sparing use of a continuation and be glad that I knew that I could do it.

If the truth is ever told, I think exceptions suffer from the curse of the GOTO as well. And I think the reason for that is partly because the exception mechanism as implemented in Java, C# and C++ is a teensy bit broken. The reason, again, is that the flow of control jumps up the stack to a potentially unknown place. I say unknown because if you write a library that throws an exception and after you release it you change the contract of how that exception is thrown, well then you’re in trouble.

Trouble, that’s what

The catching of exceptions is based on type, and inheritance rules apply, so the inheritance hierarchy of the exception classes you use affects the flow of control. This is clearly the intention since you use the type to differentiate a NullArgumentException and a NullPointerException and how you deal with these will probably require different code. However, if the handling code was the same well then I why couldn’t I catch the base class instead and have one code-block? This is the temptation and the danger because if someone else decides to rejig the exception hierarchy you might not catch the exception you want. Worse still, you might end up catching exceptions you don’t expect and providing incorrect handling code for them.

So what?

Well like Peter Parker says:

Whatever life holds in store for me, I will never forget these words: “With great power comes great responsibility.” This is my gift, my curse. Who am I? I’m Spider-man.

And whilst we may not be Spidey we need to recognize the gifts and curses that 50 years of programming languages has given us … oh, and we should take note about the power/responsibility thing too …

Paul Graham wants me to think he is mad.

Paul's book I’ve been reading Paul Graham’s Hackers and Painters. Published in 2004, it only just found its way off of my reading shelf and into my brain. This Amazon reviewer really sums up the book nicely although I mildly disagree with some of the statements.

Most of the book is a long-winded lecture about how great Lisp is, and about how that is the language of the cool, smart people.

There is very little about painting, it is only briefly mentioned in the beginning.

The first chapter is quite good, then it gets more preaching and more dull rapidly.

Mr Graham is obviously a smart guy and a capable writer. The fact that he was part of a dot-com start up that actually succeeded seems to have gone to his head though. Somehow the fact that he was in the right place with the right idea at the right time enables him to declare Lisp as the uber language, and everybody who doesn’t see that is a dullard?

The title suggests a book which is whimisical and fun. This book is a preachy diatribe by a pompous hacker who things he has the proper world-view for everyone.
— Kevin Stokes

But to give Paul his due he covers some excellent topics. One of the most intriguing ones is the third chapter about thinking the un-thinkable. The suggestion is that by thinking the un-thinkable you will enter a new head-space where perhaps you can make a concerted difference to yourself or others. When I read it, I thought: “I like this idea, I might try it out”. A few more chapters passed and he is talking about Lisp (which I will return to shortly) and I’m thinking: “I’m bored. Where’s my unthinkable thought got to?”. I thought really hard and I got one: “Paul Graham is mad”. Plain and simple. As instructed by Mr Graham I started to explore this idea right away to test its validity.

  • If Paul had a mania what would it be?
  • Where should we look for manifestations of this mania?
  • Will trying out Lisp make me as mad as I perceive all the other people that use it to be?
  • Am I mad and if I am would I be able to answer this question?

And I came to the conclusion that Paul Graham is not mad. Sorry folks. It’s just not true. However, the majority of his un-thinkable ideas (at least in this book) all seem to originate from a single perspective point (you get these in art too I understand!) that choosing Lisp allowed him and his colleagues at Viaweb to produce something others could not. Which is in itself a remarkable statement if it is as true as he suggests. So I thought I’d have another try at Lisp. I tried it once before and wasn’t totally turned off by the idea I just never really got going.

Now don’t get me wrong. I’ll not be trying it just yet. This is just a warning you understand, especially since I wouldn’t want Paul to think that he’d succesfully goaded me into it when he said:

… but I don’t expect to change anyone’s mind (about Lisp) over the age of 25 …

I reserve the right to not let Paul Graham into my head and eat my brain. It’s mine and I’m keeping it mine until I decide otherwise.

One of the cricisms of Lisp is that it doesn’t have very good library support. A little investigation into Unit testing frameworks and MySQL wrappers seemed to bear out what the Reddit development team are saying.

What makes Lisp so much greater (it seems) is that data and program are inter-changeable in as much as you can treat pieces of program like they’re data and vice-versa. And it’s difficult to conceive of a programming language that has the two concepts so fully inter-twined. Reflection is a way of achieving it but in my view reflection is evil because it makes compiler optimisations and function call traceability harder. These two arguments don’t apply to Lisp so Lisp wins again. It is partly because Lisp has all those brackets that it is so powerful.

And so we link into another one of Paul’s un-thinkable ideas which is to design the programming language of the future. His belief is that by imagining what we will want we can make this language today. Anyone can do it. And probably either has or will, we just don’t know it yet. He believes that this language will be optionally OO, with very few axioms, and support something like macros. Smells like Lisp to me. Change the meat on the barbecue Paul.

I refuse to believe that Lisp is the future. It would be quite extraordinary and quite exciting if it were to become so, but my crystal ball says no-way padre. But it is one of the languages in the evolution of languages that has played a significant role. And I do believe it has more to offer. Its time will come again and some spark will incorporate the final pieces of high-hanging Lisp fruit into the language of tomorrow.

It’s just that we’ll be able to do it without having a permanant speech impediment. Thhhorry Paul.