12 March 2007

Having Fun With Erlang

It seems that the times when programs run sequentially using one main processor are finally getting to an end… Not only more and more computations get so complex you need to use a whole network of computers to do them, not only computer clusters are increasingly popular, but also our own home computers finally have become somewhat “parallel”. As we reached the border where we can’t tune the processor frequencies so much, chip vendors seem to look for hope in multi-CPU, multi-core circuits as in Intel Core 2 Duo for example. This has multiple benefits, but also raises one big issue - the troubles with writing programs that use the multiple chips or even multiple computers efficiently. At the same time, the Web still grows, and we have more and more systems where hundreds of thousands of people interact with the software in real time (examples vary from MMORPGs to web servers).

Now, have you ever tried to do a different thing with your left hand and a different one with your right at the same time? Ask somebody playing a piano how hard it can be without a good amount of practice. Most of the time, our brain isn’t too good at doing more than one thing at once - no wonder we have also problems conceptualizing computer programs that do multiple things simultaneously. The only thing we can do is use our intelligence and write tools that will ease the job. Here also lies the biggest problem - most of our tools for writing concurrent applications are dumb and more often they cause a lot of confusion than help us build better software. When you say “concurrent” in the presence of a programmer, he will probably have those four associations - Java, threads, deadlocks, headache.

Threads as most of us know them suck big time, even on a conceptual ground, as a model of computation, what to say about the practical realizations. It’s hard to get something non-trivial done with them, it’s hard to prove the correctness of a thread-based computer program and when later something goes wrong it’s sometimes almost impossible to find the bug and even harder to remove it. Of course, it’s possible to write correct and efficient concurrent applications like this, but it requires a lot of knowledge and skills from the programmer, which he could otherwise use to work on the actual program logic.

Actually, the problem does not only apply to threads, it’s also the paradigm of the most popular languages that makes concurrency a serious issue. It’s hard to write a parallel program when state is shared between multiple processes running at the same time and in commonly used imperative languages there is a lot to share. That’s the chance for Erlang. It’s the only relatively known language that tackles concurrency in a different way. First of all, it is declarative, so there is no state to share. The declarativeness will strike you from the very beginning - once you “assign”, or rather match a variable with a value you cannot change it in Erlang, so most of your Java/C++/Python/Ruby tricks won’t work and you will have to learn a different way of thinking about programming.

The basic unit of Erlang concurrency model is called a process. Processes are a bit like threads, but they can communicate only using messages, there is no other way they can share anything. They are also very lightweight, so you can have a whole lot of them without worrying too much about hogging resources. Processes are also a way to model real-world objects, as there is no traditional OOP in Erlang. Each process has a unique id, through, and you can easily replace one process accepting a set of messages with another one accepting the same set as a form of polymorphism, so you can think of processes also like of a substitution for classes.

As you see, even when you forget about the parallel stuff, Erlang has some interesting traits, and we didn’t touch everything yet. You will find pattern matching, higher order functions, list comprehensions and other weird stuff in the language. Besides, it has some commercial support behind it and they were some serious projects at Ericsson using it, unlike many of the other languages that are theoretically interesting in some way. The libraries are quite big and well enough documented (at least for my standards) so there is a solid base for doing something useful. By being a declarative language it is also totally different from the stuff you usually code in “software factories”, so it also seems to be fun and challenging to learn.

I tried it a month or two ago, mostly by accident, researching options for writing a backend server needed for my job. I didn’t have to write it in the end, but instead I found some nifty and surprising stuff in Erlang and got into it a bit. One more unusual thing that strike me was the runtime environment - it’s very powerful and you can do a wide variety of things using it. Its something between an interpreter, compiler, debugger etc. all driven by a single shell. The system has also the advantage Steve Yegge blogged about - it doesn’t have to reboot. You can hot swap a piece of code without problems while the rest of the system is running. When one of the processes fails, the rest can continue to operate perfectly, like you can see in the Erlang movie. Like most good languages, Erlang has also a Emacs mode bundled ;)

There is so much new stuff to discover here, that I feel a lot of fun trying to explore it - the last time I felt like this was when I started learning Ruby a few years ago. Of course, the field where you can apply it in “real world” maybe isn’t that wide, but the fun factor makes other things less important…

If you think Erlang may be interesting, check out some more resources:

The Problem With Threads (through Algorithm.au) - A in-depth paper explaining why traditional threading sucks

Erlang Homepage - with a complete reference, tutorials, programming guidelines, the interpreter etc.

Erlang In Real Time - A free, quite comprehensive book about Erlang

Thinking In Erlang [pdf] - A short (30 pages), but really nice tutorial on the basics of Erlang

Joe Armstrong’s thesis - The best introduction to the language you can get for free, if you don’t mind the formal, scientific tone

Programming Erlang - A new, in-progress book from the Pragmmatic Programmes about Erlang, you can download a beta pdf right now

Erlang For C, C++, and Java Programmers - Another short tutorial, aiming to teach Erlang quickly to people with experience in more traditional, imperative languages.

Erlide - An eclipse plugin for Erlang

Yariv’s Blog - A weblog mostly about writing web applications in Erlang

Why I choose Erlang - An interesting rant about programming languages

TrapExit - A wiki about Erlang, with lots of excellent practical articles

Comment