Language of the gods? C.
Sunday, 18 February 2007, in categories: Programming
When you look at popular articles/posts about programming in the last few years, one topic comes back over and over and each time its extremely popular - the “enlightenment” that comes with learning Common Lisp, Scheme, Haskell or sometimes yet another not-so-mainstream language like Smalltalk. Recently people started even bragging about the “language of the gods”. I think the thing that started the buzz was the “Howto become a hacker” text. But I didn’t seen any particular too much wonderful Lisp applications written by ESR…
After thinking about it for a while, come on, it’s total bullshit. Someone can conclude from those articles that learning macros, continuations, monads or whatever will turn one into a alpha programmer, or a super-human. Yes, those are all interesting concepts, they stretch your mind, you get a better programmer as you learn it. I greatly appreciate it and I try to learn things like that myself all the time. But don’t get trapped into believing that this is THAT important - there are other as important or even more important fields no-one is blogging about. Its also a lot like saying that just getting a new guitar will make you a better guitarist. The language is just a tool, maybe it’s time to stress more other topics like problem solving techniques, good design skills, which are current all the time.
The next time someone will talk about the unequalled expressive power of Lisp or Haskell, take him here. Those simple benchmarks show some interesting tendencies. Our old, good, not so tough to learn, not so “enlightening” Ruby language often ends up with shorter programs than those written in any of the “languages of the gods”. And you don’t have to study Ruby for 10 years and have a PhD to became proefficient in it. In fact, when learning Lisp I found a lot of the basic language constructs well known from Ruby (ie. mapcar, dolist, dotimes). They are also macros, which aren’t available in Ruby, but I’m not exactly sure if I feel too enlightened by them. Well, maybe I didn’t get it yet, but as for now the real value came rather from understanding the fundamentals, the patterns Lisp/Scheme programmers use and analyzing programs or exercises from books like SICP. But as soon as I learnt them I could apply the most important ideas using any language.
Another things is that when I started programming as a kid I wanted to write programs that I could use myself - I wanted to create games, a music player, a text editor, that kind of stuff. Now, lets take a look at the languages programs I use were created with (as far as I remember, correct me if I’m wrong):
C - Linux itself, XFCE (which I replaced Gnome with recently), Emacs (an EmacsLisp interpreter written with C with a set of macros written in Elisp) Sylpheed, Rxvt-unicode, Zsh, coreutils, Subversion, Audacious, Mplayer, Ruby, Apache, Nginx, PostgreSQL, Exim, screen, GIMP
C++ - Firefox, OpenOffice, Inkscape
Things I use written in other languages - none of the utility programs I use is actually written in another language than C/C++ (if you don’t count the Elisp part). The only other non-C part of my system, is Portage, the package system in Gentoo Linux, written in Python.
Sincerly, I can’t think of an application in Lisp, Scheme, Haskell, Smalltalk or OCaml which I would like to use and I tried out a lot of applications in my life. What are all those “gods” programming? And its not only about open source or Linux, I doubt you find something really interesting written in Lisp which isn’t a Lisp programming tool. If you play games, most of them also probably are written in C++, sometimes with some scripting engine on top of it. Web server? Compilers? Operating Systems? To program anything interesting, you need to know a whole lot of C.
Take a look at Linus Torvalds, when he found himself in the strange situation with BitKeeper. He just smashed out a distributed version control system with a first prototype in a few days, then people started developing it futher and now, not so much time passed and git is from what I’ve seen a really, really great VCS. And it was created with the same old boring C, not with any of the hot, enlightening languages. “Beating the averages” isn’t only about using a “better” hammer, but also about knowing how to use it efficiently.
Thus, in the next few years, I would like to became proefficient with 4 languages - C++ (because if you know C++ well you shouldn’t have troubles with C, and it doesn’t work vice versa), Ruby, Common Lisp and JavaScript (which I should have start mastering one year ago or earlier). This way I will have something for expanding my mind and learning (Lisp - a lot of interesting books use it for teaching compiler/interpreter design, AI, etc.), something for getting shit done and making it run fast (C/C++), something for everyday scripting and writting things that don’t require that much speed (Ruby, which is both practical and mind expanding, but slooow and hogging memory) and something for the Web (JavaScript). The D language, Ruby 2.0 or Perl 6 could switch things around a bit, but no one knows when they will be release (with Perl/Ruby) and if they gain enough publicity to became useful. And they won’t bury C anytime soon.
Comments ():
seban, 18 February 2007, 9:05 pm
I’ve read some time ago about next specification od C++ . There should be some improvements in C++. It was written in Software Developer Journal ona year ago. If you wish I have pdf with this text somewhere (in polish).
I would like to know better C/C++, Java, Ruby, and start to learn JavaScript and maybe Python.
PS. Your english texts impel me to get better my (poor) english. ;-)
stiff, 18 February 2007, 9:31 pm
Are you talking about C++0x? The release is scheduled on 2009, I wonder if it will have any impact so late…
js, 18 February 2007, 10:18 pm
You may not see it, but nothing will bury C/C++ because it’s already six feet under, when it comes to application programming (not OS or VM programming). There’s a big deal of legacy apps from earlier days, so those languages are still good to know, but that’s sorta paliative care, like it is with Cobol for the last 15 years.
Of course, some new applications do get written in C, but that does not defeat the prominence of more modern languages, like Java, C# or Python.
Kartik Agaram, 18 February 2007, 10:18 pm
“I can’t think of an application in Lisp, Scheme, Haskell, Smalltalk or OCaml which I would like to use and I tried out a lot of applications in my life.”
Have you tried darcs? It got me into haskell.
mrlachatte, 18 February 2007, 10:31 pm
Darcs was written in Haskell.
Ricky Clarkson, 19 February 2007, 12:05 am
Ruby itself could be considered a language of the gods too - it is often likened to Lisp and to Smalltalk. I think that C, C++, Java, etc., are the languages of the commoner, but any decent god knows how to program some of those.
Ultimately, you can, thanks to Turing-completeness, do anything in any language, so talking about specific applications isn’t that useful.
C doesn’t have to be buried, there’s no harm in it continuing to exist, as long as the relevant programmers (application programmers, etc.), know about the higher level constructs. A problem with C is that even application programmers often think like (amateur) kernel developers when writing in it - of course, this is a social problem, not a technical limitation.
A lot of the appeal of macros, of dynamic typing, of excellent static typing systems like Haskell’s, is that it reduces code duplication. Having the drive to get rid of duplication is an important trait of any good programmer, and when using C, there are some necessary duplications. This is one reason not to be happy with C, but it doesn’t make C useless. It’s better to know you’re duplicating, and to know how you would deal with it in a more capable language, than not to notice that anything’s wrong.
In short, being unhappy with your code is good.
Shaper, 19 February 2007, 1:05 am
“Its also a lot like saying that just getting a new guitar will make you a better guitarist.”
It will. If you learn on a one-string guitar then learning a five-string guitar will make you a better guitarist.
It’s just that we already use the “guitar design of the gods”.
FWIW, the “language of the gods” doesn’t have to be the language that everyone uses, just the language that is technically the “best” or most mind-(or ability)-expanding.
Lisp is, many people believe, the most sophisticated language for expressing algorithms - just because it’s also hard and (historically) inefficient doesn’t impact its quality as a medium for expressing algorithms.
Michał Kwiatkowski, 19 February 2007, 4:02 am
Desktop applications are just a small subset of software, so the fact that you don’t use those written in languages other than C or C++ doesn’t prove anything. Games were also written in Lisp. One good example was Crash Bandicoot by Naughty Dog group (http://lists.midnightryder.com/pipermail/sweng-gamedev-midnightryder.com/2005-August/003789.html).
Power of Lisp lies in a fact that it is a programmable programming language. You can change it in a way you want, so you can implement literally *any* programming concept in it and it won’t look awkward. In general sense all languages you mentioned are Turing-complete, so you can achieve the same thing using them. This thinking may lead you to implementing everything in machine code. ;-) But if you look deeper during implementation of your application in C you’ll see that you’re actually re-implementing patterns which can be easily abstracted out in Lisp. Ruby is quite close to Lisp with its use of eval and open classes, so this aspect may not be very apparent to you.
As Ricky pointed out, a good programmer should work constantly to reduce duplication in his code, which is strictly related to the notion of defining new abstractions. A language of gods should provide you with tools to do unlimited abstractions. Languages other than Lisp have a wall, which eventually every programmer will step into. Perl, Python or Ruby allow you to express lot of abstractions, while C or C++ really suck in this regard, forcing you to think about irrelevant things all the time (types, machine constraints, etc). Be able to focus only on relevant things help you express your ideas faster and more precisely.
Having said that, there is more to programming languages than their power, like politics or habits. And, while there is an answer for a question which language is most powerful, there isn’t (and won’t ever be) a definitive answer to which language is the best.
Mike Cantelon, 19 February 2007, 4:50 am
Well said. It’s much easier for fragile egos to play in the sandbox of academic irrelevance rather than trying to create program software that helps people.
stiff, 19 February 2007, 8:15 am
Hey, I know that Lisp is powerful and I wouldn’t like to learn it if I thought it isn’t. I just want people to notice that in the end, most of the best software today is still written in C/C++ and the authors of this software are the “gods” of programming for me, not necessarily people known mostly for writing blog posts. And I’m not only talking about desktop applications, but about all the software I use everyday, together with interpreters, web servers, databases etc. Its still a subset, but it actually consists of the most interesting (at least for me) / most useful stuff.
There is Darcs, maybe even it’s good (I haven’t tried it), but still, two most popular VC systems (CVS/SVN) in the Open Source world are still written in guess what. Another counter-example that comes to mind is MLDonkey, but it isn’t that popular either. So, a bunch of mind-expanding, ultra-expressive programming languages (Smalltalk, Lisp, all the MLs, Haskell) and THREE applications someone even heard about, together with Crash Bandicoot. The rest of the software consist mostly of utilities for writing more Lisp, more Smalltalk, etc.
The reasons? C/C++ can produce small and fast applications that are easy to build and deploy, you don’t have to ask the user to install a interpreter / framework / virtual machine and a bunch of libs to make the application run. Also, I know that “premature optimization is the root of all evil”, but those days it would be better to withdraw this slogan. Back in the old good days I could play mp3 files on a 486/100Mhz with 64 megabytes of RAM, with Windows running on it and eating half of the memory. In the world of modern software, running a music player can eat you 256 megabytes of RAM, because the programmer forgot how to write good C/C++ code and used C# and five thousands of libraries. It was certainly easier for the developer, but the end result sucks for the user.
js, 19 February 2007, 10:22 am
Premature optimization IS evil, you seem to mistake it for no optimization at all.
Pah, 19 February 2007, 2:57 pm
if you don’t count the Elisp part
Riiight. Do you understand just how much of emacs is implemented in elisp? That’s like not counting the C parts of linux because it’s “really” implemented in assembly (asm shim needed to boot kernel, C shim needed to boot the Emacs elisp VM).
Michał Kwiatkowski, 19 February 2007, 3:17 pm
OK, I’ll say it again: popularity doesn’t prove anything. From this perspective, next to C you’ll get Java and PHP as languages of gods. I’m very surprised that you didn’t count Emacs in your Lisp applications summary. If every programmer would had to configure and extend Emacs in C it simply won’t be used. It’s Lisp that got Emacs popular.
C and C++ aren’t languages in which you can easily “build and deploy” applications. Compared to Python/Ruby/Perl number of hacks programmers have to do in order to make an application portable is really astonishing. System libraries differ and C programmers have to reinvent the wheel every time to do simple things like sockets or graphics. They end up with another layer of complexity, which have a high probability of having bugs and being actually slower than cleaner and portable solutions in more powerful languages.
I’m using Firefox, OpenOffice, Gimp, Inkscape and others as you do and I must admit they eat quite a bit of my system memory. They’re also not particularly fast. In fact I think they will benefit by being rewritten in language like Python or Ruby. Codebase will be smaller, bug will be easier to find, and benchmarks could be done more effectively, thus producing faster systems. The biggest problem in optimization is not how to optimize it, but when and what. Writing programs “for speed” in C is just stupid, bottlenecks are often not even in your application, but in a database or network connection. You’re seeing the problem but fail to understand the cause. I know you use Rails extensively, so listen to what David says about optimization: http://www.loudthinking.com/arc/000596.html . Watch Dave Thomas talking about it as well: http://video.google.com/videoplay?docid=-7327520943909344563 .
EOT from my side, don’t have much time for Language Wars nowadays. ;-)
stiff, 19 February 2007, 4:21 pm
We’re not talking about popularity, but about the number of wonderful applications written in various programming languages. If all smart programmers use languages like Lisp, why is the amount of great and useful applications written in it so small?
OpenOffice in Ruby certainly would not be too fast ;)
DHH is talking about writing web applications, which are an exception from what I’ve written about ie. nobody uses C to write them and you can use any language you want - if it looks good in the browser, no one will care about the internals.
To clarify - it’s not that I like C very much or consider it “a better language” (whatever it means) than XYZ, I’ve just made a simple observation that almost all useful stuff is still written in the same good old thing, not in any of the new hot languages.
Ondrej Svitek, 19 February 2007, 5:06 pm
“The next time someone will talk about the unequalled expressive power of Lisp or Haskell, take him [here].”
Ok, eat your own dog food and have a look at http://shootout.alioth.debian.org/debian/benchmark.php?test=sumcol&lang=all
Surprised, aren’t you? Haskell uses some very clever optimizations there. Sure, the same thing could be achieved in C, but you are left on your own and have to implement it by yourself. You can write some kind of library, incorporating clever ideas and optimizations, but that’s about all you can do in C. What you can’t (easily) do is to pass information how to generate these optimizations automatically for user supplied code in the library.
I’m not trying to imply that these microbenchmarks are of any real value, though.
For the Lisp part, I’ll give you an example of my personal macro usage:
I encounter lots of html parsing in my work. At the beginning, I didn’t have much experience of how to do it properly (heck, I didn’t even know regular expressions :)). Somehow I felt, that regexps might be helpful here. So I downloaded a library for lisp and gave it a try. Well, it worked. But I wasn’t very happy with it, because through extensive regexp usage my otherwise nicely structured programs started to look like a line noise. And I hate that, because readability and maintainability stand very high in my code production. I changed my view of the problem, because approaching it with regexps was unnecessarily low-level. To say it simply, I was ignoring one major feature of html code - its structure - and reimplementing ad hoc hacks for it time and time again, scattered all over my sources. That’s not nice, as I also love a tight locality of code. Nobody loves repair-here-break-in-another-file kind of problems, it just takes too much time and effort to track all these dependencies. It also frightens you to experiment with your code, which is being in bad health and fragile. But how am I supposed to program (= continual code evolution), when it is plagued like this? That’s what I thought, being tired of endless structure handling hacks. I made an obvious step and downloaded a library, which parses html from strings into a tree representation, to be able to do matching on that. What a relief! No more bothering with nasty regexp. Still, I had to reinvent a wheel once again. I had to roll my own tree matching routines. It was fun, but I got tired of it soon. My code started to look cloberred once again, as I was adding more and more specific routines. And none of them were easily reusable. Geez, having to think about such a straightforward thing as structural pattern matching? Maybe a few times OK, but hundreds of times? Hell, I have better things to do with my time. What kind of tool is good at this task? Prolog! Oh, yes, but how can it help you, when you have chosen to suffer in Lisp, you might ask. Actually, believe it or not, there are quite a few prologs out there, implemented as a bunch of lisp macros. I used one with a lispy syntax, so I wouldn’t be left out in the cold, should the need of macros producing prolog code arise. And it sure did arise soon! While being very nice and almost ideal for my usage pattern, there still was some boilerplate code present. Mainly some initialization and directing the style of matching (all vs. first match only, etc.). After writing it time and time again I gained an intimate knowledge of these inner workings, deep enough to write a program, that generates them for me according to a given matching pattern and a few switches. This kind of program is called a lisp macro. What it does is to take a lisp code as input and compute a transformed code as output (and beware, all of this is done on abstract syntactic tree representation of source, not strings). I won’t go in further details about it, just say, that I am *very* happy to have a handy macro like this around. I admit it does not solve every one of problems, but corner cases can usually be solved with nested matching and a little bit of lisp programming (as I have a great control over the matching while it is running). I have even incorporated regexp matching, but this time only where it is needed - matching of useful string data, not html tags. Now I can really concentrate on what’s really relevant for me to program and let the machine do the tedious work (a few simple lines of input are expanded into not so few lines of prolog, that are later transformed into impenetrable lisp code implementing all those prolog backtracking and unification stuff).
I could continue more and more, but I hope, you have an idea now. Ultimate power of Lisp for me lies in the ability to create whatever domain specific language would-be-nice-to-have *now*, I don’t have to wait for a compiler creator to incorporate some nifty feature (and I have no illusions they would implement my personal wishes).
I tried to be as personal as possible, because I make no assumptions of what is good for you. I don’t know. But what I know for sure is, that Lisp was (and still is) a great mind bending trip for me. Just my $0.02
(incf lisp-preaching-counter)
Ricky Clarkson, 19 February 2007, 7:29 pm
“We’re not talking about popularity, but about the number of wonderful applications written in various programming languages. If all smart programmers use languages like Lisp, why is the amount of great and useful applications written in it so small?”
Because there aren’t that many smart programmers, probably, in relation to the rest. An island in the Carribean might be the best place in the world to live - just because it isn’t highly populated or doesn’t export much compared to bigger countries doesn’t disprove that.
Kronikarz, 19 February 2007, 11:47 pm
Language of the gods, huh? Hmmm… That would be the language the “universe” was written in, innit? Ho hum… Also, the language that is the most powerful in terms of speed, usability AND with the most direct connection to the “universe”… Also, the language in which “gods” write…
Assembly? No, not very easy to use…
Lisp? A bit too abstract…
Ruby? That’s the slow one, right?
What’s just a tiny bit more usable than the assembly language, but with enough abstraction and still with a lot of power?
Oh, wait.
hosiawak, 4 March 2007, 12:10 am
Orbitz is powered by Common Lisp. You can read the nitty-gritty details at http://www.paulgraham.com/carl.html but overall it gives them the flexibility/abstractions and speed AND Orbitz is a very popular site so I guess that’s a popular application that everyone uses :)
johan, 17 December 2007, 3:24 pm
You have a point here, dude, C will not die anytime soon although it may suck at high level programming. However, you restrict yourself by making a religion out of programming languages.
Religion is basically an argument over who has the best imaginary friend. Languages though are just tools, and luckily there will always be people who break the limits, lest we end up in Dark Ages.
E.g. Scheme, a dialect of Lisp, can be compiled to straight C with the Chicken interpreter; ejabberd is written in Erlang and outplayed jabberd, which is written in C, in terms of reliability.
Mors, 13 January 2008, 10:25 am
A mi sie wydaje, ze dobry programista poprostu dobiera jezyk do zadania. Lisp jest moim zdaniem swietny jako jezyk rozszerzen np. w takim autocadzie sie sprawdza znakomicie. Kazdy jezyk po za tym ma swoja killerapp :) W takim LISPie sa to zagadnienia AI np. Program ktory na podstawie tabeli danych bedzie podawal wzor matematyczny (nie chodzi o aproksymacje tylko wzor postaci y(x) = sin(x)^cos(x**2) + 2/exp(1/x) ). Gdy uzyjemy algorytmow genetycznych albo metody gradientowej do rozwiazania tego zadania, mozna to w latwy sposob zapisac w LISPie. W C to nawet ciezko mi sobie wyobrazic jak by to napisac. Na obrone LISPa poda jeszcze ze znakomicie sie nadaje do rekreacji programistycznych :) Czasami po prostu mozna napisac cos for fun :)
it2051229, 25 June 2009, 7:23 am
if C is the language of gods… then I guess those people who do assembly is more than a god.