Sunday, March 23, 2008

All those parenthesis are counting on you

Let's make something clear, everyone designing their own dialect of lisp should be solving real problems instead. Still people continue to do that.
Because :
a It's fun
b It looks easy.
So what every lisper ought to do before he dies:
"Plant a tree, write a book, create his own dialect of lisp"

It's not that there is no benefit in making a better lisp, but the problem is that core language is only a part of the equation, the rest of the parameters in the formula are implementations, libraries, community and learning material. There are probably more but those will do. Also, the core language of lisp is well designed, showstoppers are few and controversial (*), so you're left with polishing language quirks that could be done within the language using metaprogramming facilities. Suddenly your new dialect of lisp is only a little bit better than language you're trying to supersede, but guess which one has more libraries, implementations, books, tutorials and users. Your chances aren't looking good now. You could design a language with different trade-offs and win some *markets* but don't attack a fortified hill. You'll need a lot of firepower to stand a chance.

Having a spare time and lacking a computer, left my old pc at home and still deciding what kind of notebook to buy, I started thinking about polishing some quirks and integrating some features I like from other languages. And the thing I hate most in lisp is the compound loop macro.
Why? It's such a useful thing. And why don't you complain about format too?
I don't use format that much so even if someone fix it, that won't buy me a lot. Something like if black pepper price doubles it won't hurt your budget because it's only 0.0002 % of your income but imagine paying twice for rent or gasoline. Ouch, that hurts.Because whatever I write it involves some kind of looping. Sometimes you can get away with higher order functions, sometimes dolist & dotimes will do, but more often than I want, loop is the best, read succint and efficient, solution. You can beat(**) loop on succinctness or effectiveness, but it's very hard or impossible to beat loop on both. Loop macro is special kind of demon, a DSL. Generic tools does very little damage on it.
Take this code for example source:

(defun separate (lst)
(mapcar #'(lambda (n) (mapcar #'abs (remove-if-not n lst)))
(list #'plusp #'zerop #'minusp)))

(defun separate (list)
(loop for element in list
if (plusp element)
collect element into positives
else if (minusp element)
collect (abs element) into minuses
else collect element into zeroes
finally (return (list minuses zeroes positives))))

My code is far shorter than Rob St Amants' but guess whose the consing whore (***). So I'm struggling to find a replacement to loop. Yes I've tried iterate, and it's better than loop but it's not that better. Like somebody described it : putting few extra parenthesis doesn't make iterate much lispier than loop. Also I've tried series and it's just not it. You can't argue with taste.

So what's the current situation in my mind. I have a good idea how to get rid of many stinking loops, that will require uniform access to lisp containers, which is good because that's another quirk that could be polished, also I have an idea how to simplify anonymous function generation in special cases. The problem is what to do with loops that collect multiple values like the code above. Using let is like rising the white flag of defeat, but I still don't have a graceful idea how to solve the problem. Could anybody implement above loop efficiently with higher order functions and without let? I'm really hungry for inspiration.
I'm thinking about a graceful solution, but good design is hard to get right. Unifying concepts of initializing, stepping, modifying and returning multiple results looks like trying to make a crossing of race car, jeep and truck and make sure that it doesn't look or behave like Frankensteins' monster.

(*) Continuations vs unwind-protect, better reflective abilities against cost of implementing those, etc
(**) Equal is fine also.
(***) Paraphrasing Ken Tilton.


  1. Hi Slobodan

    you should have a look at the SERIES package.


  2. I did, but I have serious attack of NIH syndrom :). I have some weird ideas and I want to try them out.

  3. Oh, who cares about a little extra consing. Go for the elegant solution. It's still O(n) anyway. It's bizarre how much time c.l.l residents waste trying to eliminate garbage conses; it very rarely matters in real life.


Note: Only a member of this blog may post a comment.