Wednesday, November 10, 2010

Flood of lisp

Its been a great season for lisp folks with two new common lisp books arriving:
1 Land of Lisp by Conrad Barski
2 Practical Semantic Web and Linked Data Applications by Mark Watson

Already bought both of them and started with land of lisp, which I plan to review it after I finish it. The dry season is over we're currently spoiled of choices.

Saturday, July 24, 2010

Recruiters dirty trick

Remember the Paul Graham essays Great Hackers & The Python Paradox. To quote an paragraph for those too busy to read those two:

But when you choose a language, you're also choosing a community. The programmers you'll be able to hire to work on a Java project won't be as smart as the ones you could get to work on a project written in Python. Great Hackers

In a recent talk I said something that upset a lot of people: that you could get smarter programmers to work on a Python project than you could to work on a Java project.

I didn't mean by this that Java programmers are dumb. I meant that Python programmers are smart. It's a lot of work to learn a new programming language. And people don't learn Python because it will get them a job; they learn it because they genuinely like to program and aren't satisfied with the languages they already know.
The Python Paradox

How that translates into recruiter talk. Just imagine that you need to fill a position for some boring mainstream language. After few days (or weeks, months, quartiles) of interviewing cram school students you're exhausted with candidates who fail your variation of a FizzBuzz test so you need a way to drastically improve quality ratio of your applicants or your head is going to explode very soon. The solution is very simple just add some esoteric language to weed out resumes and hurray you'll have much smaller stack to sort.
Why this works?
Majority coders took programming as a job, something you do for money so you could spend those money on things that you need and things that interests you. If there was another job requiring same effort and payed more most coders would be doing that instead. I mean why would somebody be reading about tower of Hanoi when there is a good game on TV? If average coder needs something to advance their career so (s)he could make more money so (s)he could buy bigger TV they will learn it, it doesn't matter to them. It's the people who see programming as more than work who are complaining. Just imagine if there is enormous demand for violin players, it pays good money and work is easy to find. So if you aren't 100% tone deaf you might say : I can't find programming job, at least not one that pays well, so here's a good career choice for me, in the end everybody has to earn their bread somehow. So you buy the book learn violin playing in 24 hours and maybe get certified. Afterward you're doing your job and get payed, than your colleague is asking you do know to play Niccolo Paganini Fifth Caprice. Shit man we only do pop music here,what the hell do you need those kind of crap for? So you continue playing vanilla pop
at work and program at home why the bozo practices Fifth Caprice. (*)

So whenever I see job posts of niche languages mingled with mainstream languages I'm asking myself how much of the niche language the person is going to work with, if at all, or this is just a recruiter dirty trick to improve signal to noise ratio.
And if you think that I'm just fooling around you could see this technique mentioned in The Passionate Programmer chapter 5 where Chad Fowler uses Smalltalk to find out Java programmers in India.

Everybody could be taught how to paint, Michelangelo must be taught not to(**)

(*) This part is slightly edited post from, comp.lang.lisp original is found here
(**) I don't know where this quote is from , I think its pg but I can't find the reference.

Regarding my post about Haskell

Yesterday blogger team send me their quarterly? spam I mean newsletter about new features added to blogger (template designer, accurate preview,sharing buttons, blog stats ..). After I changed my background with template designer and added the sharing buttons as any small village celebrity I've opened the blog stats. I knew that google offered some kind of analytic but it was hard to use or at least I think that is hard to use and since I'm lazy I never used it. But now with stats neatly integrated into the dashboard I couldn't resist. And there was the surprise the my old posts Lisper's first look at Haskell and Farewell Haskell having hundreds views per day. It seems that some necromancer :) brought to life that thread on reddit and hacker news. Anyway for all those interested in update here's a short preview of my present situation.
Currently I'm using regularly common lisp, java, c#,c/c++ and SQL. I bought Programming Clojure & Practical Clojure and went through them. I would have bought both Clojure in action and Joy of Clojure (written by starter of this thread fogus) if Manning accepted credit card payment from Macedonia. I'm little tired of everything Java related, that includes Clojure, so I'm using my spare time to learn OpenCL. Regarding Haskell. I haven't programmed in it since I wrote the post farewell haskell. The reason for that is my programming style for which Haskell is ill suited which described in the farewell Haskell post in the paragraph starting with: It may interest you to know that most of my development .. I don't hate Haskell but its not in list my favorite language either(*) though as one favoring functional style I cheer news like this . Also I prefer to say what I'm thinking and be rude if necessary then hide my message between being politically correct watered down politician talk. On the other hand I have high opinion about SPJ and enjoy his talk and many of his papers. In the end language wars are pointless, use the language that suits yourself and have a marry hacking.

(*) For those interested my favorite languages are lisp dialects (cl,scheme and clojure), array languages (j & q), Prolog, Erlang and concatenative languages.

Sunday, July 18, 2010

Enter the Rewrite

The best writing is rewriting -- E.B.White

If the software you’ve built is complex enough that it needs to be rewritten, it’s probably also so complex that it’s not discoverable in this way.--Chad Fowler

Rewrites are one of the things that you shouldn't do. They're singled out by Joel Spolsky as the single worst strategic mistake that any software company can make. I couldn't agree more as the rewrites usually gain you nothing but a sirens promise that there will be payoffs in the future. But, there is always but.
Let me explain you my situation first. My problem occurred due to my decision to use a subroutine threaded code which lead me to c++ function pointers. In the beginning it was easy, as I was using a single type (float) for pretty much everything, but as soon as I added the whole pack the problem occurred with my code starting to become crowded with things like below:

void* createBool(void* p){return new bool(*((bool*)p));}
void* createLong(void* p){return new long(*((long*)p));}
void* createInt(void* p){return new int(*((int*)p));}

c++ function templates helped somewhat but since I needed to manipulate tokens I started using c macros. I know about all that talk that macros are evil, but above sample is only for using single type functions which pointers I need, just imagine when you add interaction of two different types like : short & float, int & long, etc and you quickly realize that I was facing a combinatorial explosion. Beside the containers were just around the corner and sheer thinking of vector,list, etc holding simple types scared me. Obviously my approach wasn't working. Smart people in this case would sit down and rethink the approach, like those Haskell types who will sit thinking whole day long and finally write some five liner that will solve the problem. Unfortunately I'm not one of them, if I sit down thinking for more then 15 minutes will use the momentum and end up doing something else. Thinking is just not my style. I wanted to be able to specify the type or container once, maybe add some rules then everything else to be generated programmatically. c preprocessor just wasn't powerful enough for my needs. It could do some simple function generation but insertion of code according to rules into arbitrary places was way out of its league.So in order to continue with my approach I needed a code generation facility.

Handling the rewrite

I used common lisp as my code generation language for reasons such as: rapid interactive development, versatility of list data structures and my familiarity with it, but the code generation facility code be written in any Turing complete language. I fallowed three rewrite rules to keep the task manageable:
1 Divide and conquer
Forget about rewriting any non trivial program from scratch. Divide the rewrite into manageable chunks. Due to issues with c++ linker I switched the multi-file project into single file approach. Afterward I split the file into logical regions such as print procedures, create procedures, etc. Each of it could be worked on separately.
2 Use tests
My main test case was the c++ compiler being able to compile the generated file. I've also added several correctness tests. I don't find TDD very useful with real life problems which are usually ill defined. But when you are at certain level and you know what you need to test for adding tests is smart move.
3 Keep it working all the time
My prime directive is to refuse to work on a broken code. Its a reminiscence of the lisp philosophy of little bit program little bit debug. If the code is broken you don't know why the hell is it broken until you fix it. Telling yourself that it will be fine after you do X is deluding yourself. Adding features to broken code is insane.

So the whole program was rewritten in cycles. In each cycle one region or part of a region was programmatically generated and c++ code from hand written version disappeared piece by piece. After each change commented the handwritten code, then generated the program file, compile it and run the tests. If everything was fine the commented code was removed and I continued with the next piece. In the end I'm more then satisfied with the resulting rewrite though I would preferred if I was able to add features instead of rewriting but sometimes you have to remove some mess in order to survive. Rewrites still suck but sometimes you have to do them and when you do, rewrite the smart way.

Wednesday, May 26, 2010

Becoming popular

There is an excellent article titled How to design a popular programming language. The whole formula is very simple:
1. Find a new platform that will be a huge success in a few years.
2. Make your language the default way to program on that platform.
3. Wait.

I can't agree more with the author, excluding language enthusiasts and religious zealots, programming languages are mere tools that enable the programmers to write software that (usually) solves some problem . So nobody gives a damn does a language has tail recursion, multiple inheritance or monads. I mean those who know about such things would whine for some time, sometimes even for a long time, but you could safely ignore them as in the end the problem will be solved with either simple workaround or using a different approach, this is guaranteed if the language is Turing complete.

However the author is quoting two languages as exceptions (Java and Python) to this rule. But these two languages only make his definition of a platform oversimplified. To counter with an oversimplification of my own the platform can be seen as nothing more then a market. From this point of view both exceptions aren't exceptions at all, rather a fine examples of riding a platform. Java's platform is writing cross platform enterprise applications, the whole promise of "Write once, run, anywhere" enabled the programmers to rely on JVM being able to run their code across plethora of operating systems. On the other hand if your market is productive language fallowing There's Only One Way To Do It and batteries included philosophies there is no other choice but Python. To conclude anything could be defined as a platform, but most platform aren't economically viable.

If you understood the value of platform then what choice is left for the unpopular languages. Alessio Stala half jokingly suggested that: all that great but unpopular language has to do to in order to gain popularity is to deeply integrate with a popular platform and make it easy and convenient to use pre-existing libraries written in other languages for that platform.

My humble opinion is that this wont work for two reasons:

First reason is that if the platform is already popular that means that there is at least one standard language for programming in that platform and its impossible to dislodge it, unless it shoot itself in the head and alienates its users. So the only way for the language to become THE language for that platform is to become one before the platform becomes popular.

But why couldn't the new, shiny, perfectly designed language dislodge the old, rusty, haphazardly bundled language? The newcomer will better at everything. Better libraries, better tools, better books, better ...

This concept is best described by the book Crossing the chasm (*). The short story is that life cycle of adopting a new technology starts with early adapters fallowed by a huge ditch called chasm which you need to cross in order to reach the pragmatists which we usually call mainstream. The early adapters are easy to get, you just need to have something interesting, those are the people that want to tinker. The pragmatists are the tough ones. They are hard to get but very loyal once won. They want proven technologies, vendors with credentials and most of all standards. They want to buy from the market leader and you ain't one. Pragmatists control most of the money and are the ones that are fallowed by Conservatives. Try to make problems by disturbing existing market and they will cut your throat.

But what about Clojure or Scala? Bear with me for a while.
The only way for a new language to succeed is to dominate a viable platform. What is the Clojure biggest selling point? Is it Macros? Lisp and Scheme have them for decades. What about interoperability with existing Java libraries? ABCL, Jython should I say more etc. The only killing feature of Clojure is concurrency. In order to successfully cross the chasm you need a pragmatist in pain. Pragmatist that suffers from writing concurrent code in Java so much that he is willing to give a chance to a fledgling newcomer like Clojure. Don't get me wrong aforementioned features help a lot, especially backward compatibility, but if processors were still single core and we all enjoyed the free lunch of increasing gigahertz-es nobody would give a damn about some weird functional language with immutable data structures. Clojure has no chance to dislodge Java as premier language for writing enterprise applications, but it could win as the language for writing concurrent applications. That is Clojure platform.

The second reason why this corollary wouldn't work is because integrating in some platform requires the language to change.But mature languages can't change easily if at all. This situation is described in theory of frozen evolution. Basically the evolution of language is similar of the evolution of a new specie. Evolution happens in short burst when the new language(specie) is created. In this period language design is plastic, there are few if any users, little amount of code and libraries and the community consists of enthusiasts. This short period of plasticity is fallowed by a long period of stability. The evolution of the language is frozen. There are few tweaks here and there but nothing radical. And nobody , not even the original creator could change that. Just imagine if Bjarne Stroustrup says to ANSI C++ committee that he wants to adapt post fix syntax, they would laugh at him just before they send him to mental institution to see what's wrong with his nerves. On the other hand Hickey , Tarver and Pestov could redesign their languages when new opportunity arise, but a new standard for Common Lisp is nothing but a dream.

(*) Eric Sink has a very nice article describing Geoffrey Moore ideas, where I shamelessly hot linked the bell curve image.

Sunday, May 23, 2010

Judging the language by its implementation

Paul Graham wrote an article about judging a programming language by its cover, explaining the use of hacker radar to infer knowledge about a language that you never wrote a code in it. I won't comment on his reasoning however while traveling through the land of programming languages I've found another quick method that will tell you a lot about the language without the need to write any code in it. Its called judging the language by its implementation.

First type of languages lack a functional implementation. There's web site saying how this language will be solving whatever is bothering the language designer and you might find paper(s) describing the language. These aren't languages at all, move over there's nothing to see. Unless you are interested in vaporware.

The second type are those with interpreter written in some other high level language that looks like it was bolted together over the weekend by some student hastily preparing the seminal assignment. Just skip those too, if you found the concept really appealing bookmark the site and visit it after few months to see if there is any progress.

The third type is minimalistic interpreter but this time the interpreter feels good, supporting several operating systems and on each it just works without tweaking. Usually this type of languages started their life as a research toys or maybe a scripting tools that proved itself useful thus gaining life of their own. Though minimalistic this languages might be battle proven in production environments for many times. If the field this type of language targets interests you, you might be well served. I believe in this case existence of viable community should be deciding factor. If there are at least 100 active users, go for it as people that gather around this niche languages tend to be very knowledgeable and helpful.

The fourth type are languages that I call pioneering languages. You could recognize them claiming tools that exist only for the mainstream languages, like integrated development environments, code browsers etc but instead of providing their own they piggy back as plug ins on mainstream languages tools. This should be warning sign, if the plug ins work without tinkering the language might be usable, else you are in the land of pioneers and pioneers are the ones with arrows in their backs. Proceed at your own risk.

The fifth types are the ones with development environments of their own but you need to install like gazillion dependencies, many of those are external (usually dated) versions of popular run-times like .NET or JVM or many popular libraries. Avoid this like a plague. This a sign of language that was once popular but has fallen from grace and now instead of choosing its niche is over reaching trying to everything to everybody.

I could mention a sixth type with maybe several mature implementations but those languages are usually in the mainstream or well known, so you could gain enough knowledge about them with 5 minute goggling.

Saturday, May 22, 2010

OpenCL Programming

I've finished the OpenCL Programming Book and I'm quite struck by low-levelness of OpenCL programming. Lisper in me is already tempted to make some scripting language around it but I need more practice first. So next step is going through SDK examples which are plenty. And it seems that literature is growing fast. ATI is finally investing resources into OpenCL, that will help OpenCL define itself as a real standard, NVIDIA would rather have everybody writing CUDA.
I'm buying a new machine with 2 x PCI express slots. Will need it in order to code with both ATI and NVIDIA cards simultaneously. However I'm little puzzled about my gaming subconscious using OpenCL as an excuse for buying new machine in order to be able to play mafia 2.

Friday, May 14, 2010

How to game a technical interview

After reading the Yes, I know article and especially the comment below I thought how easy was to game the interview.
If there is an opening you're qualified for but there will a tough competition and the interviewers value highly candidate technical competency your answers to interview technical questions would carry a lot of weight. So if the interviews are scheduled in different days all you need is an accomplice to be the brilliant at your interview. Choose someone who has a great resume or at least good enough to be called for the interview but isn't interested in that opening. When HR calls let your accomplice schedule the interview day before you then tell you the interview questions.
The interviewers might change the questions afterward but I seriously doubt it. First preparing good interview questions is a tough work and second and more important if the questions are changed a lot it becomes very hard to compare skills of the different candidates without having a common reference point.

And next time I'm preparing a unique question for those looking suspicious on the ones previously asked. Sometimes paranoia is your friend.

Friday, May 7, 2010


I've just bought the OpenCL Programming Book and start to dive in the world of parallelism. In the world of multi-core processors and programmable graphic cards I have an uneasy feeling that my knowledge is becoming dated and that's not something that I feel comfortable with.

Thursday, April 15, 2010

Equalp and the modern world

Its weird how many good ideas designers of common lisp had so long ago. Today I had a problem with the combo not being able to recognise value typed from keyboard if the user entered it too fast. Checked the vendor forums and the issue seemed to be mentioned several times. Luckily the combo provided the raw value entered by user as a string which immediately reminded me of Equalp. Just iterating over the options and comparing their string representation with the raw entered value allowed me to create a quick workaround. Maybe learning lisp doesn't make you worse programmer.