Saturday, July 24, 2010

Recruiters dirty trick

Remember the Paul Graham essays Great Hackers & The Python Paradox. To quote an paragraph for those too busy to read those two:

But when you choose a language, you're also choosing a community. The programmers you'll be able to hire to work on a Java project won't be as smart as the ones you could get to work on a project written in Python. Great Hackers

In a recent talk I said something that upset a lot of people: that you could get smarter programmers to work on a Python project than you could to work on a Java project.

I didn't mean by this that Java programmers are dumb. I meant that Python programmers are smart. It's a lot of work to learn a new programming language. And people don't learn Python because it will get them a job; they learn it because they genuinely like to program and aren't satisfied with the languages they already know.
The Python Paradox

How that translates into recruiter talk. Just imagine that you need to fill a position for some boring mainstream language. After few days (or weeks, months, quartiles) of interviewing cram school students you're exhausted with candidates who fail your variation of a FizzBuzz test so you need a way to drastically improve quality ratio of your applicants or your head is going to explode very soon. The solution is very simple just add some esoteric language to weed out resumes and hurray you'll have much smaller stack to sort.
Why this works?
Majority coders took programming as a job, something you do for money so you could spend those money on things that you need and things that interests you. If there was another job requiring same effort and payed more most coders would be doing that instead. I mean why would somebody be reading about tower of Hanoi when there is a good game on TV? If average coder needs something to advance their career so (s)he could make more money so (s)he could buy bigger TV they will learn it, it doesn't matter to them. It's the people who see programming as more than work who are complaining. Just imagine if there is enormous demand for violin players, it pays good money and work is easy to find. So if you aren't 100% tone deaf you might say : I can't find programming job, at least not one that pays well, so here's a good career choice for me, in the end everybody has to earn their bread somehow. So you buy the book learn violin playing in 24 hours and maybe get certified. Afterward you're doing your job and get payed, than your colleague is asking you do know to play Niccolo Paganini Fifth Caprice. Shit man we only do pop music here,what the hell do you need those kind of crap for? So you continue playing vanilla pop
at work and program at home why the bozo practices Fifth Caprice. (*)

So whenever I see job posts of niche languages mingled with mainstream languages I'm asking myself how much of the niche language the person is going to work with, if at all, or this is just a recruiter dirty trick to improve signal to noise ratio.
And if you think that I'm just fooling around you could see this technique mentioned in The Passionate Programmer chapter 5 where Chad Fowler uses Smalltalk to find out Java programmers in India.

Everybody could be taught how to paint, Michelangelo must be taught not to(**)

(*) This part is slightly edited post from, comp.lang.lisp original is found here
(**) I don't know where this quote is from , I think its pg but I can't find the reference.

Regarding my post about Haskell

Yesterday blogger team send me their quarterly? spam I mean newsletter about new features added to blogger (template designer, accurate preview,sharing buttons, blog stats ..). After I changed my background with template designer and added the sharing buttons as any small village celebrity I've opened the blog stats. I knew that google offered some kind of analytic but it was hard to use or at least I think that is hard to use and since I'm lazy I never used it. But now with stats neatly integrated into the dashboard I couldn't resist. And there was the surprise the my old posts Lisper's first look at Haskell and Farewell Haskell having hundreds views per day. It seems that some necromancer :) brought to life that thread on reddit and hacker news. Anyway for all those interested in update here's a short preview of my present situation.
Currently I'm using regularly common lisp, java, c#,c/c++ and SQL. I bought Programming Clojure & Practical Clojure and went through them. I would have bought both Clojure in action and Joy of Clojure (written by starter of this thread fogus) if Manning accepted credit card payment from Macedonia. I'm little tired of everything Java related, that includes Clojure, so I'm using my spare time to learn OpenCL. Regarding Haskell. I haven't programmed in it since I wrote the post farewell haskell. The reason for that is my programming style for which Haskell is ill suited which described in the farewell Haskell post in the paragraph starting with: It may interest you to know that most of my development .. I don't hate Haskell but its not in list my favorite language either(*) though as one favoring functional style I cheer news like this . Also I prefer to say what I'm thinking and be rude if necessary then hide my message between being politically correct watered down politician talk. On the other hand I have high opinion about SPJ and enjoy his talk and many of his papers. In the end language wars are pointless, use the language that suits yourself and have a marry hacking.

(*) For those interested my favorite languages are lisp dialects (cl,scheme and clojure), array languages (j & q), Prolog, Erlang and concatenative languages.

Sunday, July 18, 2010

Enter the Rewrite

The best writing is rewriting -- E.B.White

If the software you’ve built is complex enough that it needs to be rewritten, it’s probably also so complex that it’s not discoverable in this way.--Chad Fowler

Rewrites are one of the things that you shouldn't do. They're singled out by Joel Spolsky as the single worst strategic mistake that any software company can make. I couldn't agree more as the rewrites usually gain you nothing but a sirens promise that there will be payoffs in the future. But, there is always but.
Let me explain you my situation first. My problem occurred due to my decision to use a subroutine threaded code which lead me to c++ function pointers. In the beginning it was easy, as I was using a single type (float) for pretty much everything, but as soon as I added the whole pack the problem occurred with my code starting to become crowded with things like below:

void* createBool(void* p){return new bool(*((bool*)p));}
void* createLong(void* p){return new long(*((long*)p));}
void* createInt(void* p){return new int(*((int*)p));}
...

c++ function templates helped somewhat but since I needed to manipulate tokens I started using c macros. I know about all that talk that macros are evil, but above sample is only for using single type functions which pointers I need, just imagine when you add interaction of two different types like : short & float, int & long, etc and you quickly realize that I was facing a combinatorial explosion. Beside the containers were just around the corner and sheer thinking of vector,list, etc holding simple types scared me. Obviously my approach wasn't working. Smart people in this case would sit down and rethink the approach, like those Haskell types who will sit thinking whole day long and finally write some five liner that will solve the problem. Unfortunately I'm not one of them, if I sit down thinking for more then 15 minutes will use the momentum and end up doing something else. Thinking is just not my style. I wanted to be able to specify the type or container once, maybe add some rules then everything else to be generated programmatically. c preprocessor just wasn't powerful enough for my needs. It could do some simple function generation but insertion of code according to rules into arbitrary places was way out of its league.So in order to continue with my approach I needed a code generation facility.

Handling the rewrite

I used common lisp as my code generation language for reasons such as: rapid interactive development, versatility of list data structures and my familiarity with it, but the code generation facility code be written in any Turing complete language. I fallowed three rewrite rules to keep the task manageable:
1 Divide and conquer
Forget about rewriting any non trivial program from scratch. Divide the rewrite into manageable chunks. Due to issues with c++ linker I switched the multi-file project into single file approach. Afterward I split the file into logical regions such as print procedures, create procedures, etc. Each of it could be worked on separately.
2 Use tests
My main test case was the c++ compiler being able to compile the generated file. I've also added several correctness tests. I don't find TDD very useful with real life problems which are usually ill defined. But when you are at certain level and you know what you need to test for adding tests is smart move.
3 Keep it working all the time
My prime directive is to refuse to work on a broken code. Its a reminiscence of the lisp philosophy of little bit program little bit debug. If the code is broken you don't know why the hell is it broken until you fix it. Telling yourself that it will be fine after you do X is deluding yourself. Adding features to broken code is insane.

So the whole program was rewritten in cycles. In each cycle one region or part of a region was programmatically generated and c++ code from hand written version disappeared piece by piece. After each change commented the handwritten code, then generated the program file, compile it and run the tests. If everything was fine the commented code was removed and I continued with the next piece. In the end I'm more then satisfied with the resulting rewrite though I would preferred if I was able to add features instead of rewriting but sometimes you have to remove some mess in order to survive. Rewrites still suck but sometimes you have to do them and when you do, rewrite the smart way.