Wednesday, August 6, 2014

Why should you use node and avoid compatibility breaking "improvements"

There is a lot of hate this days against Node, I have a feeling that vocal minority is skewing the signal to noise ratio so I want to write few paragraphs why I'm using node and why it might be a good fit for your next web application.

Reasons to use it:

The platform works out of the box without tweaking. I've installed it on several different versions of the supported platforms (Win,Linux, Mac) It always worked without a hitch. This might seem trivial to most mainstream programmers but many niche languages require shovel and lantern before they're usable.

JavaScript supports first order functions, which coupled with lodash/underscore offer an usable option for writing functional code as your needs and taste fits. 

Node has at least two quality IDEs that work out of the box without tweaking, which I've used and could personally vouch for: 
I'm a visual learner I must see the project structure with syntax highlighting and be able to run the code & set breakpoints from the IDE to be productive. Editor mode won't gonna cut it, unpolished obscure plugin written by enthusiasts in their spare time and only tested under Gentoo compiled with specific CFLAGS is definitely not a viable replacement.

Full stack development is much easier by enabling writing the client code, server code & database code in JavaScript.  Lack of context switching and writing glue code enables staying focused on  the problem. Partial reuse of the code on the both ends represents a nice option.  

Huge very supportive community that makes a living by writing web applications and is ready to share their real world experience. Who would you rather have with you as the guide in the desert ? Someone who wrote a book about it or someone who lives there.

And last but actually the most important is the npm ecosystem.  As of end of July 2014 npm surpassed maven thus becoming the leader in the number of available repositories:
Source 06.08.2014

Node has a plethora of production ready and battle tested libraries offering you the luxury of concentrating on your application. Whatever your external dependencies are there is a good chance that official module already exists. And I personally prefer writing my code instead of wrestling with half baked driver for some niche database I need to use.

Use the de facto standards:

Node doesn't have a committee nor a dictator to enforce its standards but streams and error first callback pattern emerged as tacit agreement judging by number of libraries written.
It's a decision being made and whether vocal minority like it or not, complaining about it has the same effect like complaining about significant white space in python or parenthesis in lisp.

In true Darwinian contest the fallowing libraries emerged as de facto standard in the platform:
- Lodash / Underscore javascript utilities
- Async asynchronous programming
-Express minimalist web framework
-mongoose mongodb object modelling

Start your projects using the standard libraries and switch to alternatives only if you really, really  need something very specific and you are willing to pay the price of breaking the compatibility. 

Mature communities don't usually tolerate forks in compatibility, this is true across many platforms and the less academic a community is the less tolerant it will be of ideas of for ideas sake and the node community is anything but academic- Mikael

Beside the de facto standards there are many libraries where the jury is still out:
- Templating  - I  use Jade  but Handlebars is very popular
- Testing - I like Mocha + Chai combination but there is also intern 
- CSS - The usual suspects Stylus, SASS & Less
- Real time web frameworks- I use socketstream but development has been lacking lately. The gorilla in the room is meteor which got 11 M$ in funding so they might become THE standard.  I avoid it mostly because its not officially supported on windows and their walled garden approach with meteorite makes me little bit uneasy. Sails looks as an interesting approach and is actively maintained.

Avoid languages that compile to JavaScript:

My dislike for the languages who treat  JavaScript as glorified assembly has nothing to do with the quality of those languages per se and everything with fragmentation that arises from it. When someone writes code in those "improved" languages my initial reaction is to avoid it, and I have a feeling that reaction is almost universal. We all have our favorite syntax or pet feature from other languages JavaScript serves as a language that we all understand without deciphering the flavor of the week. 

If you want to write code running in the browser JavaScript is the only choice.You could either write it yourself or use some language/technology that compiles to it. And no matter how many benefits the authors of those "improvements" claim  whenever you start writing something non trivial you'll get into nasty problems and all your high level savings will be wiped out by law of  leaky abstraction. All the tooling built in the browser to debug your source code is built with JavaScript is mind.

In the end use whatever tool makes you most productive and leads you to best results, in many cases node is the right tool.

Tuesday, July 23, 2013

Express web application development

There are two kind of development books worth reading, first breed concentrates on quickly becoming proficient on the most needed tools without going into details, the second type is more aimed to developers who already learned the basics and need learning intermediate to advanced proficiency of the technology. Though Express web application development contains grains of the former it is fine example of the latter.

At the time of writing this review I've been using Express for more then a year and I'm duly impressed with the quality of the framework and the feel of production ready, everything you really need approach. The creator of  Express TJ Holowaychuk has become my  first option for high quality node.js libraries, express,jade,stylus,mocha,mongoose to name a few.
The author of this book  fallows a fine tradition of presenting real life needs using code that just works and builds on top of it with explanations of the express functionality .
All the  area covered routes, response, jade, stylus etc improved on my understanding of express and this book has become my reference material in my development.
My only nitpick is crave for more material, in order for this book to become "bible" of end to end web application development  it needs to add two chapters that cover the database layer with  mongoose and testing your application with mocha.

In conclusion I recommend this book for both those new with express and especially those who already wet their feet with express and need book that could teach real life usage and serve as a reference book for express.

The fine folks at PACKT publishing offered two free electronic copies to be raffled among users of this post, please comment with your name and email if you are interested.
The winners of the books  are two commentators who wrote their name and email address before  August 1st  2013: Alessio Stala & Aftab congratulations. I've send their info to PACKT folks and they will contact you.

Wednesday, November 10, 2010

Flood of lisp

Its been a great season for lisp folks with two new common lisp books arriving:
1 Land of Lisp by Conrad Barski
2 Practical Semantic Web and Linked Data Applications by Mark Watson

Already bought both of them and started with land of lisp, which I plan to review it after I finish it. The dry season is over we're currently spoiled of choices.

Saturday, July 24, 2010

Recruiters dirty trick

Remember the Paul Graham essays Great Hackers & The Python Paradox. To quote an paragraph for those too busy to read those two:

But when you choose a language, you're also choosing a community. The programmers you'll be able to hire to work on a Java project won't be as smart as the ones you could get to work on a project written in Python. Great Hackers

In a recent talk I said something that upset a lot of people: that you could get smarter programmers to work on a Python project than you could to work on a Java project.

I didn't mean by this that Java programmers are dumb. I meant that Python programmers are smart. It's a lot of work to learn a new programming language. And people don't learn Python because it will get them a job; they learn it because they genuinely like to program and aren't satisfied with the languages they already know.
The Python Paradox

How that translates into recruiter talk. Just imagine that you need to fill a position for some boring mainstream language. After few days (or weeks, months, quartiles) of interviewing cram school students you're exhausted with candidates who fail your variation of a FizzBuzz test so you need a way to drastically improve quality ratio of your applicants or your head is going to explode very soon. The solution is very simple just add some esoteric language to weed out resumes and hurray you'll have much smaller stack to sort.
Why this works?
Majority coders took programming as a job, something you do for money so you could spend those money on things that you need and things that interests you. If there was another job requiring same effort and payed more most coders would be doing that instead. I mean why would somebody be reading about tower of Hanoi when there is a good game on TV? If average coder needs something to advance their career so (s)he could make more money so (s)he could buy bigger TV they will learn it, it doesn't matter to them. It's the people who see programming as more than work who are complaining. Just imagine if there is enormous demand for violin players, it pays good money and work is easy to find. So if you aren't 100% tone deaf you might say : I can't find programming job, at least not one that pays well, so here's a good career choice for me, in the end everybody has to earn their bread somehow. So you buy the book learn violin playing in 24 hours and maybe get certified. Afterward you're doing your job and get payed, than your colleague is asking you do know to play Niccolo Paganini Fifth Caprice. Shit man we only do pop music here,what the hell do you need those kind of crap for? So you continue playing vanilla pop
at work and program at home why the bozo practices Fifth Caprice. (*)

So whenever I see job posts of niche languages mingled with mainstream languages I'm asking myself how much of the niche language the person is going to work with, if at all, or this is just a recruiter dirty trick to improve signal to noise ratio.
And if you think that I'm just fooling around you could see this technique mentioned in The Passionate Programmer chapter 5 where Chad Fowler uses Smalltalk to find out Java programmers in India.

Everybody could be taught how to paint, Michelangelo must be taught not to(**)

(*) This part is slightly edited post from, comp.lang.lisp original is found here
(**) I don't know where this quote is from , I think its pg but I can't find the reference.

Regarding my post about Haskell

Yesterday blogger team send me their quarterly? spam I mean newsletter about new features added to blogger (template designer, accurate preview,sharing buttons, blog stats ..). After I changed my background with template designer and added the sharing buttons as any small village celebrity I've opened the blog stats. I knew that google offered some kind of analytic but it was hard to use or at least I think that is hard to use and since I'm lazy I never used it. But now with stats neatly integrated into the dashboard I couldn't resist. And there was the surprise the my old posts Lisper's first look at Haskell and Farewell Haskell having hundreds views per day. It seems that some necromancer :) brought to life that thread on reddit and hacker news. Anyway for all those interested in update here's a short preview of my present situation.
Currently I'm using regularly common lisp, java, c#,c/c++ and SQL. I bought Programming Clojure & Practical Clojure and went through them. I would have bought both Clojure in action and Joy of Clojure (written by starter of this thread fogus) if Manning accepted credit card payment from Macedonia. I'm little tired of everything Java related, that includes Clojure, so I'm using my spare time to learn OpenCL. Regarding Haskell. I haven't programmed in it since I wrote the post farewell haskell. The reason for that is my programming style for which Haskell is ill suited which described in the farewell Haskell post in the paragraph starting with: It may interest you to know that most of my development .. I don't hate Haskell but its not in list my favorite language either(*) though as one favoring functional style I cheer news like this . Also I prefer to say what I'm thinking and be rude if necessary then hide my message between being politically correct watered down politician talk. On the other hand I have high opinion about SPJ and enjoy his talk and many of his papers. In the end language wars are pointless, use the language that suits yourself and have a marry hacking.

(*) For those interested my favorite languages are lisp dialects (cl,scheme and clojure), array languages (j & q), Prolog, Erlang and concatenative languages.

Sunday, July 18, 2010

Enter the Rewrite

The best writing is rewriting -- E.B.White

If the software you’ve built is complex enough that it needs to be rewritten, it’s probably also so complex that it’s not discoverable in this way.--Chad Fowler

Rewrites are one of the things that you shouldn't do. They're singled out by Joel Spolsky as the single worst strategic mistake that any software company can make. I couldn't agree more as the rewrites usually gain you nothing but a sirens promise that there will be payoffs in the future. But, there is always but.
Let me explain you my situation first. My problem occurred due to my decision to use a subroutine threaded code which lead me to c++ function pointers. In the beginning it was easy, as I was using a single type (float) for pretty much everything, but as soon as I added the whole pack the problem occurred with my code starting to become crowded with things like below:

void* createBool(void* p){return new bool(*((bool*)p));}
void* createLong(void* p){return new long(*((long*)p));}
void* createInt(void* p){return new int(*((int*)p));}

c++ function templates helped somewhat but since I needed to manipulate tokens I started using c macros. I know about all that talk that macros are evil, but above sample is only for using single type functions which pointers I need, just imagine when you add interaction of two different types like : short & float, int & long, etc and you quickly realize that I was facing a combinatorial explosion. Beside the containers were just around the corner and sheer thinking of vector,list, etc holding simple types scared me. Obviously my approach wasn't working. Smart people in this case would sit down and rethink the approach, like those Haskell types who will sit thinking whole day long and finally write some five liner that will solve the problem. Unfortunately I'm not one of them, if I sit down thinking for more then 15 minutes will use the momentum and end up doing something else. Thinking is just not my style. I wanted to be able to specify the type or container once, maybe add some rules then everything else to be generated programmatically. c preprocessor just wasn't powerful enough for my needs. It could do some simple function generation but insertion of code according to rules into arbitrary places was way out of its league.So in order to continue with my approach I needed a code generation facility.

Handling the rewrite

I used common lisp as my code generation language for reasons such as: rapid interactive development, versatility of list data structures and my familiarity with it, but the code generation facility code be written in any Turing complete language. I fallowed three rewrite rules to keep the task manageable:
1 Divide and conquer
Forget about rewriting any non trivial program from scratch. Divide the rewrite into manageable chunks. Due to issues with c++ linker I switched the multi-file project into single file approach. Afterward I split the file into logical regions such as print procedures, create procedures, etc. Each of it could be worked on separately.
2 Use tests
My main test case was the c++ compiler being able to compile the generated file. I've also added several correctness tests. I don't find TDD very useful with real life problems which are usually ill defined. But when you are at certain level and you know what you need to test for adding tests is smart move.
3 Keep it working all the time
My prime directive is to refuse to work on a broken code. Its a reminiscence of the lisp philosophy of little bit program little bit debug. If the code is broken you don't know why the hell is it broken until you fix it. Telling yourself that it will be fine after you do X is deluding yourself. Adding features to broken code is insane.

So the whole program was rewritten in cycles. In each cycle one region or part of a region was programmatically generated and c++ code from hand written version disappeared piece by piece. After each change commented the handwritten code, then generated the program file, compile it and run the tests. If everything was fine the commented code was removed and I continued with the next piece. In the end I'm more then satisfied with the resulting rewrite though I would preferred if I was able to add features instead of rewriting but sometimes you have to remove some mess in order to survive. Rewrites still suck but sometimes you have to do them and when you do, rewrite the smart way.

Wednesday, May 26, 2010

Becoming popular

There is an excellent article titled How to design a popular programming language. The whole formula is very simple:
1. Find a new platform that will be a huge success in a few years.
2. Make your language the default way to program on that platform.
3. Wait.

I can't agree more with the author, excluding language enthusiasts and religious zealots, programming languages are mere tools that enable the programmers to write software that (usually) solves some problem . So nobody gives a damn does a language has tail recursion, multiple inheritance or monads. I mean those who know about such things would whine for some time, sometimes even for a long time, but you could safely ignore them as in the end the problem will be solved with either simple workaround or using a different approach, this is guaranteed if the language is Turing complete.

However the author is quoting two languages as exceptions (Java and Python) to this rule. But these two languages only make his definition of a platform oversimplified. To counter with an oversimplification of my own the platform can be seen as nothing more then a market. From this point of view both exceptions aren't exceptions at all, rather a fine examples of riding a platform. Java's platform is writing cross platform enterprise applications, the whole promise of "Write once, run, anywhere" enabled the programmers to rely on JVM being able to run their code across plethora of operating systems. On the other hand if your market is productive language fallowing There's Only One Way To Do It and batteries included philosophies there is no other choice but Python. To conclude anything could be defined as a platform, but most platform aren't economically viable.

If you understood the value of platform then what choice is left for the unpopular languages. Alessio Stala half jokingly suggested that: all that great but unpopular language has to do to in order to gain popularity is to deeply integrate with a popular platform and make it easy and convenient to use pre-existing libraries written in other languages for that platform.

My humble opinion is that this wont work for two reasons:

First reason is that if the platform is already popular that means that there is at least one standard language for programming in that platform and its impossible to dislodge it, unless it shoot itself in the head and alienates its users. So the only way for the language to become THE language for that platform is to become one before the platform becomes popular.

But why couldn't the new, shiny, perfectly designed language dislodge the old, rusty, haphazardly bundled language? The newcomer will better at everything. Better libraries, better tools, better books, better ...

This concept is best described by the book Crossing the chasm (*). The short story is that life cycle of adopting a new technology starts with early adapters fallowed by a huge ditch called chasm which you need to cross in order to reach the pragmatists which we usually call mainstream. The early adapters are easy to get, you just need to have something interesting, those are the people that want to tinker. The pragmatists are the tough ones. They are hard to get but very loyal once won. They want proven technologies, vendors with credentials and most of all standards. They want to buy from the market leader and you ain't one. Pragmatists control most of the money and are the ones that are fallowed by Conservatives. Try to make problems by disturbing existing market and they will cut your throat.

But what about Clojure or Scala? Bear with me for a while.
The only way for a new language to succeed is to dominate a viable platform. What is the Clojure biggest selling point? Is it Macros? Lisp and Scheme have them for decades. What about interoperability with existing Java libraries? ABCL, Jython should I say more etc. The only killing feature of Clojure is concurrency. In order to successfully cross the chasm you need a pragmatist in pain. Pragmatist that suffers from writing concurrent code in Java so much that he is willing to give a chance to a fledgling newcomer like Clojure. Don't get me wrong aforementioned features help a lot, especially backward compatibility, but if processors were still single core and we all enjoyed the free lunch of increasing gigahertz-es nobody would give a damn about some weird functional language with immutable data structures. Clojure has no chance to dislodge Java as premier language for writing enterprise applications, but it could win as the language for writing concurrent applications. That is Clojure platform.

The second reason why this corollary wouldn't work is because integrating in some platform requires the language to change.But mature languages can't change easily if at all. This situation is described in theory of frozen evolution. Basically the evolution of language is similar of the evolution of a new specie. Evolution happens in short burst when the new language(specie) is created. In this period language design is plastic, there are few if any users, little amount of code and libraries and the community consists of enthusiasts. This short period of plasticity is fallowed by a long period of stability. The evolution of the language is frozen. There are few tweaks here and there but nothing radical. And nobody , not even the original creator could change that. Just imagine if Bjarne Stroustrup says to ANSI C++ committee that he wants to adapt post fix syntax, they would laugh at him just before they send him to mental institution to see what's wrong with his nerves. On the other hand Hickey , Tarver and Pestov could redesign their languages when new opportunity arise, but a new standard for Common Lisp is nothing but a dream.

(*) Eric Sink has a very nice article describing Geoffrey Moore ideas, where I shamelessly hot linked the bell curve image.