Monday, December 10, 2007

Adding new weblocks type

Note:
For better formatted version of the code mentioned in this text please check:
http://paste.lisp.org/display/52281
http://paste.lisp.org/display/52282

Weblocks supports several built in types for specifying behaviour of slots. The list includes : boolean, member, keyword, us-states & symbol (Dec 11 20007). But if none of those suits your needs you have to write your own.
I needed a slot that will be rendered as textarea tag for entering input, so after a little consultation at the weblocks group I started with coding.
First I created a new file called text.lisp at /cl-weblocks/src/types/ directory, defining a new type called text.

(in-package weblocks)
(export '(text))
(deftype text () 'string)
Than I needed a function that will do the actual rendering of html code. File html-utils.lisp located at /cl-weblocks/src/snippets/ contains functions for rendering various kinds of elements : links, buttons, checkboxes, dropdowns, radio-buttons, & close-buttons (Dec 11 2007]. Unfortunately the functionality I needed, rendering textarea wasn't there. So I added it myself:
;;;html-utils.lisp
(defun render-textarea (name value maxlength rows cols &key id (class "textarea-class"))
"Renders a textarea in a form.
'name' - name of the html control. The name is attributized before being rendered.
'value' - a value on html control. Humanized name is default.
'id' - id of the html control. Default is nil.
maxlength - maximum lentgh of the field
rows - number of rows in textarea
cols - number of columns in textarea
'class' - a class used for styling. By default,
\"textarea\"."
(with-html
(:textarea :name name :id id :maxlength maxlength
:rows rows :cols cols :class class
(cl-who:str (or value ""))
)
)
)

The above code is based around cl-who with-html-output macro. I faced a problem with generating string from my value variable, somehow it vanished from generated string, but that problem was quickly solved with a help from c.l.l. denizens Sohail, Maciej & Alessio. Thank you guys. Afterwards I wrote a test cases in /cl-weblocks/test/snippets/html-utils.lisp. Every bit of code in Weblocks that could be tested is covered by test cases, and after adding yours be sure that you didn't break any of the previous tests.

;; test render-textarea
(deftest-html render-textarea-1
(render-textarea 'name1 'value1 200 4 20 :id 'id1 :class 'class1 )
(:textarea :name "NAME1" :id "ID1" :maxlength 200 :rows 4 :cols 20 :class "CLASS1"
(cl-who:str (princ-to-string 'value1))
)
)


(deftest-html render-textarea-2
(render-textarea 'name2 'value2 200 4 20)
(:textarea :name "NAME2" :maxlength 200 :rows 4 :cols 20 :class "textarea-class"
(cl-who:str (princ-to-string 'value2))
)
)


Back to my text.lisp file I've added several globals that contain default values of rows, columns, length of the input and number of rendered characters. Also I added corresponding generic functions so user could easily override default values:

(defparameter *textarea-rows* 5
"Default number of rows rendered in textarea"
)


(defgeneric textarea-rows (obj slot-name slot-type)
(:documentation
"Must return a maximum length of user input for a given
slot. Default implementation returns the value of
*max-raw-input-length*."
)
)


(defmethod textarea-rows (obj slot-name slot-type)
*textarea-rows*
)


(defparameter *textarea-cols* 20
"Default number of columns rendered in textarea"
)


(defgeneric textarea-cols (obj slot-name slot-type)
(:documentation
"Must return a maximum length of user input for a given
slot. Default implementation returns the value of
*max-raw-input-length*."
)
)


(defmethod textarea-cols (obj slot-name slot-type)
*textarea-cols*
)


(defparameter *text-max-input-length* 200
"Default value of the text type maximum input length"
)


(defparameter *text-max-rendered-characters* 10
"Default value of text maximum characters that will be rendered"
)


(defslotmethod max-raw-slot-input-length (obj slot-name (slot-type (eql 'text)))
" Returns default maximum input length for the text type, override for customization"
*text-max-input-length*
)

Note that everything is commented, weblocks documentations is automatically generated by tinaa so document your code, your effort will be appreciated. The meat of the text type functionality is consisted of three slot methods:
Render-form-value specializes corresponding generic function with text as slot-type argument. It's job is to call render-textarea with appropriate arguments.

(defslotmethod render-form-value (obj slot-name (slot-type (eql 'text)) slot-value &rest
keys &key slot-path intermediate-fields &allow-other-keys
)

"Textarea slotmethod specializes on rendering string into textarea, simigliar code to render-form-value"
(let ((attributized-slot-name (attributize-name (if slot-name slot-name (last-item slot-path))))
(intermediate-value (slot-intermedia-value slot-name intermediate-fields))
)

(render-textarea attributized-slot-name
(if intermediate-value
(cdr intermediate-value)
(apply #'form-print-object obj slot-name
slot-type slot-value keys
)
)

(max-raw-slot-input-length obj slot-name slot-type)
(textarea-rows obj slot-name slot-type)
(textarea-cols obj slot-name slot-type)
)
)
)

Text type will hold large strings, default maximum is 200 characters, if we render all of them in data-view within datagrid & dataform widgets, both of them will become unreadable. That's why I decided to render only 9 characters fallowed by 3 dots. If the string is smaller than 10 characters it'll be rendered completely.

(defslotmethod data-print-object (obj slot-name (slot-type (eql 'text)) slot-value
&rest args
)

"Renders maximum amount of characters allowed for the slot + ... or the full length of the slot if it is smaller than the maximum length allowed"
(format nil (if (< (length slot-value) *text-max-rendered-characters*)
slot-value
(concatenate 'string
(subseq slot-value 0 *text-max-rendered-characters*) "..."
)
)
)
)


The problem with above specilization is that I screwed up my rendering in the form view. The above render-form-value uses form-print-object that uses data-print-object specialized. So if I enter something like "Very long and boring text " in textarea, it'll be rendered as "Very long..." in data view, that's something that I intended. But it will be also rendered like that in form view, and when we try to modify it the string will be cut to only 9 characters fallowed by 3 dots, something that I don't want. Because I want a full text in order to make adjustements. So I used form-print-object specialised in text type to dispatch printing of the string as usual.


(defslotmethod form-print-object (obj slot-name (slot-type (eql 'text)) slot-value
&rest args
)

"Used to dispatch rendering slot-value to data-print-object with (slot-type standard-object)"
(when slot-value
(apply #'data-print-object obj slot-name t slot-value args)
)
)


I've added test cases for the above functions in a new file text.lisp placed in /cl-weblocks/test/types/


(in-package :weblocks-test)

(defclass text-employee ()
((text-slot :initform nil :type text :initarg :text-slot))
)


;;; Create instances for introspection testing
(defparameter *text-joe* (make-instance 'text-employee :text-slot "Joe"))
(defparameter *text-bob* (make-instance 'text-employee :text-slot "Bob"))


(deftest textarea-rows/1
(textarea-rows t t t)
5
)


(deftest textarea-cols/1
(textarea-cols t t t)
20
)


(deftest-html render-form-value/text1
(render-form-value *text-joe* 'text-slot 'text "Joe")
(:textarea :name "text-slot" :maxlength 200 :rows 5 :cols 20 :class "textarea-class"
(cl-who:str "Joe")
)
)


(deftest-html render-form-value/text2
(render-form-value *text-bob* 'text-slot 'text "Bob")
(:textarea :name "text-slot" :maxlength 200 :rows 5 :cols 20 :class "textarea-class"
(cl-who:str "Bob")
)
)


(deftest data-print-object/1
(data-print-object *text-joe* 'text-slot 'text "123456789012345678901234567890")
"1234567890..."
)


(deftest data-print-object/1
(data-print-object *text-joe* 'text-slot 'text "123456789")
"123456789"
)


(deftest form-print-object/1
(form-print-object t 'text-slot 'text "123456789")
"123456789"
)


(deftest form-print-object/1
(form-print-object t 'text-slot 'text "12345678901234567890123456789")
"12345678901234567890123456789"
)


And the final thing was including my new files into corresponding asd files weblocks.asd and weblocks-test.add to enable automatic loading of my new type. My new type is ready for use, you can try it with specifying new slot as :type text .


cheers
Slobodan Blazeski

Friday, December 7, 2007

Choosing the right lisp implementation

There were ups and there were downs in my road with lisp, but today was one of the days that I wouldn't turn on my computer only if I knew what would happen. Everything started great in the morning. I implemented the functionality of the textarea type for the weblocks framework in a record time. But as soon as I tested with Firefox, as Opera is my primary browser , I found that my new text type is rendered as text input field instead of textarea. After unsuccessful debugging I decided to see how will other browsers behave, like IE & Safari and that ment booting windows.
I started my Lispworks Personal and got myself into nasty problems with some classes that worked great on sbcl & linux but failed miserably with lispworks & windows, so I downloaded the starter pack and reinstalled some libraries, something that solved some of the problems and introduced a new ones. After a while I got nervous and depresses so I bothered Slava,the author of weblocks in a chat with my problems, like how the other people are wasting his precious time while I was doing the completely same thing, and out of my frustrations proposing that he should proclaim sbcl standard and dump support for all other implementations until weblocks goes production ready. A proposal that he rightfully denied.

So the question with my problem boils to one thing. What lisp implementation should you use for your development? For a starter it doesn't really matter, just grab one that didn't stopped being developed in 1993 and start your lisping. The real question goes to those who achieved certain jedi-apprentice level, and want to use lisp for real problems. Leaving financial and religious beliefs aside, I found one simple advice that goes like this. Use the same implementation that authors/mainteneers of the library you need happens to use.
I see you starting to disagree with me saying that many of the lisp libraries use only the ANSI part of lisp, which would make them portable across all complying implementation, or maybe the library you like is well supported across many OS & lisp
Well I haven't been that lucky myself. The libraries I liked happened to work best with sbcl/linux . The authors of those weren't one of those yobos you probably encountered dwelling in c.l.l that are pro sbcl/linux and against everything else.
Absolutely not, those were nice people trying to help but I was always only one using it with ACL & LW under windows, well except one guy, what was his name? Oh the one that hang around with them couple of years ago.
I hear the voice that library could be patched to work with your beloved implementation. Of course it can. But someone have to do it . The question is who?
Well how about the authors and community? They will try to help and they understand the system. But they will be shooting blindfolded because they don't see the problems. Everything works smooth on their systems, it's you the one having problems. Last time something like that happened was with Allegro IDE under linux. I was having problems with the latest Ubuntu 7.10 while Ken Cheetham was debugging it from some far older version where everything worked great. Franz is great company, just look at their sexy products called AllegroCache & AllegroGraph, and Ken did a great job developing a patch, thank you Ken. But in the vacuum until he gain understanding that the problem was not in the setup, missing library or something like that, I felt again like failing to explain to nice folks from Elephant that I'm unable to compile FiveAm under Lispworks and do their tests.

So let's see the famous fix it yourself solution. If you're someone like Edi Weitz, taking my hat down, than you could certainly fix it while you're drinking your first morning coffee and reading Süddeutsche Zeitung at the same time . Damn if you're like Edi you could make one from scratch, portable across any lisp / OS combination and with superior speed and documentation while those bozos who made the original crap, advance from version 2.0 to 2.00001 . But if you are like me, you'll need much longer to understand it to make a good solution. And every day spent patching libraries is one day less working on your application.

happy lisping
Slobodan Blazeski

P.S.
Daniel Weinreb did an extensive survey of the all of the ten currently supported implementations. Youd can find details at Common Lisp Implementations: A Survey

Wednesday, December 5, 2007

Delimited continuations with cl-cont

As continuations are becoming popular for web-development and with growth of web frameworks and servers using them like: PLT Scheme Web Server, Seaside for Smalltalk, Apache Cocoon and finally Uncommon Web and Weblocks for Common Lisp, like most of us who never programmed in Scheme I didn't understood what are they? After searching the web, reading dozen of articles and revisited On Lisp especially the chapter about continuations which I skipped last time I finally understand them and found that they are quite simple concept. Continuations is an abstraction representing the rest of the program at some point. Lets make it clear through some examples. Common Lisp doesn't have continuations built-in so I will use cl-cont package as demonstration facility. Cl-cont implements delimited continuations for the needs of the weblocks framework.

In the below program :

(with-call/cc
(+ 1 (call/cc
(lambda (k)
(funcall k 3)))))
4

the rest of program is simply adding one, while the current program is the whole between the square brackets as seen in (+ 1 []).
Why should something like this be useful to anybody ?
Well the continuations could be saved and than called at any time, let's see that:

(with-call/cc
(+ 1
(call/cc

(lambda (k)
(setf cc k)
(funcall k 3)))))
4


In above we saved current continuation, read adding one, into the variable cc , now we could call it, like this :

(funcall cc 1)
2
(funcall cc 10)
11

Let's do little bit more complex example :

(with-call/cc
(list (+ 1 (call/cc
(lambda (k)
(setf cc k)
(funcall k 3))))))
(4)

(funcall cc 77)
(78)

Ok now we are ready for our first non trivial thing, implementing generators.
Let's implement a generator that returns multiplies of seven using continuations, the code

(defun test ()
(with-call/cc
(let ((i 0))
(call/cc
(lambda (k)
(setf cc k)))
(incf i 7))))

We start the generator by simply calling

(test)
#

and than we call the saved continuations, like this:

(funcall cc)
7
(funcall cc)
14
(funcall cc)
21

The reset is done by calling test again:

(test)
#

(funcall cc)
7

Ok enough with toy examples. Imagine that newbie came into c.l.l asking for help about his homework, saying write a program that flattens a tree. And though you sympathize with a poor kid that have to learn artifical intellegence and half a dozen programming languages in a single course,he didn't tried to show you that at least he opened his copy of Practical Common Lisp. Such ignorance can't be tolerated. You look at your poster of Kenny Tilton, the one you bought as soon as you finally understood Cells, even without documentation, and write something like below


(defun tree-gen (tree)
(let (saved)
(labels
((get-atom (tree)
(with-call/cc
(cond ((null tree) nil)
((atom tree) tree)
(t (call/cc
(lambda (k)
(push #'(lambda ()
(funcall k (get-atom (cdr tree)))
)

saved
)

(get-atom (car tree))
)
)
)
)
)
)

(yield ()
(if (null saved)
(values nil nil)
(let* ((cont (pop saved))
(val (funcall cont))
)

(if val (values val t) (yield))
)
)
)
)

(let ((res (list (get-atom tree))))
(loop
(multiple-value-bind (val more)
(yield)
(if more
(push val res)
(return (nreverse res))
)
)
)
)
)
)
)

Better formatted version at http://paste.lisp.org/display/51958
And you can call it like this:

(tree-gen '(((0 (1))) (2 3 (4)) (5 ((((6) 7) 8 ) 9) 10) 11))
(0 1 2 3 4 5 6 7 8 9 10 11)


Few references that helped me understood continuations:
1.Tim Peters thread at
http://mail.python.org/pipermail/python-dev/1999-July/000467.html
2. Chapter about continuations at Teach Yourself Scheme in Fixnum Days:
http://www.plt-scheme.org/software/drscheme/
3. Paul Graham chapter about continuations at On Lisp
http://www.paulgraham.com/onlisp.html

And finally you can find cl-cont project at:
http://common-lisp.net/project/cl-cont/ also take a look at weblocks framework that's using it at http://common-lisp.net/project/cl-weblocks/

kind regards
Slobodan Blazeski