Writing HTML in Lisp

I assume if you are reading this you are already motivated to write HTML in some language other than, er, HTML, which is at best tedious and, not to put too fine a point on it – butt-ugly, when it gets down to actually reading it (like all markup languages).

If you've worked with any of the GUI tools that make web pages, you've no doubt been horrified later when you looked at the code they produced (like opening up a wall in the movie Brazil) – it may be Turing-equivalent but who'd want to prove it?

Inside a wall (from the Movie: <em>Brazil</em>)

What most self-respecting (lazy) programmers do is write their own HTML-generating tools in their favorite language. I prefer Lisp, which is particularly well suited for this sort of thing.

CL-WHO

I didn't start from scratch. There are a number of basic HTML-writing Lisp packages around. CL-WHO written by Edi Weitz containes a fairly complete set of primitives which work very well. But after putting together a few dozen web pages design patterns begin to cry out for greater abstraction.

CL-WHO-EXT

I'll use the first part of this page as an example so you can see what I'm talking about. Here is what the Lisp looks like to generate this page up to this point:

  (make-page-generator
      (cl-who-ext :title "Writing HTML in Lisp" 
                  :description "Writing HTML in Lisp."
                  :css-files '("special/jkc-horz-menu.css"
                               "special/jkc-site-2a.css")
                  :js-files js-files
                  :js-code local-javascript)
    (with-normal-body
      (:h1 "Writing HTML in Lisp")
      (:p "I assume if you are reading this you are already
      motivated to write HTML in some language other than, er, HTML,
      which is at best tedious and, not to put too fine a point on it
      – butt-ugly, when it gets down
      to actually reading it (like all markup languages). ")
      (:p "If you've worked with any of the GUI tools that make web
      pages, you've no doubt been horrified later when you looked at
      the code they produced (like opening up a wall in the movie
      Brazil) – it may be Turing-equivalent but who'd want to
      prove it?  "
          (float-right
            (embed-image  "brazil-wall4-t.jpg"
                          :alt "Inside a wall (from the Movie: Brazil)")))
      (:p "What most self-respecting (lazy) programmers do is
      write their own " (:i "HTML-generating tools")
      " in their favorite language. I prefer Lisp, which is
            particularly well suited for this sort of thing. ")
      (:h3 "CL-WHO")
      (:p "I didn't start from scratch. There are a number of basic
        HTML-writing Lisp packages around. "
          (ahref "http://weitz.de/cl-who/" "CL-WHO")
          " written by "
          (ahref "http://weitz.de" "Edi Weitz") " containes a fairly
            complete set of primitives which work very well. But after
            putting together a few dozen web pages design patterns
            emerge which call for further tools to further simplify
            the process. ")
      (:h3 "CL-WHO-EXT")
      (:p "I'll use the first part of this page as an example so you
      can see what I'm talking about. Here is what the Lisp looks like
      to generate this page up to this point:")
      (embed-code "Ah-ah-ah! no infinite recursions allowed!")))

As you can see, the code is pretty simple. The main call creates a function named CL-WHO-EXT available to be called by the webserver whenever it needs to draw the page cl-who-ext.html. This function is created by a macro - one of the most beautiful things about Common Lisp.

This macro takes care of creating all that tedious stuff you do to create a page, like the DOCTYPE, the section with the meta-tags, embedding your lists of javascript files, css files, icons, etc. That's what the keywords are for between the top and the "with-normal-body" — you just give it lists of tags, words, phrases, and files and it takes care of the formating.

Syntax

In the body, we're looking at something like HTML with keywords instead of tags and prefix notation. For example heading lines are delineated by parenthesis and a leading :hn keyword indicating the header level ":h1", ":h2", etc. Paragraphs are encapsulated by parentheses with a leading keyword :p followed by a quoted string. Basically, keywords encapsulate the properties of everything that follows inside the parens. And like CSS, parens can nest.

Virtually any HTML tag translates into a keyword preceded by a colon: ":a :href :img :src :class :id and so on. The first keyword tag works like a function acting on all the arguments which follow. any subsequent keywords are assumed to be property=value pairs, but in this grammer you don't need to put the "=" sign, it is implied. That means to embed an image all you need is the following:

(:img :src "/virtual/img" :alt "my alt text")

Functions can be embedded that output HTML. This allows you to build your own tools.

Okay, so where's the code for the menubar and tools like that you ask? Fair enough — here's a couple:

  (defun dropdown-expander (script list &optional branch)
    (dolist (item list)
      (who
        (:li (:a :href
                 (format nil "~a?~:[~a~ ;~:*~{~a~^-~}=~a~]" script (reverse branch) (car item))
                         (str (cadr item)))
                 (when (caddr item)
                   (htm (:ul (dropdown-expander script (caddr item) (cons (car item) branch))))))))))

  (defun navbar ()
    (who
      (comment "Horizontal Menu")
      (:div :id "horiz-menu" :class "moomenu"
            (:div :class "wrapper"
                  (:ul :class "menutop"
                       (dropdown-expander "jkc.html" main-menu))))))

It's a recursive tree-parser, basically. It works great. You define your menu structure using lists of lists like below and off it goes:

  (progn
    (defparameter about-menu
      `(("contact"    "contact")
        ("website"    "website")
        ("resume"     "resume")
        ))
    (defparameter projects-menu
      `(("robotics"   "Robotics")
        ("1wire"      "1-Wire Weather")
        ("motion"     "Visual Detection")
        ("light"      "Light Sources")
        ("ts7200"     "TS-7200")
        ))
    (defparameter program-menu
      `(("cl-png"      "Image Processing")
        ("simplot"     "Graphing with Lisp"     )
        ("quaternions" "quaternions")
        ("photopages" "make-photopage")
        ("cl-who-ext"  "HTML in Lisp")
        ("pwc"         "PWC driver")
        ))
    (defparameter linux-menu
      `(("ubuntu"     "my Ubuntu")
        ("debian"     "my Debian")
        ))
    (defparameter resource-menu
      `(("linux"      "Linux"       ,linux-menu)
        ("emacs"      "Emacs")
        ("lisp"       "Lisp")
        ("slime"      "slime")
        ))
    (defparameter main-menu
      `(("home"       "Home"       )
        ("about"      "Info"        ,about-menu)
        ("projects"   "Projects"    ,projects-menu)
        ("programs"   "Programming" ,program-menu)
        ("resources"  "Resources"   ,resource-menu)))
    (defparameter jkc-page-list
      `((("about-" "about-website") "/about.html")
        (("about-contact") "/contact.html")
        (("about-resume") "/jkc-cv.html")
        (("projects-robotics") "/s/jed.html")
        (("projects-1wire") "/weather.html")
        (("projects-motion") "/motion.html")
        (("projects-light") "/light.html")
        (("projects-ts7200") "/ts7200.html")
        (("programs-cl-png") "/motion2.html")
        (("programs-simplot") "/simplot/user/simplot.html")
        (("programs-quaternions") "/quaternion.html")
        (("programs-photopages") "/make-photopage.html")
        (("programs-cl-who-ext") "/cl-who-ext.html")
        (("programs-pwc") "/pwc.html")
        (("resources-linux-ubuntu") "/ubuntu.html")
        (("resources-linux-debian") "/debian.html")
        (("linux-network") "/network-page.html")
        (("linux-security") "/notes.html")
        (("resources-linux" "linux-") "/linux.html")
        (("resources-emacs") "/emacs.html")
        (("resources-lisp") "/lisp.html")
        (("resources-slime") "/slime.html")
        )))

I am embedding these scripts here using my EMBED-CODE macro:

  (defmacro embed-code (&body code)
    "Macro puts code script in a rectangular block using a single-cell
    table with script class. This version writes to standard output. "
    `(who (:div :class "script"
                (:table :border 0 :cellpadding 0 :cellspacing 0
                        (:tr (:td (:pre (str ,@code))))))))

The image at the top was embedded with an embedding-function created specifically for the image source and virtual locations by a macro. These are handy because the paths can be fixed once and forgotten about and don't need to pollute the code space

  ;;; This makes the function EMBED-IMAGE for images which go in /images/
  (make-image-embedder 'embed-image (merge-pathnames "images/" htdocs) "/images/")

Outline formatting

I have also included tools for automating the production and linkage of TOCs (tables of contents) coordinated with (see, for example, my emacs page). I will describe this in detail at a later date.

Obtaining the latest snapshot

I've wrapped my extensions into a package called CL-WHO-EXT. You can obtain the latest snapshot by following the link.