Extracting Immutable Objects
Code: design, design patterns, email, ListLibrary.net, mailing lists, object orientation, Ruby
In the last few weeks I’ve been rehabilitating some of the first object-oriented code I wrote in the hopes of bringing my mailing list archive back online. Lately I’ve been refactoring some of the earliest, core code: the
Message class. It manages the individual emails in the system and, because I didn’t understand how to extract functionality, had turned into something of a God Class.
Yesterday I tweeted about a really satisfying cleanup:
ListLibrary.net UI Design
Code: graphic design, ListLibrary.net, mailing lists, user interface
I wanted ListLibrary.net to have a really nice user interface for browsing and reading mailing lists at length. The design leans towards terseness: get rid of the clutter and assume that people who’ve browsed a page or two will understand how things are sorted out.
I started on the list info page, and I started from the top. Instead of putting the name ListLibrary.net in as a heading, I put it down in the footer, the name of the list is the title for the page.
After that, I knew I wanted to display information about the list like homepage and description, but it’s something you’ll probably only read once. So I let it move down the page a bit.
What slid in under the title was the table of links to month pages. I’m really happy with this table, even though it takes a little puzzling out. Instead of linking down into pages for years or quarters, the table displays the number of messages and threads in each month, trading immediate clarity for depth.
The month page is the first place the thread-centric nature of the site becomes apparent. Rather than immediately list all the individual messages, they’re tucked away until you click the blue bar to expand one (or double-click for all).
I touched on the thread list in my announcement post, but it’s where I spent the most time tweaking so I’m going to write a bit more about it.
Rather than repeat the subject (which rarely changes), it displays the author’s name to make it easy to pick out posts from favorites. If the subject does change, the new subject is displayed off to the right. The date and time (always UTC, an OK compromise) is less prominent and, though this screenshot doesn’t show it, replies that happen to fall in later months will be pulled forward to appear in one single thread. You can also see the vertical line that makes it easy to pick out sibling replies.
The thread view pages repeats that thread list and then displays the messages. I played around with indenting them like the thread list, but they look much better full-width, especially on deeply-nested threads.
The header has the usual name, email (spam-protected but hopefully recognizable if you already know it), timestamp, permalink. When you hover over a message, it shows you a bit more information in the header: easy links to a message’s parent and replies.
The only other design note worth mentioning is typography. The page is high-contrast, clean type, with a vertical grid for easy reading.
I’m happy with how this design works. I think the attention I paid to these small details has paid off, and it’s a lot more attractive than, well, pretty much anything else I’ve designed. I’ve developed it on-and-off over the last 18-months, so it’s had a lot of time to give this polish.
ListLibrary.net is officially launched.
I built a site to read mailing list archives because of the list MUD-dev. It’s a high-quality, all-signal discussion of online game developers. The authors are some of the creators of the current crop of massively multiplayer virtual worlds, and the archive collects their wisdom.
I really wanted to read through the archive, but I wasn’t happy with the other presentations I’ve seen of it. The basic unit of other archives is the message: a page contains a message and links to its parent and replies, you can click between them. On ListLibrary.net, the atom is the thread. When messages are short and quote each other, it’s easier to skim and or read the entire conversation than on other sites or in mail clients that only show a message at a time.
As mentioned yesterday, the ListLibrary.net display is:
The difference is stark: instead of dominating the display with the repeated subject, the subject is highlighted once and the authors and structure of replies are the focus. It’s easy to see when conversation happened or pick out messages by particularly interesting authors or number of replies. The screenshot doesn’t show it, but hovering over my thread list shows a subtle vertical line to make it easy to pick out sibling responses.
So far I’ve only loaded a few lists I had handy archives of (about 1.3 million messages): mud-dev, its
successor mud-dev2, some Ruby lists, some local Chicago programming group lists, and theinfo.org lists. I think I may also have the largest contiguous archive of the Linux kernel mailing list, a very busy mailing list that’s been an excellent stress test during development. I’ve found a few dozen mailing lists of other open source projects I’d like to archive as well, so the site will continue to grow.
Of all my projects, ListLibrary.net has the least potential to earn money, but its goal is to maintain a public resource. Hosting a few gigs of text is cheap. The site will display text ads on the first page of a visit from a search engine to offset the expense, and I’ll explore other options if it becomes a serious drain.
Code: command/query separation, email, ListLibrary.net, referential transparency, threading
Objects contain both state (data) and methods, and methods should be classifiable into commands that change state and queries that introspect state. The principle of Command/Query Separation (CQS) expresses a design principle I’ve intuitively used as a rule of thumb. With the conscious consideration that comes from hearing it, I knew how to improve some of my own code.
I violated CQS in part of the design of ListLibrary.net (mentioned here). It was both beneficial and harmful (especially in debugging).
Emails are threaded together to show the flow of responses. In this example Chris, Mike, and Jon all responded directly to Tiago and there was a series of responses to Mike. It’s difficult to put these threads together well because email clients often don’t (accurately) include the metadata to specify the message an email is responding to. You sometimes just have to guess what goes where, and it’s further complicated by messages that are lost or (in date-organized archives) split off into other directories.
I started with the threading heuristic by JWZ. It’s a well-described, often-implemented bit of code. I implemented in two steps: first, as messages are added, their references are examined to do some basic relatinships between them (JWZ’s step 1). The rest of the steps require more computation and are more likely to be invalidated as messages are added, so I do them lazily: the threading is calculated (and cached) when it’s requested… violating CQS. The query to list the threads in a message also acts as a command to generate them.
This is both a performance win (yes, I tested) and leads to better-threaded messages, using all available messages are in reduces the amount of ambiguous situations it has to guess at, which can persist even after adding other messages that would have resolved it. But it’s also a great way to make my life difficult in testing: when I insert debugging queries they can trigger commands that change the way messages are threaded. It creates heisenbugs, bugs that appear, disappear, or change when they are being debugged.
I was really frustrated by heisenbugs for a while: I need to violate CQS for performance, but I’m creating all this extra work for myself in the most complex piece of the codebase. The solution is to create referentially transparent query commands specifically for debugging. They’ll introspect on the internal data structures and present them in their raw forms without triggering threading (making them useless for non-debugging work, but that’s OK), and their tests can confirm this.
I find the CQS principle to be a really useful tool. Its application leads to better-designed and easier-test code, and its violation can indicate bad code (a Code Smell).
My last day at the Post is Feb 20, and I’m headed to Chicago on the 22nd. I’ll be helping a family member recover from surgery, so my schedule (both day-to-day and how long I’ll be in town) is pretty vague, but I’ll be around at least a few weeks before returning to DC.
I have three smallish website projects I plan to finish in this time. They’re in various stages of completion now.
The first is a mailing list archive. I’ve been tinkering with it on and off for about 18 months, and it’s the closest to a finished state. I really enjoy reading mailing lists, but haven’t been impressed with any of the sites out there. Often each message is its own page and threads are broken up. So I’ve done a lot of work on threading above and beyond the basic In-Reply-To and Re: matching, and the site is designed to show a thread per-page with lots of keyboard controls to make it easy to skim. Depending on free time this weekend, I may actually launch this before I leave town.
Second is a site cataloging Ruby gems. When I was searching out Rails forums I realized there’s a related problem in finding libraries that I could address. I’ve done the work to collect basic info on gems, it remains to collect news and blog posts and combine it into a nice presentation.
The last small project is my RailsRumble project (which I didn’t finish): a site for collaboratively producing transcripts of conference presentations. I’ve let this one simmer in the back of my mind in the four months since that failure and I’ve given it a better name and planned which features to finish, redesign, or drop. It needs a few days of solid development, a graphic design, and some seed content.
I plan to finish and launch these three projects by the end of March, and I’ll post more as I work on them.
The commonality in these three projects is that once I’ve launched them they require minimal ongoing attention from me and can grow at their own paces. I’m doing them first because small and well-defined projects fit well with the interrupted and distracted time I’ll have. I’d also want to make sure I start out with some successes.
Next posts: NearbyGamers and The Big Project That I Really Need To Name Already.
Fatal error: Call to undefined function twentyseventeen_get_svg() in /home/malaprop/push.cx/wp-content/themes/pushcx/archive.php on line 45