We're a wanted man; everyone is searching for us.
Look under Minneapolis.
Look under Minneapolis.
As expected, Gophermail 2.0 went production in summer of 2008. There have been initial rounds of bugfixes and enhancements to the virtual host framework. Some of this work was done at the request of METNET, which may be adopting Gophermail as their webmail platform.
Overall, we have had very good feedback to 2.0, and I'd like to thank everyone who helped with ideas, suggestions, and testing. The mobile version, in particular, has received a lot of comments and we're happy for that!
In the coming months, I'll be finalizing "styles" documentation on the Gophermail wiki. Style handling in 2.0 is very plug-and-play, and I look forward to having that documented for the U Community.
At this time, however, it's looking likely that Gophermail itself (the codebase) will be in bugfix-only mode going forward due to the strategic direction of the University. The lead developer (that's me, Brian Hayden) has moved on to a position with the Minnesota Supercomputing Institute.
In the near future I'll likely be integrating the concepts of Gophermail 2.0 into the latest version of the Prayer webmail package which it was based on. The major advantage to the newest Prayer is HTML templating, so changing the presentation (the 'V' in MVC, for those of you that like that language) will not require code changes.
This is a large project that will not happen overnight, but I consider it worthwhile to have a clean, efficient, modern open-source alternative to the mostly-poor PHP webmail offerings out in the wild. I'm also excited to fully integrate the web-services portions of the Gophermail 2 beta, which did not make it into production as a proper API.
Thanks again to all of you.
It's been a while. The last couple of weeks have been a mix of usability lab, and me being sick. Usability, as always, was exhausting but very productive. Scripted activities in a lab are only part of the interface picture. There are a lot of things that environment isn't, but one thing it is: a great way to shine a light on any basic stumbling blocks that the designers/developers have become immune to during the process.
Aside from that issue, the other thorny one was the idea of deleting messages. IMAP has a two-step deletion model; you mark a message deleted, and then you "expunge" to permanently remove all deleted messages from a mailbox. It's a model of both safety and efficiency. You aren't moving a message around to delete it, so you're saving on IO, possibly network bandwidth depending on the environment, and lessening the risk of message corruption.
The downside, as we've heard numerous times in both the usability labs and from users throughout the years, is that most people flat out don't like it. They obviously like not having messages permanently disappear as soon as you hit "Delete". They don't like the message sitting there, with its big red "X" and strikethrough and whatever other "hey, I am deleted! I'm different!" indicator might be used. This isn't surprising. From POP3 clients twenty years ago to the defaults of Yahoo! Mail and Thunderbird today, deleted messages go to a "Trash" can. People are used to it, people want it.
The problem is, that isn't what IMAP is designed for. So, we're left thinking about ways to accomplish this. Thunderbird, Apple Mail, and Outlook all take a common approach. By default, deleted messages are copied to a Trash folder. They are also left in place marked for deletion, in proper IMAP style. The client hides these marked-for-deletion originals from the user's view.
What happens when the Trash is emptied?
All the copied messages in the Trash folder are permanently removed, while the originals are left there for all eternity, or until the user executes some sort of "Compact folder..." command. Which, obviously, nobody ever does unless they specifically know they should, because the client is hiding the messages that the "Compact" will remove. How can you expect someone to be a responsible user of a shared resource when they don't know that their software is leaving a trash heap under their feet?
This approach, however, does have advantages in that it satisfies the user desire for a Trash folder, and it leaves deleted messages alone in the original mailbox until directed otherwise. While the latter is problematic for the reasons noted above, it's a good thing in that it's non-disruptive if a person uses more than just that client to access their mail. Imagine you've been using Gophermail, marking messages for deletion but you aren't ready to get rid of them permanently yet; then you use Thunderbird at your desk, and it unilaterally does an expunge for you after you delete one message there. You'd go back to Gophermail the next day to find all the messages you left there gone, and then you'd end up with an expensive paperweight after giving your laptop a Kruschev-style shoe-banging.
So, there is a method to their madness.
But it has a huge cost in wasted storage.
What's a person to do! We're discussing this in an ongoing fashion. In the end, it seems that this approach may be the best idea if coupled with a clear, well-communicated policy like most e-mail providers (from Google and Yahoo to other universities) of removing any deleted messages after thirty days. This is one that is going to take a lot of talking out, but it seems a reasonable compromise.
So I've talked a lot here about accessibility, and that work is still ongoing. But, today is a day for talking about something new and different.
Received Mobile Web Design by Cameron Moll in the post yesterday, and have begun working my way through it. As I've mentioned here in the past few weeks, part of the refactoring of Gophermail towards current "semantic" standards is to make it simpler to target towards various media, including mobile. This path is currently being worn by some of the best, from Apple to Google to a hundred smaller outfits. Our goal is to take some inspiration from the things they've done, while incorporating some of our own ideas and targeting the University environment in specific ways that outside folks simply never can.
But! While that thinking is going on, the first step is simply to have something usable and readable, and to that end I'll conclude this little bit of musing (more details later, I promise!) with this screenshot of Gophermail 2.0 running on an iPod Touch. Apologies for the quality, there's no easy way to do a screenshot so this was taken with the built-in camera on my iMac. Click for full-size.
A week from today, we'll be meeting to finalize usability scenarios and go over the prototype. Five days after that evaluation sessions begin.
As we're in the home stretch now, I'm trying to finalize any feature adds that involve changes to interface elements, and beginning to move on to clean-up. Just like a good woodworker, I do my best to keep my area clean as I work. But I'll be the first to admit that during the iterative prototyping process, sometimes the sawdust accumulates around my feet a little. Today began the process of standardizing the CSS class usage across different areas of the app, fleshing out error handling to do more than writing a message to the console, things of that nature. And, of course, running all pages through the W3C validators.
Inherent in this is confirming that accessibility elements such as labels, fieldsets, and tab order have been employed in applicable areas. Heading elements, which can be used for quickly navigating a page with a high density of information (such as our message list screen), are a particularly interesting aspect when it comes to a web application as opposed to a web page. While most textual pages have some sort of outline structure, explicit or implicit, this isn't necessarily true in an application. Apps can lean in the direction of, oh, call it a widget model. There are widgets on the screen, which you try to group logically in relation to each other, but don't necessarily lend themselves to a hierarchical or list structure.
I've talked to a number of people about this issue and it sounds like I'm not the only one who wonders about this. So: for the moment I'm doing the best I can by having a "heading" (not necessarily always visible to non-screen-reader users, when visual clues render it redundant) for each major segment of the page.
For example, in the following screen shot:
I spent Monday at MinneWebCon, which was a great experience; big thanks to the organizers Kristofer Layon and Peter Fleck, et al. It was a credit to the University to have that quality of a program here.
Moving on to this week's Gophermail progress: work on finalizing some style issues is ongoing. Myself and the alpha test audience have been evaluating some small things that make big differences, such as typeface. Trying to settle on a sans-serif face that is readable without strain in both standard and bold. I'm a bit surprised, honestly, at the challenge this has posed. At least to my increasingly Mr Magoo-like eyes, most of the standard sans-serif fonts seem very thick and compacted in bold. I think we've got it now; if it poses any problems during usability we'll hear about it.
Last week saw finalization of a framework for "web services" in Gophermail, and a proof-of-concept that lets you fetch the the newest N messages for a user who is currently logged into CAH.
Today IMAP searches have received a similar treatment. Remember that all details here are preliminary and subject to change, but here's the outline. A url such as:
will return all unread messages. 'date_op' will be available for date-based searches, 'text_op' for text, etc. The current "proof" for this is a toggle on the message list page to show only unread messages. That may or may not be something that would get enough use to justify taking up space on the app's main interface. Have to evaluate that going forward. Details matter, lack of distraction matters, and anything that doesn't really need to be at the fingertips shouldn't be.
AJAX, Web 2.0. We've been through the hype, the outrageously context-inappropriate deployments, more hype, the dismissals, more hype, and the backlash. Now we seem to be getting on to the part where people are using these techniques to add a layer of responsiveness that makes web applications less frustrating than they once were, while keeping in mind fallbacks for accessibility, mobile, and the like.
So, what can these things do for Gophermail?
That said: the abstraction of the presentation layer has enabled mobile/small-screen and large-font versions, among other things, that hinge only on style sheets. This will enable, going forward, rapid development and modification of environments for a wide array of devices.
This is critical going forward as e-mail on mobile devices moves from being a toy--or something people put up with because they had to--to something that is truly usable, and in demand. The more we get devices with readable screens and low-key, usable interfaces, the more true this will become, and with this shift in the "platform" we're on our way to being able to address these needs in an agile, flexible manner.
I'm curious if people have ideas on ways that web-based e-mail interaction, whether in a desktop browser, a mobile device, or anywhere else, could be improved. Most development seems to focus on emulating the classic desktop e-mail clients, and it's fairly obvious that this isn't the most productive approach.
Background: Gophermail's underlying engine is the University of Cambridge's Prayer webmail application. Written in C, it is in a sense an IMAP client much like a desktop mail application. It encapsulates its own https daemon which spawns a persistent session on the server on login; for the rest of the user's session, they deal with this persistent child, and the child is a full-fledged IMAP client that "just happens" to use a browser for its output.
This stateful nature (and the C codebase) makes it a very efficient app, light years beyond all the PHP webmails out there. No PHP, no apache, all very self-contained and streamlined.
The downside is that Prayer/Gophermail depends on these sessions for nearly all operations. If the session times out or is otherwise interrupted, the user is faced with a "Sorry, your session went away!" and must start over from the beginning. Whether they were just trying to look at a message, or send a draft they'd spent an hour on. It's as if your postman occasionally decided, after picking up your letters, to stop in at the bar and use them for a beermat.
So: In Gophermail 2.0, the interrupted session is (for almost all cases) a thing of the past. What enabled today's proof-of-concept of a "recent mail" web service is a new regime of transparent session recovery. When a user hits the server with a "session"-type URL and the server finds no session, one is silently established, based on the UMN auth cookie in the browser, and the GET or POST request is then honored as if nothing had happened. GETs are fully functional; POSTs will require a little more work but are mostly functional. Just need a posthole digger and some guywire to finalize them...
So, as of this afternoon there is a functional proof-of-concept that exposes the IMAP message list as a web service.
Querying a url (which will likely end up being something like gophermail.umn.edu/recent?range=X) will give you an XML doc that contains the most recent X (in range=X) number of messages for the username that is currently logged into UMN Central Auth Hub in that browser. Likely representation will be something like arrival time, sender, and subject, and a url to jump into viewing that message in Gophermail.
This has been something that's been requested for MyU, among other things. Looks like it will be do-able.
The bug that was causing Drafts and Sent mail folders to appear empty for standard (mbox format) users in the 2.0 test builds has been fixed. Will be deployed to stable-test this morning.
International character support (via UTF-8) has been added to today's GopherMail 2.0 test build and has passed early checks. This was a piece of spam I happened to have in my Inbox and I have no idea what it says, if it's vulgar, let me know and I'll switch it out for something else:
EDIT: I've been informed that this was indeed some NC-17 spam :) , so I have two new examples. One is Japanese from the administrative info on the FreeBSD mailing list archives website; the other is Greek Ministry of Tourism transport info. Both were sent and then received in Gophermail 2.0.
More to come....