Where Hep is Going
With Hep 0.3 out, and the known bugs fixed, I’m thinking about where to go next. Here are my thoughts, for everyone else to read and comment on - sort of my "State of Hep" address.
What is Hep?
I don’t think I’ve ever really taken the time to explain what my vision for Hep is. If I have, I haven’t explained it well. So I’ll try to do so here.
Hep is a message server. It’s a server in the sense that it’s a program without a graphical user interface, that runs in the background, waiting for other programs to connect to it over the network. I call it a "message server" because it does things with messages, which as far as Hep is concerned are any little bits of text or HTML that you want to read, or save, or publish, or pass on to somebody else, or convert into a different format, or organize.
The goal of Hep is to make it possible to work with messages in all these ways, without having to worry about where the messages are, what format they’re stored in, or what protocol you have to use to get at them. Hep lets you use a program that was designed to to work with a specific kind of messages (like an e-mail client), and use it to work with all kinds of other types or messages (like RSS news feeds, news groups, and your weblog).
Looking Around
There are a lot of interesting message-related projects going on right now. ZOE is a message server that handles only e-mail, but has a nice web interface and lets you do cool stuff with searching messages and seeing how they relate to each other. Spaces is an GUI PIM app that can work with RSS as well as e-mail. Apparently Spaces is also going to support sending messages to weblogs, as well as some kind of peer-to-peer framework. Chandler is another GUI PIM application that will do more than just e-mail, although it’s still in the planning stages at this point. And of course there’s Radio and a whole lot of similar tools for working with RSS feeds and weblogs.
What makes Hep different from all of these is that it’s not tied too closely to any one type of message, or messaging protocol. Hep isn’t an e-mail tool, or an RSS tool. It’s more of a universal message tool.
You can connect to Hep using ZOE, Spaces, or Radio if you want. You can also use a traditional PIM program like Outlook, or Unix tools like fetchmail, procmail, and wget. And Hep will let you connect these tools in ways that were never possible before.
Where Hep is Today
Hep 0.3 supports the idea of sources and destinations. A message source is a place Hep can find new messages. Out of the box Hep supports RSS news feeds, and diaries on Advogato (or other sites that use the Virgule community software). Hep pulls messages from the sources you set up, and then stores them in it’s Inbox. You can read the messages in your Hep Inbox by connecting to Hep with your browser. Or you can download them into your e-mail program by connecting to Hep’s built-in POP server.
A destination is somewhere you send messages to. Hep 0.3 supports most popular weblog systems as destinations. Once you’ve set up a destination in Hep you can compose a message in your e-mail client and send it through Hep’s SMTP server. Hep will deliver it for you, whether that means sending the message to an e-mail recipient or posting it to your weblog.
Where Hep is Going
Getting messages from sources and sending messages to destinations is nice, but it’s also limiting. All you can do with a source is read message from it, and a destination is basically a black hole that a message disappears into, never to be seen again (at least not in it’s original form). In some cases this is an accurate representation of what’s actually going on - an RSS feed really is read-only, and an instant message really does vanish into the network - but in many cases looking at messages this way leaves out a lot of possibilities.
The next step for Hep is adding support for message stores, collections of messages that you can read, write, delete from, add to, and edit. Hep should be able to provide access to messages in its own internal folders, as well as remote stores like weblogs, newsgroups, and IMAP mailboxes. Of course this means that Hep will have to include some new server protocols: IMAP and WebDAV.
IMAP is the protocol I’m most excited about. Almost all popular e-mail clients support it, and it’s a designed specifically for managing messages. Hep’s IMAP server would let you work with both internal and remote stores, so your weblog will look like just another folder full of messages. Want to publish a set of e-mail messages as a weblog? Just copy the e-mails into the weblog’s folder, and Hep will automatically convert and publish them. IMAP also uses a constant connection between the client and server, and lets the server notify the client of changes. This would let users see new messages as soon as they are recieved, which is important for instant messaging.
WebDAV is less immediately useful, but it’s an interesting possibility. WebDAV is a protocol for working with remote files, supported by Windows 2000, OS X (I think) and Linux file managers like Nautilus. If Hep included a WebDAV server, you could manage messages with your file manager, dragging and dropping to move them around, and edit them with standard tools. I’d like to make the Hep WebDAV server support different ways of viewing messages, so you could make the same message store appear as a folder full of HTML files, or a maildir directectory, or a single RSS file. And what would be really cool is if you could switch views on the fly - to update an RSS file, just switch to the "bunch of HTML files" view, drag-and-drop the HTML file you want to add, then switch back to the RSS view and you’ve got a new RSS file that includes the HTML you just added.
FTP is a possibility here too, although I don’t think it offers the capabilities that WebDAV does.
Storing messages within Hep also will make it possible to search, filter, and group messages in interesting ways, and easily access the same messages from different client applications.
Making it Happen
So there’s a lot of work to do. Right now my TODO list looks something like this:
- Add support for storing messages, creating sub-folders, etc.
- Upgrade weblog drivers to support viewing and editiing messages as well as posting
- Write the IMAP server (Twisted doesn’t include IMAP support)
- Write plugins for Jabber, AIM, NNTP, and other protocols
- Write the WebDAV server (One of the Twisted developers is already working on support for the DAV protocol)
- Improve user interaction, error reporting for undeliverable messages, etc.
- Improve the web interface
- Add support for searching and filtering messages
- Build installers for Mac and Windows, and RPMs and debs for Linux.
If you’re interested in Hep, now is a great time to get involved with the project. Even if you’re not an experienced Python programmer, you can make a significant contribution. The more people who are using Hep, testing it out, finding bugs, and making suggestions, the faster the development will go. If you’re interested, send me an e-mail, or join the developer’s mailing list.
