Hirundo: Comfortably follow Swift Mailing Lists on OSX

✎ Published: 2016-02-02 10:30:30

Hirundo is a Mac App for reading the Swift mailing lists like Swift-Evolution or Swift-Dev. You can even bookmark messages of threads because they contain important information

Hirundo has been deprecated and is no longer available

If you're interested in Swift, the mailing lists are a fantastic resource. https://lists.swift.org/mailman/listinfo/swift-evolution discusses the future of the language and includes many interesting messages about the shape that the language may become. Following it also really helps understanding the bigger idea behind many of Swift's most interesting features. https://lists.swift.org/mailman/listinfo/swift-dev, on the other hand, goes down into the details of how Swift works internally. If that's not your thing, there's https://lists.swift.org/mailman/listinfo/swift-users which covers questions and discussions about Swift from a user / programmers point of view.

Following those lists is not only useful, but also brings tremendous benefits. However, following them is also kinda hard:

  • The web archive is not easy to navigate
  • When you subscribe to a list you're bound to the UI of a mailing list / your Mail app
  • You can't see any messages posted before you subscribed
  • Many convenience functions (Bookmarks, Muting, etc) are not available

This bugged me. I wanted to be able to follow the discussions there, but I'd rather have a nice UI for them. So over Christmas I started working on a small app to solve this dilemma.

Enter Hirundo

Hirundo is the result of this (Written in Swift, of course). The word is latin and means Sparrow or Swift, so I thought it'd be a nice fit and it sounds cool.

The app is currently in beta as I'm trying to iron out the remaining bugs. I'm already very happy with how it turned out and with the current set of implemented features. Among other things, the app supports:

  • Follow multiple Swift lists
  • Browse and search everything posted to the lists
  • Detailed search facilities over all messages in all lists
  • You can mute threads so that they don't appear in the lists anymore
  • If you are subscribed to a list, Hirundo will open Mail.app with reply headers set
  • Read messages from selected authors, search for authors, or even manage favorite authors
  • Sort messages by various properties, including a Rank showcasing the most discussed threads
  • Messages are threaded, you can fold them or collapse them
  • Bookmark any message or thread into multiple folders and revisit them later
  • Reply to threads or start new threads

Building it

Writing the app took longer than I initially expected. I sometimes tend to be naive about technical limitations and so I just started working on this app without really assessing the difficulty of implementing it. The biggest issue was that the Swift mailing list data is not available via a REST API or something which else that one could query. There're only two ways of accessing the data:

Parsing the HTML archives

However this has several drawbacks. It requires a ton of =HTTP= requests, and it is very slow crawling the server for them. Those =HTML= pages use very simple, non-semantic =HTML= which makes it difficult and error prone to parse them correctly. Even worse, it lacks certain data which is required to properly establish the message tree afterwards.

Parsing the mailing archives

This is what I used. The archives are in the Unix =mbox= format so there's a way better chance of parsing them properly. They also contain all the information required to build up the message threads afterwards. I decided to use [https://github.com/MailCore/mailcore2](MailCore 2) for the actual parsing of the mails in the =mbox= so that I'd not need to reimplement another https://tools.ietf.org/html/rfc2822 parser. However, a couple of small differences between RFC2822 and the mbox format generated by https://en.wikipedia.org/wiki/GNU_Mailman required additional effort in order to properly format the mbox contents so that MailCore could read them. The messages are then read, parsed, converted, and finally placed in Core Data.

Real Time Messages

After playing with the app for a while, I realized that the mailing list archives were not updated in real time. Instead, the compressed archives are only generated once a day. This makes the whole approach obsolete for actual following the discussions. This was also when I realized I should have invested more time up front researching this. Since the =HTML= was not feasible for import I was at an impasse. The solution, it seemed, was to host the temporary real time messages myself.

The next step (again, this took way longer than expected) was setting up an =smtp= email server which receives mails to mailing lists that I subscribe it to, and writes them out into the =mbox= format. A small webserver then offers those files as temporary caches of current mailing list activity. As soon as the official mailing list archives are updated, the cached ones are deleted. If you're running [https://www.obdev.at/products/littlesnitch/index.html](Little Snitch) this is what happens when you see Hirundo download data from =hirundo-lists.stylemac.com=.

Replying to Messages or creating Threads

When you click reply or new Hirundo opens your standard mail client with a mailto: call embedding all required mail headers for a proper reply. However Mail.app does not respect one of the required headers (=In-Reply-To=) so I had to use the (=References=) alternative header instead. That is a working solution but not optimal as it makes it more difficult for Pipermail (the mailing list) to properly thread your reply to the correct location. Other mail clients (such as Thunderbird) don't seem to have this problem. A future solution would be to send the mail directly from within Hirundo.

Building the UI

Once that finally worked, I started working on the UI of the app. I decided to limit it to 10.11 El Capitan, so that I could play around with all the new and exciting technologies released last WWDC. As it happens, I haven't even reached 1.0 and there's already a list of upcoming refactorings I'd like to perform as some initial design decisions were not as good as I thought they'd be in hindsight.

Mac App Development

I wrote a couple of interesting small classes for this project, and I'll spend the next posts on appventure building a simple Mac App based on some of the Hirundo code. There's currently a certain lack of Swift OSX development information, and I'd like to fill this void a bit.