Open source clients like Thunderbird or Zimbra are a good place to start. But sometimes it’s faster to watch an exchange than trace code. What about a TCP dump? Sometimes the server only responds to SSL and it’s not possible to sniff the handshake. Or even if you record a session, the IMAP server might have compressed parts of it. Annoyances such as these are what prompted me to write an IMAP proxy, making it easy to inspect any IMAP conversation.

To run the script you’ll need node.js and coffee-script (via npm). To start the script, just run coffee imap_proxy.coffee. Then configure your email client to connect to your machine on port 3737. Disable SSL. Save your settings and watch the conversation unfold!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

#!/usr/bin/env coffee

tls = require("tls")
net = require("net")
Socket = require("net").Socket
state = {}

IMAP_SERVER = "imap.gmail.com"
GREEN_TEXT_CODE = '\x1b[0;32m'
RED_TEXT_CODE = '\x1b[0;31m'

clientListener = (connectionToClient) ->
  # This callback is run when the server gets a connection from a client.
  isConnected = true
  console.log "Connected to mail client"

  connectionToClient.on "data", (data) ->
    console.log "#{RED_TEXT_CODE} Mail client: ", data.toString()
    connectionToImapServer.write(data)
  connectionToClient.on "end", ->
    isConnected = false
    console.log "Disconnected from #{IMAP_SERVER}"

  # Now that we have a client on the line, make a connection to the IMAP server.
  state.conn = new Socket()
  connectionToImapServer = tls.connect(
    socket: state.conn,
    # Skip TLS certificate verification. Don't use this script on sensitive data!
    rejectUnauthorized: false
  , ->
    console.log "Client connected"
    state.conn = connectionToImapServer
  )
  connectionToImapServer.on "data", (data) ->
    console.log "#{GREEN_TEXT_CODE} #{IMAP_SERVER}: ", data.toString()
    return if !isConnected

    # Do something sneaky during the CAPABILITY exchange. Remove "DEFLATE" from
    # the list (if present) so this proxy doesn't have to decompress messages.
    if data.toString().match(/CAPABILITY.*COMPRESS=DEFLATE/)
      minusCompress = data.toString().replace "COMPRESS=DEFLATE ", ""
      console.log "Proxy substitution: ", minusCompress
      connectionToClient.write(minusCompress)
    else
      connectionToClient.write(data)

  state.conn.connect 993, IMAP_SERVER

server = net.createServer(clientListener)
server.listen 3737, ->
  console.log "IMAP proxy is listening on port 3737"

I hope this helps on your next IMAP adventure. And if you’re passionate about IMAP, Handle is hiring!

As a Rails app, it would be natural to break the data into a few tables: “roommates”, “applicants”, “ratings”, “appointments”, and “correspondences”. In our case there are only two rows in the “roommates” table: Jeremy and me. Every reply to our Craigslist ad creates rows in “applicants” and “correspondences”. Assuming Jeremy likes an application, it creates a row in the “ratings” table (e.g. roommate: 1, applicant: 37, rating: “approve”). One of us replies, creating a new row in “appointments”, etc. That’s a lot of work, which is why I’m happy to have found a way to do the same thing inside Gmail.

Because Craigslist correspondences happen over email anyway, every Gmail thread is a row in “applicants”, plus >1 row in “correspondences”. Instead of tables for “ratings” and “appointments”, labels can be used to track the status of applicants (e.g. JeremyLikes, JeremyPass, InterviewScheduled, RejectionSent).

Generous labeling affords other useful tricks. Add the “Quick Links” lab and save custom, frequently used queries. Here are some that we find useful:

• Brian likes, Jeremy hasn’t decided: label:brianlikes AND -{label:jeremylikes label:jeremypass}
• Jeremy likes, Brian hasn’t decided: label:jeremylikes AND -{label:brianlikes label:brianpass}
• Needs “thanks but no thanks” note: label:brianpass AND label:jeremypass AND -{label:rejectionsent}
• Let’s set up an interview: label:brianlikes label:jeremylikes -{label:interviewscheduled}
• Needs attention: label:requiresfollowup OR -{label:brianlikes label:brianpass label:jeremylikes label:jeremypass}

I hope this trick comes in handy the next time you need to collaboratively categorize and search some data. One word of caution: as of this writing, there is a bug in Gmail involving labels containing dashes (I switched to camel case after many frustrating and fruitless attempts to fix my search queries). Good luck!