UPDATE: Nginx as an IMAP/POP3 proxy Part 2

At Gigahost we are managing a lot of mailboxes for our users.

At the moment these are all located on one high speced server with the outgoing SMTP split to another server.

We allow our users to connect via both IMAP and POP3 and support STARTTLS on ports 110/143 and SSL/TLS on ports 993/995.

Since we are constantly adding new users and these in turn add new mailboxes we are running out of options as to upgrade the current server. Hosting mailboxes via Courier, Dovecot or similar is very IO intensive and therefore in the long run disk IO becomes a problem.

The solution to this is ofcourse to scale the setup to more servers. Some hosting providers do this by simply adding users to a new mail server eg. mail2.example.com, mail3.example.com and so on.

What we would like to do is use a reverse proxy so that the user always connects to mail.gigahost.dk and the proxy ensures that the user is send to the correct server.

Enter nginx.

nginx is mostly known as the reverse proxy that drives sites such as youtube.com, wordpress.com, hulu.com, github.com and many many more.

But nginx can also act as an IMAP/POP3 proxy and does quite a good job at it.

Using nginx you can authenticate the mail user before she/he reaches the mailserver and specify i) if the user can be authenticated, ii) what server the user should be send to. You can infact also alter the username and do other magic stuff.

excerpt from nginx.conf

                http {
                  perl_modules  perl/lib;
                  perl_require  mailauth.pm;
            
                  server {
                    location /auth {
                      perl  mailauth::handler;
                    }
                  }
                }
            
                mail {
                  auth_http  127.0.0.1:80/auth;
                  auth_http_header X-NGX-Auth-Key "some secret";
            
                  imap_auth plain login cram-md5;
                  pop3_auth plain apop cram-md5;
                }
              

In the above excerpts you will see that I use the embedded perl module in nginx (you must add this a compile time). This serves up the mailauth.pm script on port 80.

Be aware that the embedded perl parser blocks the current nginx process – so you might consider running a few and ensure that the script executes fast.

The auth_http setting in the nginx config is where the magic happens. This points to the HTTP server that handles the authentication and find the server the connection should be proxied to.

mailauth.pm

                package mailauth;
                use Digest::HMAC_MD5 qw/ hmac_md5_hex /;
                use nginx;
                use DBI;
                use URI::Escape;
                my $dsn="DBI:mysql:database=postfix;host=10.0.0.1";
                our $dbh=DBI->connect_cached($dsn, 'mail-proxy', 'p@ssword', {AutoCommit => 1, mysql_auto_reconnect => 1});
              
                our $auth_ok;
                our $protocol_ports={};
                $protocol_ports->{'pop3'}=110;
                $protocol_ports->{'imap'}=143;
                $protocol_ports->{'smtp'}=25;
              
                sub handler {
                  if (!$dbh->ping()) {
                    ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst)=localtime(time);
                    $dbh=DBI->connect_cached($dsn, 'mail-proxy', 'p@ssword', {AutoCommit => 1, mysql_auto_reconnect => 1});
                    printf STDERR "%4d/%02d/%02d %02d:%02d:%02d [notice] : MySQL server connection lost. Reconnecting.\n", $year+1900,$mon+1,$mday,$hour,$min,$sec;
                  }
                  
                  my $r = shift;
              
                  my $auth_method = $r->header_in("Auth-Method");
                  my $username = uri_unescape($r->header_in("Auth-User"));
                  my $password = uri_unescape($r->header_in("Auth-Pass"));
                  my $salt = $r->header_in("Auth-Salt");
              
                  our $sth=$dbh->prepare("select clear from users where email=? limit 1"); 
                  $sth->execute($username);
                  my $hash=$sth->fetchrow_hashref();
                  my $real_password = $hash->{'clear'};
              
                  # Authorize user
                  if (($auth_method eq "plain" && $password eq $real_password) or
                    ($auth_method eq "cram-md5" && $password eq hmac_md5_hex($salt, $real_password))) {
                    # Auth OK, find mail server
                    our $sth=$dbh->prepare("select destination_mailstore from transport where domain=? limit 1"); 
                    my $domain = $r->header_in("Auth-User");
                    $domain =~ s/^.*@//; # remove @ and everything before 
                    $sth->execute($domain);
                    my $hash=$sth->fetchrow_hashref();
                    my $mailserver = $hash->{'destination_mailstore'};
                    $mailserver =~ s/smtp://;
              
                    $r->header_out("Auth-User", $username);
                    $r->header_out("Auth-Pass", $real_password);
                    $r->header_out("Auth-Status", "OK");
                    $r->header_out("Auth-Server", $mailserver);
                    $r->header_out("Auth-Port", $protocol_ports->{$r->header_in("Auth-Protocol")});
                    
                    # Shared secret to ensure that the request comes from this script
                    $r->header_out("X-NGX-Auth-Key", "some secret");
                  } else {
                    $r->header_out("Auth-Status", "Invalid login or password");
                  }
              
                  $r->send_http_header("text/html");
              
                  return OK;
                }
              
                1;
                __END__
              

The above script supports both plain and CRAM-MD5 authentication. The request headers set by nginx are visible as header_in("...") and the response headers that the script sets are header_out.

It should be pretty self-explanatory what the script does and how. The main feature here is ofcourse setting the Auth-Server response header to the mailserver where you would like to point the user.

UPDATE 2011/02/08:
nginx sends the HTTP headers for the auth script urlencoded. Therefore it is emparative that they be decoded so passwords like my little p%ony works.

I’ve updated the script here to make use of the Perl uri library (you might have to install this).

Also I’ve added a small check to ensure that the MySQL connection is still alive and if not reconnect.