robots.org.uk: MailSystem

This is a summary of how to configure an email system on a computer running Debian GNU/Linux 5.0 ("lenny").

Design

The ingredients used are:

ClamAV

scans messages for viruses, worms and other malware

Dovecot

IMAP server

Exim

mail transport agent

Mailman

mailing list manager

PostgreSQL

database

SpamAssassin

scans messages for spam

Mail will be delivered to Maildir directories located at /srv/mail/$domain/$user. These directories will be created by Exim as it receives messages. All mail will be owned by the system mail user. To set that up:

# mkdir -m 0775 /srv/mail
# chgrp mail /srv/mail

TODO

This document is not quite finished yet... things to change:

• strip spamassassin's headers from incoming mail

• stop real_local_router from accepting mail for virtual domains (move virtual stuff, etc, to before real_local router)

• move aliases before accounts
• explain how to clean up after vacation

• don't autoreply to bulk mail, etc: http://wiki.exim.org/EximAutoReply

Database setup

PostgreSQL configuration

Install the postgresql and postgresql-contrib packages.

Grant permission for the users root, Debian-exim and mail to connect to the mail database using the maildaemon database role. In /etc/postgresql/8.3/main/pg_hba.conf, add:

local mail maildaemon ident mailmap

In /etc/postgresql/8.3/main/pg_ident.conf, add:

mailmap root maildaemon
mailmap Debian-exim maildaemon
mailmap mail maildaemon

Run /etc/init.d/postgresql-8.3 reload to bring these changes into effect.

Mail database creation

Become the postgres user (using su, sudo, etc.), and then:

$ createdb mail
$ createuser maildaemon
$ createuser mailadmin
$ createlang plpgsql mail

For the security of the system's users, we do not store account passwords in plaintext in the database. Instead, we store hashes of the passwords. In order to calculate the hashes, we use the pgcrypto contrib module.

We add the functions from pgcrypto in their own schema. This will ease updating the database to work with future versions of PostgreSQL.

$ psql mail
mail=# CREATE SCHEMA pgcrypto;
mail=# GRANT USAGE ON SCHEMA pgcrypto TO PUBLIC;

Copy /usr/share/postgresql/8.3/contrib/pgcrypto.sql to /tmp/ and edit it. Change the fourth line to read: SET search_path = pgcrypto;. Then tell psql to read the file and execute the statements within it with a command such as: \i /tmp/pgcrypto.sql.

Finally, we populate a schema called mail with our tables, views, functions, etc. Download the SQL commands and feed the file through psql as you did with pgcrypto.sql.

Mail account management

Use the create_account and change_password function to mange users:

$ psql mail
mail=# SET search_path = mail;
mail=# SELECT create_account ('user', 'domain', 'password');
mail=# SELECT change_password ('user', 'domain', 'new_password');

Other management tasks can be performed by updating/deleting from the accounts table directly.

Initial Exim configuration

Install exim4-daemon-heavy. Run dpkg-reconfigure exim4-config. Choose these options:

• split configuration into small files
• "internet site" mail configuration

• system mail name: the FQDN of the default IP address assigned to the network interface from which outgoing connections will be made (usually the same as your system's hostname followed by a dot and the domain name)

• ip-addresses to listen on: leave blank unless you know better
• other destinations for which mail is accepted: `+virtual_domains'

Exim will then be reloaded, and will probably fail since we have not yet defined the virtual_domains domainlist. This is normal at this stage.

After creating the exim config files as detailed below, run /etc/init.d/exim4 reload to get Exim to re-read its configuration, and it should begin processing mail.

If you use the vim editor, put # vim: filetype=exim at the end of exim's config files to have vim automatically enable syntax highlighting when you open them.

Create /etc/exim4/conf.d/main/db. If your PostgreSQL server runs on a different port, change 5432 to match.

hide pgsql_servers = (/var/run/postgresql/.s.PGSQL.5432)/mail/maildaemon/

If you are using password authentication to connect to the database, make sure this file is not world-redable! Also make sure that the generated /var/lib/exim4/config.autogenerated file is protected by changing CFILEMODE in /etc/exim4/update-exim4.conf to 600 and reloading Exim.

Incoming mail—"virtual" domains

The setup revolves around creating a virtual_domains domain list. The list is populated by a database lookup.

When we said +virtual_domains when asked for a list of destinations for which mail is accepted, we told Exim to check the virtual_domains domainlist when deciding whether a message is destined for local delivery or not. We will now actually specify this domainlist.

Create /etc/exim4/conf.d/main/virtual:

localpartlist virtual_localparts_ignore = postmaster : abuse

domainlist virtual_domains = ${lookup pgsql {SELECT domain FROM mail.exim_domains WHERE domain = '${quote_pgsql:$domain}'}}

Create /etc/exim4/conf.d/router/350_virtual_alias:

virtual_alias:
        debug_print = "R: virtual_alias for $local_part@$domain"

        local_parts = ! +virtual_localparts_ignore
        domains = +virtual_domains

        driver = redirect
        data = ${lookup pgsql {SELECT mail.lookup_alias ('${quote_pgsql:$local_part}', '${quote_pgsql:$domain}');}}

        user = mail

        allow_filter
        #forbid_filter_dlfunc
        forbid_file
        forbid_filter_existstest
        forbid_filter_logwrite
        forbid_filter_lookup
        forbid_filter_perl
        forbid_filter_readfile
        forbid_filter_readsocket
        forbid_filter_reply
        forbid_filter_run

        forbid_file
        forbid_include
        forbid_pipe

Create /etc/exim4/conf.d/router/351_virtual_account:

virtual_account:
        debug_print = "R: virtual_account for $local_part@$domain"

        local_parts = ! +virtual_localparts_ignore
        domains = +virtual_domains

        driver = redirect
        data = ${lookup pgsql {SELECT maildir FROM mail.exim_accounts \
               WHERE "user" = '${quote_pgsql:$local_part}' \
                     AND domain = '${quote_pgsql:$domain}'}}
        more = false

        user = mail
        group = mail

        directory_transport = virtual_delivery

        cannot_route_message = "Unknown user $local_part@$domain"

Create /etc/exim4/conf.d/transport/virtual_delivery:

    virtual_delivery:
        debug_print = "T: virtual_delivery for $local_part@$domain"
        driver = appendfile
        envelope_to_add = true
        return_path_add = true
        check_string = ""
        escape_string = ""
        maildir_format

        quota = ${lookup pgsql {SELECT quota * 1024 * 1024 \
                FROM mail.accounts \
                WHERE "user" = '${quote_pgsql:$local_part}' AND domain = '${quote_pgsql:$domain}'}}
        quota_warn_threshold = 90%
        maildir_use_size_file = true
        quota_warn_message = "\
    To: $local_part@$domain\n\
    From: postmaster@$primary_hostname\n\
    Subject: You have reached 90% of your mail quota\n\
    \n\
    This message is automatically created by mail delivery software.\n\
    \n\
    Your mailbox has reached 90% of its capacity. If you do not free\n\
    up some space by deleting old mail, you will not receive any new\n\
    mail that may be sent to you.\n\
    \n\
    Regards,\n\
    postmaster@$primary_hostname"

Edit /etc/exim4/conf.d/retry/30_exim4-config. Before the final line, add:

# Bounce over-quota warnings immediatly
*                      quota

Incoming mail—IMAP

Install dovecot-imapd.

dovecot.conf

Edit /etc/dovecot/dovecot.conf and set the following options:

protocols = imaps
disable_plaintext_auth = yes
shutdown_clients = yes
ssl_cert_file = /etc/dovecot/cert.pem
ssl_key_file = /etc/dovecot/key.pem
mail_location = maildir:%h
dotlock_use_excl = yes
first_valid_uid = 8
first_valid_gid = 8
maildir_copy_with_hardlinks = yes
mail plugins = quota imap_quota
login_greeting_capability = yes

Some of them should not be set if your /srv/mail directory is on NFS. Read the comments above each option for more information.

Find the auth default section of the control file. The order of options in this section is important, as Dovecot will try each one in turn when authenticating users. After the initial mechanisms = plain option, add:

passdb sql {
  args = /etc/dovecot/dovecot-sql.conf
}

Scroll down and comment out the passdb pam section. This will prevent system users from picking up their mail via IMAP. This is done to prevent users exposing their system login credentials due to misconfigurations, accidents, etc. You can still use /etc/aliases and/or .forward files in the home directories of system users to redirect their mail to a virtual mailbox where they can pick it up.

• If you do want your system users to be able to use IMAP, you should not set mail_location as above; instead, set mail_executable to /usr/local/lib/dovecot/imap; then create that file, with the following contents. Make sure the file is executable.

  #!/bin/bash
  
  # For more info, see "Custom mailbox location detection" at
  # <http://wiki.dovecot.org/MailLocation>
  
  if [[ "$HOME" =~ ^/srv/mail/ ]]
  then
          # 'virtual' mail accounts have their mail stored in their actual home
          # directory.
          MAIL=maildir:$HOME/
  else
          # Real user accounts have their mail stored in a subdirectory of their
          # home directory.
          MAIL=maildir:$HOME/Maildir/
  fi
  export MAIL
  
  exec /usr/lib/dovecot/imap

  You must also re-run dpkg-reconfigure and set 'delivery method for local mail' to 'Maildir format in home directory'.

Now, scroll down and uncomment the userdb prefetch section.

Finally, look for the plugin section, and within it, set quota = maildir.

dovecot-sql.conf

Set the following options in /etc/dovecot/dovecot-imap.conf:

driver = pgsql
connect = dbname=mail user=maildaemon
default_pass_scheme = CRYPT
password_query = SELECT user, password, userdb_home, userdb_uid, userdb_gid FROM mail.dovecot_auth WHERE "user" = '%u'

Finally, reload Dovecot with /etc/init.d/dovecot reload.

TLS (aka SSL) encryption

Put your certificate at /etc/exim4/exim.crt and your private key at /etc/exim4/exim.key. Ensure that the private key can only be read by Exim by changing its group owner to Debian-exim and its permissions to 0640.

Edit /etc/exim4/conf.d/main/00_localmacros:

MAIN_TLS_ENABLE = yes
MAIN_LOG_SELECTOR = +tls_certificate_verified +tls_cipher +tls_peerdn

Outgoing mail—SMTP authentication

Create /etc/exim4/conf.d/auth/postgres:

# $2 is the supplied username; $3 is the supplied password
virtual_auth_server_plain:
        driver = plaintext
        public_name = PLAIN
        server_advertise_condition = ${if eq{$tls_cipher}{}{no}{yes}}
        server_condition = ${if crypteq{$3} \
                {${lookup pgsql {SELECT password FROM mail.exim_auth WHERE email = '${quote_pgsql:$2}'}}} \
                {yes} \
                {no}}
        server_set_id = $2

Clients can then send mail via your server, using their email addresses as their username.

Mailing lists

Mailman configuration

Install the mailman package. Read /usr/share/doc/mailman/README.Debian.gz. Disregard the paragraphs about Exim; the following files configure Exim to automatically recognise mailing lists.

Stop the newlist command from printing irrelevant info about how to configure your MTA to recognise a newly-created list. Edit /etc/mailman/mm_cfg.py:

MTA=None

Exim configuration

Create /etc/exim4/conf.d/main/mailman:

MAILMAN_HOME = /var/lib/mailman
MAILMAN_WRAP = MAILMAN_HOME/mail/mailman

MAILMAN_UID = list
MAILMAN_GID = list

MAILMAN_LOCALPART_SUFFIXES = -admin: -bounces: -confirm : -join : -leave \
        : -owner : -request : -subscribe : -unsubscribe

The mailman router is run after the various virtual routers are run. This ensures that a mailing list named foo does not clobber delivery to foo@virtual.example.com. It also means that mailing list addresses live 'in' the mail server's primary hostname. For example, the list mylist's address is mylist@hostname.example.com. If you want to create a list for a virtual domain, create a virtual alias that redirects to the list's real domain name.

Create /etc/exim4/conf.d/router/360_mailman:

mailman:
        debug_print = "R: mailman for $local_part@$domain"
        driver = accept
        require_files = MAILMAN_HOME/lists/$local_part/config.pck
        local_part_suffix_optional
        local_part_suffix = MAILMAN_LOCALPART_SUFFIXES
        transport = mailman

Create /etc/exim4/conf.d/transport/mailman:

mailman:
        debug_print = "T: mailman for $local_part@$domain"
        driver = pipe
        command = MAILMAN_WRAP \
                '${if def:local_part_suffix \
                        {${sg {$local_part_suffix} {-(\\w+)(\\+.*)?} {\$1}}} \
                        {post}}' \
                $local_part

        user = MAILMAN_UID
        group = MAILMAN_GID

        home_directory = MAILMAN_HOME
        current_directory = MAILMAN_HOME

Content scanning—antispam/malware

Install clamav-daemon.

Allow ClamAV permission to scan files in Exim's mail spool:

# adduser clamav Debian-exim

Install spamassassin.

By default, spamd is configured to run as root so that it can run on behalf of any user. We don't want this; we will run it as its own user.

# adduser --system --group --home /var/run spamassassin

Edit /etc/default/spamassassin:

ENABLED=1
OPTIONS="-u spamassassin --nouser-config --max-children 3"

Edit /etc/spamassassin/local.cf:

# These are not suitable for system-wide scanning
use_auto_whitelist 0
use_bayes 0

clear_report_template
report "hits=_HITS_: _TESTSSCORES(, )_ with SpamAssassin _VERSION_ (_SUBVERSION_) on _HOSTNAME_"

Start SpamAssassin and ClamAV:

# /etc/init.d/spamassassin start
# /etc/init.d/clamav-daemon restart 

Exim configuration

Edit /etc/exim4/conf.d/main/00_localmacros:

CHECK_RCPT_LOCAL_ACL_FILE = /etc/exim4/acl-rcpt
CHECK_DATA_LOCAL_ACL_FILE = /etc/exim4/acl-data

# enables sender address verification for messages from remote domains
# see acl_check_rcpt and section 39.31 of the Exim specification
CHECK_RCPT_VERIFY_SENDER = yes

Anything that recieves more than 10 points from SpamAssassin will be rejected at SMTP-time. Users can filter off mail with lower scores by matching the X-Spam-Bars header. Create /etc/exim4/conf.d/main/antispam:

# Desired values are multiplied by ten
SPAM_FLAG_SCORE = 50
SPAM_REJECT_SCORE = 100

av_scanner = clamd:/var/run/clamav/clamd.ctl

Create /etc/exim4/acl-rcpt:

# Greylist hosts known to send spam and exploits
#
defer
        !acl = acl_local_deny_exceptions

        dnslists = dnsbl.sorbs.net : sbl-xbl.spamhaus.org : bl.spamcop.net

        condition = ${if eq {${lookup pgsql{SELECT COUNT(*) FROM mail.greylist_disable \
                    WHERE address = '${quote_pgsql:$local_part}@${quote_pgsql:$domain}'}}} {0}}
        condition = ${if eq {${lookup pgsql{SELECT mail.greylist_process \
                ('${quote_pgsql:$sender_host_address}', \
                '${quote_pgsql:$sender_address}', \
                '${quote_pgsql:$local_part@$domain}')}}} {t}}

        message = Please try later
        log_message = greylisted

Create /etc/exim4/acl-data:

# malware
deny message = Message contains malware ($malware_name)
        malware = *

# spam
deny message = Message classified as spam ($spam_score points)
        spam = mail:true
        condition = ${if >{$spam_score_int}{SPAM_REJECT_SCORE}}

warn
        spam = mail:true
        add_header = X-Spam-Bars: $spam_bar

warn
        spam = mail:true
        add_header = X-Spam-Report: $spam_report

Vacation

Used for out-of-office messages, etc. The vacation auto-reply won't be sent more than once every three days.

Create /etc/exim4/conf.d/router/347_virtual_vacation:

virtual_vacation:
        debug_print = "R: virtual_vacation for $local_part@$domain"

        local_parts = ! +virtual_localparts_ignore
        domains = +virtual_domains

        driver = accept
        condition = ${lookup pgsql {SELECT DISTINCT message FROM mail.vacation WHERE "user" = '${quote_pgsql:$local_part}' AND domain = '${quote_pgsql:$domain}'}}

        transport = virtual_vacation
        unseen

Create /etc/exim4/conf.d/transport/virtual_vacation:

virtual_vacation:
        debug_print = "T: virtual_vacation for $local_part@$domain"

        driver = autoreply
        user = mail
        group = mail

        return_message
        once = /srv/mail/$domain/$local_part/vacation.dbm
        once_repeat = 3d

        from = $local_part@$domain
        to = $sender_address
        subject = Re: $h_subject
        text = ${lookup pgsql{SELECT message FROM mail.vacation \
                WHERE "user" = '${quote_pgsql:$local_part}' \
                AND domain = '${quote_pgsql:$domain}'}}

Subaddressing

It is useful to redirect mail for sam+foo@example.com to sam@example.com. This makes it easy for users to create arbitrary mailboxes on the fly. The original destination address is preserved in the message's To header, so users can use it for filtering, or to see which sites sold their email addresses to spammers, etc.

Create /etc/exim4/conf.d/router/305_subaddress:

subaddress:
        debug_print = "R: subaddress for $local_part$local_part_suffix@$domain"

        driver = redirect
        data = $local_part@$domain

        local_part_suffix = +*

Mail for system users

We want mail for system users to be directed to root's mailbox. We do this so that such mail does not languish unread in /var/mail for years.

The Debian Policy Manual defines a system user as one whose UID is outside of the range 1000‒29999.

Create /etc/exim4/conf.d/router/405_system_user:

# Redirect messages for users with a uid outside of the range 1000 - 29999 to
# root. The range is defined in Policy section 9.2.2:
# <http://www.debian.org/doc/debian-policy/ch-opersys.html#s9.2.2>
system_user:
        debug_print = "R: system_user for $local_part@$domain [$local_user_uid]"
        driver = redirect

        domains = +local_domains
        check_local_user
        data = root
        condition = ${if or {{< {$local_user_uid} {1000}} {> {$local_user_uid} {29999}}}}

Mail for human users

As noted earlier, mail for real, human users should be redirected by means of a .forward file or similar. If a user does not have a forward file, we want to prevent their mail from clogging up /var/mail. Instead, we will bounce any mail for a human user who has not specified a forwarding address. However, this router should not run when performing address verification--the user exists, they just don't want to actually receive anything.

Create /etc/exim4/conf.d/router/875_no-local-mail:

no_local_mail:
    debug_print = "R: no_local_mail for $local_part@$domain"

    domains = +local_domains
    check_local_user

    driver = redirect
    no_verify

    allow_fail
    data = :fail: no forwarding address specified for this user

# vim: ft=exim

Mail for groups

I configured my server so that sending mail to groupname.group would cause the mail to be delivered to all the members of the local group groupname, as defined in the /etc/group file. This is pretty esoteric, but may be useful to someone else:

Create /etc/exim4/conf.d/router/910_local_group:

local_group:
  debug_print = "R: local_group for $local_part@$domain"

  driver = redirect
  domains = +local_domains
  data = ${extract{3}{:}{${lookup{$local_part}lsearch{/etc/group}}}}
  local_part_suffix = .group

The local_part_suffix serves to namespace the groups off from other local_parts.

Testing

Fake SMTP session from specified IP address: exim4 -bh 1.2.3.4

Address testing ("what will Exim do with this address?"): exim4 -bt

Configuration options testing ("what is the effective value of an option?: exim4 -bP

Sources

• PostgreSQL 8.3 documentation

• Exim Specification

---

CategoryTechnote