This is a quick’n’dirty tutorial on how to get e-mail working on a basic Linux (Redhat/Fedora/CentOS) installation like the one you may get if you deploy a node in AWS with a predefined AMI (Amazon Machine Image).
The Redhat linux distributions come by default with postfix as MTA (Mail Transport Agent) and Cyrus IMAP as the MDA (Mail Delivery Agent) so this small tutorial is focused on these. Having the packages installed we can go to the configuration files:
/etc/postfix/main.cf: the postfix file, where we configure which e-mails we accept and what to do with them.
/etc/imapd.conf: settings for Cyrus IMAP – where to look for the mailbox passwords, what authentication mechanisms should be supported for pop/imap.
/etc/sasl2/smtpd.conf: for smtp authentication, if an e-mail relay is required (NB: this file can also be located in /usr/lib/sasl2/smtpd.conf).
There is also another file, /etc/cyrus.conf that usually contains the proper defaults upon installation so there may be no need to look into that. It contains settings like the supported protocols (pop3/pop3s/imap/imaps) and the lmtp socket location (the interface between postfix and Cyrus IMAP).
In the postfix configuration file, the following settings are essential:
# cat /etc/postfix.main.cf ... mailbox_transport = lmtp:unix:/var/lib/imap/socket/lmtp virtual_transport = $mailbox_transport virtual_mailbox_domains = /etc/postfix/virtual_domains virtual_mailbox_maps = hash:/etc/postfix/virtual_maps ...
Explanation: the e-mails for the domains specified in virtual_mailbox_maps are to be, before anything else, accepted for further processing. The final mailbox must be determined by looking into virtual_mailbox_maps (if no such mailbox is determined, some error will be returned to the sender). The effective delivery should be done through the socket specified at mailbox_transport.
Some example file contents:
# cat /etc/postfix/virtual_domains brainware.ro # cat /etc/postfix/virtual_maps john.doe@brainware.ro brainware.ro/john.doe
The “virtual_maps” file is to be “hashed” with postmap (have a hashmap generated out of it):
# postmap hash:/etc/postfix/virtual_maps
On the Cyrus IMAP side, we must first check that the daemon listens on the proper lmtp socket (by default it should):
# cat /etc/cyrus.conf | grep lmtp lmtpunix cmd="lmtpd" listen="/var/lib/imap/socket/lmtp" prefork=1
NB: at this point one may want to disable SELinux in order to allow for the socket communication between postfix and Cyrus IMAP.
Continue Reading →
One of the less known features of Gmail is the ability to receive e-mails sent to particular aliases of the main e-mail address, e.g. mails sent to john.doe+22@gmail.com will get to the main mailbox, john.doe@gmail.com. (NB: the address is made up, hopefully this is not someone’s real address).
How would you replicate such feature on a local node you manage? I won’t cover all the details, at most I will just scratch the surface a bit. When using Postfix (the default MTA installed with Redhat/Fedora/CentOS), one must first look to the place where such functionality can be put in, the virtual_alias_maps option in the configuration file (main.cf).
The mapping concept in Postfix is common throughout its options. A “map” must be able to provide a translation between an input (e.g. an e-mail address) and some desired output (e.g. the mailbox location on the disk, the real destination e-mail). The “virtual” part is understood by Postfix as anything that is not tied to a real Unix account. So, in order to be able to provide the Gmail-style aliasing, one must essentialy create a (virtual) conversion map that would relate an input (addresses with “+“) to the desired output (the main mailbox).
Looking back to the wildcard e-mail addresses that we may get e-mails sent to, the only solution to get them matched to the real destination is by using some sort of regular expression. Simple map types accepted by Postfix (e.g. hash tables) perform exact matching, e.g. we may be able to alias john.doe+1 to john.doe but not john.doe+2, neither +99 (well, we are able, but we must put in a line for every such alias). Postfix does support regular expression matching through a built-in module, so we may have in the configuration something like:
# cat /etc/postfix/main.cf | grep virtual_alias_maps virtual_alias_maps = regexp:/etc/postfix/virtual_alias
And the “virtual_alias” file may contain something like:
# cat /etc/postfix/virtual_alias /^sample.name+(.*)@brainware.ro$/ sample.name@brainware.ro
This solution just works, but there are some drawbacks to it:
One must manually edit the “virtual_alias_maps” file to add new aliases. This is not practical for large installations with e-mail accounts created and deleted from web interfaces.
The regular expression matching is done within Postfix, theoretically slowing down the mail system throughput. On a busy server with thousands of such aliases this may become a noticeable issue (maybe not much of a serious problem with modern hardware, though).
For both problems, the solution is to offload the address matching to a database installation (e.g. MySQL, PostgreSQL) and do the regex matching from a stored procedure. I may come back to this on a later date, though. But for now, thank you for your time!
Everybody’s favorite way of sending spam: the mail() function in PHP. But what about the bounced e-mails?
Yeah, about them. Where do they get sent back? Well, not so fast. Not always to the “From” address of the original e-mail, that’s for sure. The e-mail servers return e-mails to the “envelope sender” which may or may not be the same as “From”. There used to be a special header, “Errors-To”, but it got deprecated and nowadays may simply be ignored.
Anyway, let’s go back to the “envelope sender”; this is set by the smtp client when connecting to the server and one would expect that the mail() from PHP be smart enough to deal with it. Let’s see how this function really works. First, let’s take a look in php.ini:
[mail function] ; For Win32 only. SMTP = localhost smtp_port = 25 ; For Win32 only. sendmail_from = me@example.com ; For Unix only. You may supply arguments as well (default: "sendmail -t -i"). sendmail_path = /usr/sbin/sendmail -t -i
On Windows it looks like mail() really handles everything by itself. On Linux it looks like the work is delegated to an external component – sendmail. Let’s take a look in the sendmail man page (well, actually the one from Postfix):
-i When reading a message from standard input, don´t treat a line with only a . character as the end of input.
-t Extract recipients from message headers. These are added to any recipients specified on the command line.
It looks like mail() fixes up the e-mail contents and then pipes it to sendmail; but what about the “envelope sender”? The sendmail man page is even more revealing:
-f sender Set the envelope sender address. This is the address where delivery problems are sent to. With Postfix versions before 2.1, the Errors-To: message header overrides the error return address.
No -f parameter is provided in this case; this means that bounces get sent to one of the following:
- for older e-mail systems, to the Errors-To: header e-mail address (if any provided in the 4th parameter of the mail() function);
- to the Return-Path: header e-mail address (if any provided in the 4th parameter of the mail() function);
- to the e-mail address that was automatically determined for the local user that executed the sendmail command (e.g. apache, nobody). Postfix stores it in the X-Postfix-Sender: header (added automatically).
If we want to avoid the 3rd scenario we must do some programming work. First, let’s notice that in php.ini one could change the e-mail sending program with a script of our own that can do some magic and then call sendmail:
sendmail_path = /usr/local/sbin/sendmail_wrapper
The script should obviously have the right permissions:
rwxr-xr-x 1 root root 814 Oct 4 15:45 sendmail_wrapper
One may write such script in Python or Ruby; PHP works wonderfully, though, as long as you do not call mail() in the script:
#!/usr/bin/php -q <?php set_time_limit(0); $mtext = ""; $from = ""; $has_return_path = false; // Feel free to replace regex with a better e-mail match $regex="/[a-z0-9]+([_\\.-][a-z0-9]+)*@([a-z0-9]+([\.-][a-z0-9]+)*)+\\.[a-z]{2,}/i"; $stdin = fopen('php://stdin', 'r'); while(!feof($stdin)) { $cline=fgets($stdin); if (!((strpos($cline, "Return-Path:")===false))) if (preg_match_all($regex, $cline, $matches, PREG_PATTERN_ORDER)) $has_return_path = true; if (!((strpos($cline, "From:")===false))) if (preg_match_all($regex, $cline, $matches, PREG_PATTERN_ORDER)) $from = $matches[0][0]; $mtext.=$cline; } fclose($stdin); if (!empty($from) && !$has_return_path) $from_opt = ' -f'.$from; $fd=popen('/usr/sbin/sendmail -t -i'.$from_opt, 'w'); fwrite($fd, $mtext); pclose($fd); ?>
Well, that’s about it. I hope you enjoyed it.