This covers Postfix, Courier Maildrop and Courier IMAP . This is an update of my 2001 notes which were relevant to Redhat 7.1 and the versions of Postfix, Courier Maildrop and Courier IMAP which were current then. You can find that page from the parent directory ../ but it was getting out of date, so here is an updated version.
In October 2002 I added material to that RH7.1 page on another page regarding how to integrate Spam Assassin and the Anomy Sanitizer for automatically filtering spam and viruses (malware), and on how to report on the message numbers and storage requirements of each mailbox. Below I try to keep this RH 9.0 material in sync with that older documentation. Those programs are integrated via Maildrop's filtering function - the .mailfilter file.
This is a verbose documentation - because I find that a lot of problems are caused by overly terse doco which already assumes you know things that you don't. Also, I think more people's problems will be solved if the description is full and search engines find this page better.
Please let me know of any corrections or improvements to this page.
It is not a proper How-To - it is How I Did It, simplified.
By the time you are reading it, the latest versions of software will probably differ somewhat from what I am describing, but to my knowledge, the same principles hold. If you find significant differences I should mention, please let me know!
Alternatives to Postfix, Maildrop and Courier IMAP
The most common mail server is Sendmail http://www.sendmail.org - but it is old and hard to configure. Other mail servers of note are Qmail http://www.qmail.org and Exim: http://www.exim.org .
There is a new IMAP server: Dovecot from Timo Sirainen in Finland: http://dovecot.procontrol.fi . I found this easy to install compared to Courier IMAP - but with the doco below, you should find Courier IMAP installation pretty straightforward. Dovecot can have mailboxes alongside the Inbox, rather than having to be inside the Inbox as with Courier IMAP, but I don't think this matters much. Dovecot seems to be at fast - but I haven't done direct comparisons. (The version I tested had a problem which made it slow and chew memory when writing messages to a mailbox which contained thousands of messages. I reported this to Timo Sirainen via the mailing list and 6 hours later he released a new test version with the problem fixed.). Dovecot can work with Mbox mailboxes too - but why would anyone want to? Courier IMAP has a longer development history and is a widely used program. My only gripe is how hard I found it to understand the installation process. The other alternatives are the University of Washington IMAP server which only works with Mbox mailboxes, and so is slow and causes trouble, and the big-iron Cyrus IMAP http://asg.web.cmu.edu/cyrus/ which is probably the best choice for the most intense IMAP servers.
The main, more established, alternative to Maildrop for local delivery and filtering is "procmail" http://www.procmail.org . Like "sendmail" its old and hoary . . . I like Maildrop and its filtering language! Maildrop is a fresh project, not something old and overgrown - and being the work of one person may be an advantage too.
Back to the web-mail page.
Back to the First Principles page for the world's
longest Sliiiiiiiiiinky, corsetry ads, and all sorts of things
. . .
This page describes how I installed three sets of software on a Red Hat 9.0 system in May 2003. This was on a different machine - not my main mailserver which I documented in 2001, but I have adapted what lies below to be as if it was for my main mailserver. My main mailserver is running RH7.2 now and I will probably install the new versions of Postfix, Maildrop and Courier IMAP on that machine now I have them running on another machine with RH 9.0.
This is not a How-To or a complete tutorial on setting up mail servers etc. for those who have never done it before, but it is a more complete documentation than is typical of IT stuff. Part of the reason for that is that I will forget all this quite soon (Unix systems admin is not all I do) and so will want the various related things well explained for the next time I come this way in the years to come.The three sets of software are:
Please see the parent directory for links to pages which cover how I:
Postfix installation is well documented elsewhere and is relatively straightforward - but I discuss it below. Installation of Courier IMAP and the required Maildrop program to allow Postfix to write to Courier's Maildir format mailboxes is not at all trivial - but with the doco below, it should be pretty straightforward. Programs such as these three need to be highly flexible to run on various operating systems in various ways, so the documentation will usually involve lots of options which are of no direct interest in any one situation. Fighting your way through these, and the fact that the documentation may be somewhat out of date and inconsistent with the software, and the fact that the documentation may be badly organised and hard to navigate, whilst trying to install something and also deciding which of the mounting questions in your mind are really caused by your own misunderstanding, actual problems with the software, actual problems with the situation you are trying to install it in, changes to the situation which were not present when the software was developed, misunderstandings and/or omissions by the person who wrote the software and documentation . . . . its hard work!
(In 2001 I also investigated using a recent version of the widely respected text-based Unix email program mutt . It can access mail accounts via IMAP, and I did eventually configure it to work, but I had great difficulty selecting mailboxes and with a few other things. I didn't desperately need it then or now. Information on mutt with IMAP can be found at: http://mutt.sourceforge.net/imap/ . Also, I found that it couldn't obviously read messages which are tagged for deletion - which is something I often want to do. mutt was designed initially for accessing local mailspool and mailbox files, so IMAP support is not a central part of its functionality. Maybe it has changed a lot since I looked at it in 2001)
My needs here are modest - just a few users accessing the server, which is connected to the Net via a 56kbps modem. But I do make heavy use of IMAP email (using Netscape 7.02 and Mozilla Communicator at present) over the LAN, and sometimes when I am away via the 56kbps modem. (Actually, with large mailboxes and large numbers of mailboxes, running a remote IMAP client like Netscape's Messenger is very slow and a pain, especially via a flaky GSM cell-phone at 9.6kbps if the Gods are smiling, and then it seems like half-duplex . . . A better idea for me is to use a web browser and the web-mail program which is running on this machine. That way, there is no need for all the details of all the mailboxes to travel over the GSM link.)
I do not use web-mail myself, except when I am away from my home/office. The main users of the web-mail system are several friends who could want to check their email from many locations and need to do so via a web browser. I may consider using a text-mode IMAP compatible mail program such as mutt, running on the server (via SSH) if I am away - if I can find one which works nicely and which shows the messages which are tagged for deletion.
There are no fancy performance requirements on this system - other than my desire that the IMAP server provide emails in multi-megabyte mailboxes very quickly - without having to read the whole file. The University of Washington IMAPD does it simply by reading the entire mailbox file. Courier does not keep all the contents of a mailbox in a single linear "mbox" (AKA "mailbox" format) file (as UW IMAP does). It uses Qmail's "Maildir" arrangement - several directories per mailbox, with each email a separate file. This is faster, eliminates the need for lock files (for multiple programs accessing a mailbox at the same time) and enables the mailboxes to be accessed on other machines via NFS, which is not usually possible with single-file mailboxes due to NFS's inability to support proper locking arrangements.
I want the resulting system to provide:
All these will be on the same machine, but in practice the mailboxes
could be on multiple computers accessible via NFS. Postfix and
Maildrop must be on the one machine, IMAPD (the IMAP Daemon) could be
on a separate machine and so could Postman. This would involve
some fancier authentication systems than the standard PAM -> Shadow
Password system I am using. The web browser could, of course, be
anywhere in the world. Other email clients (and web-mail programs
- for instance I also have SqWebMail running) can access the IMAPD and
Postfix.
Maildir format Web-mail email client
mailboxes
/-----------\ /----------\ *********** /---------\ /----------\
| | | | * Inbox * | | | |
| | ====> | Maildrop | ===> * * <======> | Courier | | Postman |
<=== | Postfix | | | =\ *********** | IMAPD | | with | /---------\
===> | MTA | \----------/ | | | IMAP | Apache | | |
SMTP | | | |====> ************ | | <---> | | <-----> | User's |
to & | (Message | Filtering | * YYY * | | | | HTTP | web |
from | Transfer | rules in \=>************ * <==> | | | | or | browser |
other | Agent) | .mailfilter * XXX * *** | | IMAP | SMTP | HTTPS | |
MTAs | | <--\ *** * * <======> | | <-\ | Out | | |
\-----------/ \ ************ \---------/ \ \----------/ | |
\ \ V \---------/
\---<----------------------------------------<---------------/
SMTP outgoing mail \ \ /------------------------\
\ \-> | Netscape 7 etc. |
\ | IMAP capable Email |
\ | client |
\ | |
\-----------<--------- |< SMTP outgoing mail |
| |
\------------------------/
*** Spam and virus filtering
This is where extra programs can go for filtering viruses ("malware") and spam. Another approach is to have Postfix detect and refuse to accept spam messages from remote MTAs (on the left-side of the Postfix box in this diagram). This would cut down on the volume of mail which needs to be delivered locally. However this approach is often unacceptable because some false positives are inevitable in spam detection and the problems resulting from refusing to accept non-spam email, without the intended recipient ever knowing that it has been sent or rejected, cannot be tolerated.. This *** location is actually two points in my .mailfilter file, for each user, which is executed by Courier Maildrop. One point is where I call Anomy Sanitizer and the other is where I call SpamAssassin.
Postfix is a collection of programs, and one way of dealing with spam is to configure Postfix to reject SMTP sessions based on the IP address of the machine starting the session (such as by a real-time black-hole lookup of the IP address with a remote service for this) and also on criteria such as sender address and other things in the header. This, however, rejects the message entirely, so the recipient never knows what has been rejected - meaning that if legitimate messages are rejected, they don't know about it. It has the advantage of reducing the communications traffic by rejecting spam, rather than accepting and filtering it, but it involves Postfix in more work and delays with looking up remote servers to check the IP address.
My approach to running Spam Assassin and Anomy Sanitizer is to use the xfilter command from within Maildrop's .mailfilter file. I have a page on how I did this, which you can find from the parent directory.
- Spam Assassin http://spamassassin.org This is a widely respected spam detection system, which works in various ways, including by reference to remote blacklists etc. and Bayesian recognition of spam based on the contents of the user's personal spam pit.
- The Anomy Sanitizer http://mailtools.anomy.net Not actually a virus scanner, in terms of identifying specific viruses (which would require constant updates to its filtering rules) but a program to eliminate problems in attachments such as Javascript in HTML emails, and to rename or delete executable files and to do other things which generally render them less harmful. (This is my rough summary - see the web page - for instance it can call a virus scanner.) I have it "defang" various suspect HTML elements, and "drop" attached files which are executable.
Another way to run Spam Assassin and Anomy Sanitizer is described by Advosys in Ottawa:http://advosys.ca/papers/postfix-filtering.html .Whether Advosys' or my approach is used, the aim is to label each suspected spam or virus message by adding distinctive headers and/or distinctive and human-readable material in the body of the message. Anomy Sanitizer typically also changes suspect message bodies to "defang" suspect HTML tags, Javascript, executable files etc. In the standard configuration this has the effect of greatly shortening long virus messages. Then, it is easy in Maildrop's filtering rules to decide what to do with suspected spam or malware messages. I use Maildrop with my "Delivered to Inbox Tagged for Deletion" system, with Cc:ing to separate mailboxes for suspected spam and malware, with the addition of a "[~~~SPAM]" or "[~~~VIRUS]" subject header for when they are delivered tagged for deletion in the Inbox. This way, all messages are visible to the intended recipient - me - but I don't have go manually select and do something with the individual spams and viruses which are increasingly flooding my Inbox.
The Postfix configuration changes required for running the Advosys script for these two programs are entirely separate from how Postfix is configured to deliver messages with Maildrop. However, this leads to them running on outgoing mail as well, not just mail to be delivered to local users' mailboxes. (Actually, the script becomes part of Postfix's smtp program which also handles the messages arriving from local clients via the line entering the bottom right of the Postfix box above.) Advosys solve this by running two instances of Postfix, with the first one doing the virus/spam filtering as usual and the second one responding to a second IP address purely for SMTP outgoing messages from local mail clients - therefore requiring all the clients be reconfigured. For my own purposes, I think a better approach is to put Anomy Sanitizer and Spam Assassin in the local delivery section of Postfix, just before Maildrop. See my separate page, from the parent directory for how I did this - running them from within Maildrop. However, Advosys.ca have a lot of experience with large mail systems, for companies and educational institutions, and there are good reasons why they run two instances of Postfix, and do other things to improve efficiency. For instance, their clients may want to filter for spam and viruses on all incoming email, but only filter for viruses on outgoing mail. This is not because they are spammers, but because spam detection is an imperfect science.
I think it is best to store these highly suspected spam and virus emails somewhere, rather than to delete them manually or automatically as they are first viewed or arrive at the server. To delete this stuff manually or automatically risks accidentally deleting a valid email - because I can't be super cautious and 100% correct in my judgements whenever I do this, which could be any time of the day. With largely automated detection and copying to spam and virus mailboxes, supplemented by manual steps for the same purpose with what gets past the filters, at least I can periodically look into those mailboxes to ensure I haven't accidentally turfed something good into these cesspits.
Earlier in 2003, when I did, I found three good messages falsely labelled as spam - from friends! Also, one purchase order authentication email got classified as spam. In May I decided to methodically check the spam pit every month or so to see if a good message has been turfed in there automatically. This takes the stress out of my manual filtering actions, which makes it faster - manually dragging and dropping those messages which were not automatically identified as spam or virus emails, but which looked like it to me just based on their Subject and Sender. This manual selection and turfing into a mailbox is becoming a necessary step many times when I check my email, and so it is a daily and hourly curse, which should be made as easy and as possible.
Mail serving, IMAP and any web-mail system are all potentially
intensive applications. They are commonly used for large systems,
such as for schools and universities. Some special tweaking is
likely to be needed for these systems, such as using separate machines
to run the various servers, or running SpamAssassin as a daemon, rather
than firing up an instance of it for every incoming message. I
have no such
intense requirements. The mailing lists for Postfix, Courier and
IMP are the best place to find out about high-performance and
high-availability systems. There is a general Courier users mailing
list - mainly for the mail server, which I am not using her, and
specific lists for Courier Maildrop and Courier IMAP.
Pros and Cons of Maildirs vs Mbox mailboxes
There are some drawbacks, but mainly benefits, with this Maildir mailbox scheme:
- There's no single file to search if I want to search a mailbox by ordinary means. (Of course the IMAP server can search mailboxes.) To create a single linear file, I could copy the mailbox contents to a local mailbox file on the computer running the email client (Netscape/Mozilla's local mailboxes are linear Mbox files), or use a client which supports multiple IMAP servers at once (Netscape 7 / Mozilla and no doubt others) to copy entire mailboxes from a Courier IMAP server to one running UW IMAP. (Generally, for me, the easiest way is to copy the messages to Netscape's local mailboxes and search them there - especially with an excellent Windows file manager called Z-Tree Win, which highlights the searched for text and quickly steps to the next occurrence. http://www.ztree.com )
- Likewise, migration issues - how to get a bunch of mailboxes in Mbox linear file format into maildir format. I wrote a perl script for this, which you can find via the parent directory..
- Searching a Maildir mailbox is likely to be slower and more CPU intensive than a linear mailbox, since there could be thousands of files to read. However, adding, deleting or modifying emails will be much faster since there is no long file to rewrite. The latter point is much more important than the former, I think.
- The multiple file approach must take up more space on disk than a single linear file, unless you have a fancy file system. I guess the Reiser File System would be good for this: http://www.namesys.com as well as having many other advantages. My mail server is a 800 MHz Celeron with dual 20 Gig hard drives with software RAID-1 for redundancy. Red Hat enables this to be set up via the graphic installer.
- When backing up a set of mailboxes, the result is a mess of files in an archive, rather than a single file - so only an IMAP server could make sense of the results. This is an issue for archaeologists - including myself - when going back through backup CDRs in the future looking for particular things. (There's a program which can create a linear file from a Maildir mailbox directory. . . http://www.qmail.org/qmail-manual-html/man1/maildir2mbox.html More on this below.)
- Unix mail programs, elm at least, don't know about the Maildir format Inbox. Maybe pine would be better? No. Later versions mutt can access an IMAP server, but in 2001 I found it was awkward and it did not show messages tagged for deletion.
- Mbox mailboxes involve a pesky, occasional, mangling of messages. Any message line which begins with "From x" where "x" means one or more characters, will be saved in the mailbox as ">From x". This is because the delimiter in the Mbox between messages is a line starting with "From ". Maildirs have no such problems.
Overall, I want Maildir format mailboxes for performance and so I can run Maildrop and its mail-filtering facilities. A comparison of the standard linear file "Mbox" format and the Maildir format for mailboxes is at: http://www.courier-mta.org/mbox-vs-maildir/ .
In May 2003, after using Courier IMAP for two years, I believe this is a vastly better arrangement than the UW IMAP server. I had all sorts of slowness with UW IMAP, and Netscape 4.77 would often get tangled with its cached notion of the IMAP server being out of sync with the actual state of the IMAP server. To resolve Netscape 4.7 or 7.x or Mozilla getting into this state, look in the ImapMail sub-directory of the profile directory and delete all the files and directories in that ImapMail directory.
For those with really heavy-duty needs, it might be best to dedicate a high-powered machine to Cyrus IMAP http://asg.web.cmu.edu/cyrus/ , which does not rely on Unix accounts - and stores messages in its own internal format.
rpm -qpl packagename.rpm > list.txt Lists all the files in an RPM file irrespective of whether it is installed.. Use the full file name, or, for instance, a shortened version wget* .
rpm -qa > rpmlot.txt Lists the names of all installed packages, I assume in order of their installation.rpm -qa | sort > rpmlot.txt Ditto alphabetically sorted.
rpm -qal > rpmlotfiles.txt Lists all the files of all installed packages. There is no clear indication of what package the files belong to - but it is not too hard to figure it out:
rpm -qf file-name Lists the package a file belongs to.
There is a way of turning off Sendmail and tuning on Postfix. I haven't done this, but someone wrote to me about it. The commandapparently does the trick, for later RedHat systems at least. But what package is this from? The command exists on my RH9.0 system, but there is no man or info page. Running the command on its own gives the following help-text:
- update-alternatives --auto mta
alternatives version 1.3.8 - Copyright (C) 2001 Red Hat, Inc.What does it do? I haven't looked into this further, but Google is your friend:
This may be freely redistributed under the terms of the GNU Public License.
usage: alternatives --install <link> <name> <path> <priority>
[--initscript <service>]
[--slave <link> <name> <path>]*
alternatives --remove <name> <path>
alternatives --auto <name>
alternatives --config <name>
alternatives --display <name>
alternatives --set <name> <path>
common options: --verbose --test --help --usage --version
--altdir <directory> --admindir <directory>
http://www.google.com/search?q=%22alternatives+--auto+mta
I chose not to compile Postfix from the source, but to use an RPM which is ready to go. In this Redhat 9.0 instance, as part of the install process, I selected Postfix to be installed. Sendmail was already installed, and I had to disable it and noted below. (There seems to be no way with the Redhat 9.0 installer of not installing Sendmail, if the mail (I don't recall exactly) section of the installation process was enabled, which was needed to install Postfix as an option.). Probably most of what I write here will be suitable for those who install Postfix by some other means, but I can't be sure. Compiling Postfix from source is probably the best way, rather than relying on someone's packages.
(Later - 2003 June 1 - for my main mailserver, which still runs Red Hat 7.2, I found it easy to upgrade from 1.1.2-1 to the latest Postfix 2.0.0-10 release, which I compiled and installed from a tarball. I saved the config directory /etc/postfix somewhere, used postfix stop and then removed the old RPM with rpm -e --nodeps postfix . Then I followed the instructions in the tarball, and it modified my main.cf file (which I restored from the copy, because I think RPM took it away). The instructions were straighforward (but don't forget to run newaliases!) and soon I was up and running again. The upgrade process used my old main.cf which already had inet_interfaces = $myhostname, localhost . Note that this is not the default state of main.cf in a fresh install, so as noted below you have to make it like this unless you only want to send email to yourself on the one computer. The other thing I had to do was add: "/usr/sbin/postfix start" to /etc/rc.d/init.d/local.rc because I could find no postfix script to go in /etc/rc.d/init.d/ .Considering the power and robustness of this program, this is a really easy installation. The price is right too!)
I don't have any fancy requirements for the very latest version. In several years of using Postfix, it has never to my knowledge put a foot wrong. (But watch out if it ever tries to deliver mail to an Mbox mailbox - such as an ordinary mailspool file at /var/spool/mail which gets to 50,000,000 bytes! This is a limit somewhere in Postfix, compiled in if I remember, and it spits the dummy if this is reached. The fix is to use Maildir mailboxes, or to have some logrotate system to chop Mbox mailboxes which accumulate somewhere into weekly chunks) Salute Wietse Venema, who carries on this open-source work at IBM!The Postfix Web abode is:
http://www.postfix.orgThere is a very busy mailing list postfix-users (with several web archives) and an occasional postfix-announce list too - archives are listed at the above site.
If you have already installed RH 9.0 without Postfix, then you can install it from CD-ROM by grabbing the package from the /RedHat/RPMS/ directory of the second of the three installation CDs. Copy it a directory somewhere and the command "rpm -i postfix*" should do the trick.
The version on the Redhat 9.0 CD is postfix-1.1.11-11.i386.rpm .
A good place to get precompiled Postfix packages is a site by Simon J Mudd with RPMs for Intel, Alpha and for System 390 (IBM mainframes):http://postfix.wl0.org/en/Courier IMAP and Courier Maildrop are not currently distributed with Red Hat. SpamAssasin is, but I didn't use it in my RH9.0 installation. The way Red Hat integrate it with Sendmail or Postfix would be very quite different from the way I call it, and Anomy Sanitizer, from my Courier Maildrop .mailfilter file.
Updates and security
It is vital to keep any Internet-connected machine up to date. I use Redhat's Red Hat Network - https://rhn.redhat.com - and the up2date program. I currently use a demo account and it is fine. You really need a broadband connection for this. I set it to update the kernel too. Then after the update process, I reboot and everything works fine, so far. This is impressive! By the way, its a good idea to have a larger boot partition than the 20 Megs I chose, because over time, with multiple updates, it will hold multiple versions of the kernel.
It is always possible that some security problem will be found with Postfix, so requiring a new version to be installed. If you installed, as I do in this example, the original package which came with the Redhat 9.0 installation, then in the event of Postfix needing to be updated for security reasons, regular updates via the Red Hat Network should make this happen with minimal drama. I find that if it has a new config file as part of the update, it does not clobber the old one, but generates the new one with an extended name including rpmnew or similar.
I subscribe to the BugTraq mailing list and keep an eye on it - http://www.securityfocus.com - as well as the Red Hat security announce list.
As part of the Redhat 9.0 installation, Sendmail was installed: sendmail-8.12.8-4.i386.rpm.
For installation and configuration information I consulted the Red Hat Postfix FAQ and HowTo:
http://www.redhat.com/support/resources/mail_news/mail.htmlThese are dated 2000, so they are little old.
The key steps are to stash the old Sendmail stuff somewhere, delete the Sendmail package, kill the Sendmail processes and then install the Postfix RPM. As the FAQ 3.2 says:mkdir /root/sendmail-old
cp /etc/aliases /root/sendmail-old/
cp /etc/sendmail.cf /root/sendmail-old/
cp /etc/sendmail.cw /root/sendmail-old/
cp /etc/mail/* /root/sendmail-old/
rpm -e sendmail sendmail-doc sendmail-cf --nodeps
killall sendmail
rpm -Uvh postfi*This caused an error message, about "myhostname" or "mydomain" needing to be set in /etc/postfix/main.cf.
In my situation, having installed both Sendmail and Postfix, and not caring at all about the Sendmail files, all I did was this:
rpm -e sendmail sendmail-doc sendmail-cf --nodeps
though I then found that sendmail-docs was not installed. This seemed to stop Sendmail from running too.
Postfix is set up to run via the System V init department (/etc/rc.d/init.d/) and can be restarted with:
/etc/rc.d/init.d/postfix restart
It is also necessary to turn on Postfix being run at boot up via this /etc/rc.d/init.d system. The proper way to manage this is to use this command to list which daemons, services etc. are currently configured to run with:
chkconfig --list > chkconfig.txt
Initially I found that Postfix was off in all runlevels. I want it on in runlevels 2, 3, 4 and 5. The way to do this is:
chkconfig postfix on
Then, by repeating the previous command, it can be seen that the desired changes have been made. (I also had to do this to make NFS and Samba - smb - work at boot up.)
Before we try to run Postfix, there is some configuration to be done.
At least one important item of Postfix's main.cf file is different from that of the version I installed in 2001: Postfix will, by default, refuse to accept any SMTP connections from any other machine. The symptoms, for the benefit of those searching the Net, are:
- Mail is not delivered to the machine by some other server or client (MTA - Message Transfer Agent or MUA - Message User Agent). Scrutiny revealed "status=deferred" and "Connection refused" in the sending mail server's /var/logs/maillog file. This is despite the /etc/sysconfig/iptables packet filtering system having a line to allow tcp port 25 (SMTP) and the command iptables --list indicating that this was currently implemented.
- Attempts to telnet from another machine to port 25: telnet xyz.example.org 25 fail with "Connection refused" whereas this should produce a line of text with Postfix identifying itself and being ready for business.
- Likewise, attempts to telnet to port 25 of the machine, from the same machine *fail* when directed to the machine's hostname or IP address - but work fine when directed to localhost: telnet localhost 25 .
More below on what this new config item is.
Here are some things to change in /etc/postfix/main.cf The names are fictitious. I always make changes to config files in a way which is easy to find. I put a note at the start with my initials that I changed things, sometimes with the date and the reason. Then just before each change, I add a comment line with my initials and a bunch of dashes to make it easy to find visually, with one or more blank lines before and after.
myhostname = zair.firstpr.com.au
myorigin = firstpr.com.au
mydestination = firstpr.com.au zair.firstpr.com.au
mynetworks = 203.36.57.192/27
The first tells Postfix the name of the computer it is running on.
The second makes outgoing mail from user blah be sent with this username but with the domain part of the email address being "firstpr.com.au" rather than the machine name "zair.firstpr.com.au".
The "mydestination" line specified the domains for which this machine is the primary mail server. I have another server configured as the secondary, and my DNS set up appropriately with zair.firstpr.com.au as the primary mail server - the one with the lowest number in the relevant zone files:
IN MX 10 zair.firstpr.com.au.
IN MX 20 postoffice.telstra.net.
The mynetworks line specifies the subnet of my LAN. Postfix will relay mail sent from any machine on the subnet, but not from any machine outside. This is essential. Firstly to enable it to forward mail from local machines and secondly to stop it being used to relay SPAM. There are fancier ways of specifying what machines Postfix will relay mail from - but this is fine for me.
Postfix also has extensive support for dealing with SPAM (UCE = Unsolicited Commercial Email or UBE), including by refusing to accept a message based on the IP address of the computer sending it. But this doesn't work if the sender sends it to the secondary (which I don't run, and therefore does not have such controls) which then sends it to the primary. Spammers are in the habit of sending to the secondary, for this very reason. Having Postfix refuse to accept the message based on the sender address or some other item in the headers works OK if the message was sent to the secondary mailserver, but that server is still burdened by having to handle the spam.
The one new thing we need to change is:
For me, to allow any other machine to open an SMTP session with this instance of Postfix, I had to alter the above so that probably any one of the these exceptjust "localhost" was selected. Since this machine has only a LAN connection, I used all. In the past, with my real mailserver, which at one stage had the LAN, a permanent dialup modem to the Net (and hence the 32 IP addresses of the subnet) and an Ethernet link to a cable modem, I would have needed something fussier to make sure I was accepting SMTP sessions from localhost, the LAN or the dial-up modem, but not the cable modem - because the cable modem was not the "official" address of this machine.# RECEIVING MAIL
# The inet_interfaces parameter specifies the network interface
# addresses that this mail system receives mail on. By default,
# the software claims all active interfaces on the machine. The
# parameter also controls delivery of mail to user@[ip.address].
#
inet_interfaces = localhost
#inet_interfaces = all
#inet_interfaces = $myhostname
#inet_interfaces = $myhostname, localhost
So I changed this to:#inet_interfaces = localhostThere may be all sorts of things specific to your situation which I have not anticipated. The main.cf file explains some options, but there are over 100 or them, so please refer to the Postfix documentation, such as at: http://www.postfix.org/docs.html .
inet_interfaces = all
#inet_interfaces = $myhostname
#inet_interfaces = $myhostname, localhost
Restarting Postfix is fine now. In the past, and for my mail server, I needed to set up the aliases in order that it receive any emails for accounts on zair. The file /etc/postfix/aliases maps the user names of incoming email addresses to local account names where they may be delivered.
As far as I know, if Postfix (or Sendmail) is receiving mail for multiple domains, then this aliases arrangement makes no distinction between the same user name of different domains. So xyz@foo.mil is treated the same as xyz@bar.mil
No doubt there is a way around this.
Q: How does one set up aliases so that everything addressed to a domain which is not covered by an existing alias will be sent to a particular account?
A: Virtual Domains. I haven't looked at this, and apparently Postfix's approach to this changed in early 2003. See http://www.postfix.org/virtual.8.html , http://www.freebsddiary.org/postfix-virtual-domains.php and Google for: postfix "virtual domains".
Here is an example of some aliases I added - but in fact the first two are not needed at all:
lucy: lucy
diamond: diamondrw: firstpr
rwhittle: firstpr
rwi: firstprThe first two are straightforward mappings of an email address to a local account. The others map my normal email address and two alternatives to the account which receives all my mail: firstpr.
I originally wrote the text below in red:
After changing the /etc/postfix/aliases file, it is necessary to run the "newaliases" program which reads this file and compiles them into a database for Postfix to use. I created a script in this directory to do this and to restart Postfix.
#!/bin/bash
# Recompile the aliases file and restart postfix.
newaliases
/etc/rc.d/init.d/postfix restart
but I stand corrected. Jussi Silvennoinen told me:
I thought you should know that it is not necessary to restart postfix
after changing aliases-map or any other map. You can simply run 'postfix
reload' which refreshes all maps. Postfix will even do it by itself in
time, every n minutes. Postfix is not sendmail, you don't need to restart
the whole MTA every time you make a small change. All
configuration-changes can be applied the same way.
Aliases can be used to run mailing-list programs. I used to have some incantations for Majordomo - but if I was doing a mailing list now, I would use GNU Mailman .I tested Postfix, partially at least, by sending an email to an account on this machine and checking that it arrived in that user's mailspool file at /var/spool/mail/ .
Testing the UW IMAP server - config and "firewall" settings
Then I checked that the already installed University of Washington IMAP server (rpm -i imap) was working by accessing the user's account via IMAP Netscape 7.02's Messenger on another machine. (See the parent directory ../ for how I set this up.) Before I could do this, I had to enable IMAP reception by xinetd, by one of the following sets of steps:
1 - The old way:
- Comment out, with a leading "#", the line " disable = yes" in file /etc/xinetd.d/imap.
- Restart xinetd with the command "killall -USR2 xinetd". (Or "/etc/rc.d/init.d/xinetd restart".)
2 - The modern way - chkconfig which reports on and controls not just the /etc/rc.d/init.d System V Init system, but also the protocols accepted by the xinetd daemon, via the settings in files in /etc/xinetd.d/:Both the above achieve the same result in slightly different ways. In the first approach I manually comment out a "disable" line in a file. In the second approach, the chkconfig system changes the line to read "no", and moves it to the front of the bracketed set of lines, whereas it was at the end.
- Run chkconfig --list and see that in the "xinetd based services" section that imap is "off".
- Run chkconfig imap on to turn it on.
- Repeat step 1 to see that it is on.
In addition, if there is packet filtering (AKA, rather loosely, a "firewall") then maybe we need to change that to alter that as well to allow IMAP sessions into the machine. But I found this was not necessary. So I am not convinced this so-called firewall is doing anything. See a separate page for my investigation. iptables-rh-sus.html .
So with the above (and creating user accounts, sending test messages, looking at /var/log/maillog ) I was able to get Postfix running, and retrieve messages via IMAP.
The first time I installed these programs I did a lot of to-and-fro-ing getting it right. I can't test Courier IMAP without having Maildrop running, and in May 2003, I want to modify Maildrop with my "Tagged for Deletion" mods - but the source code has changed significantly since I did these mods: one of the files I modified no longer exists and the other is very different. So first of all I worked on compiling and installing Maildrop. Later after getting Maildrop and Courier IMAP installed and running, I went back to Maildrop and figured the changes I wanted to make to it..
Courier IMAP also provides a POP3 server but I don't use it.
Courier does not work with the standard linear file user (AKA "Mbox" or "Mailbox" format) Inboxes: the files typically created for each user in /var/spool/mail. Instead it needs the Inboxes to be in the Maildir format, which is explained here: http://www.qmail.org/man/man5/maildir.html and http://cr.yp.to/proto/maildir.html . Postfix can be configured to deliver to Maildir mailboxes, with a specification for where they are located in the file system.
However, I want to do some fancy mail filtering, which I understand Maildrop is good at, so I am installing Maildrop: http://www.flounder.net/~mrsam/maildrop/ . (Note, to make this easier to read, I am capitalising the first letter of a number of program names.)
Courier IMAP is GPL open source software from Sam Varshavchik (Double Precision, Inc.) http://www.inter7.com . The home page is:
http://www.inter7.com/courierimap/The home of the Courier Mail Server is: http://www.courier-mta.org/ and this has a complete set of man pages for Courier MTA and its separately available components, Courier IMAPD and Maildrop.
It may be possible to find ready-to-install RPMs of Maildrop and Courier IMAP at: http://rpmfind.net/linux/RPM/ but since a number of things need to be configured at compilation time (as far as I know there is no alternative, but maybe some of Maildrop's options can be set in /etc/maildroprc ) I think that a precompiled binary RPM is not at all what most people would want. However, I don't claim to know these programs well. Presumably if someone produces RPMs then there are ways of configuring them . . . but presumptions such as these in IT have turned out to be unrealistic!
The source for both programs is at: http://www.courier-mta.org/download.php . On 26 May 2003, I got:
- maildrop-1.5.3.tar.bz2 (Latest files seem to be 2003 May 2)
- courier-imap-1.7.3.tar.bz2 (Latest files seem to be 2003 May 16)
I have always found it hard to understand and remember exactly how to compile, install and configure these programs. Ordinarily, with most programs, one gets a source tarball (tar.gz) unzips it, compiles it and installs it. Usually configuration is done after that, but sometimes there are things which need to be configured before compilation.
While it is possible to compile and install in this way, the documentation of both programs suggests it may be, or is, a good idea to use a special RPM approach to compilation which produces an RPM package, by working from the tarball directly, not the unzipped tarball. The resulting RPM or RPMs can then be installed or removed from the system as a package. In this scheme, as I understand it, one must somehow set config items before this RPM approach to compilation - but I found it hard to understand how this should be done. Also, for Courier IMAP you can't do this stuff as root! I remember being confused about what could be configured after the installation, since I found the documentation hard to find and navigate.
In May 2003, I decided to first work on Maildrop, not using the RPM method at all. Then I would install it and configure Postfix to use it. Then received mail would be written to Maildirs, and I could see this directly in the Linux machine, even though I would not then have an IMAP server to access them with. Only then I would compile and install Courier IMAP.
These pages are the ones I found in May 2003 for documentation on compiling and installing Maildrop. The boldface ones are essential reading before compiling and installing. DO NOT just rely on what I have written here. Maybe it will work for you if your situation is identical to mine, but how will you know unless you read the Maildrop documentation carefully?
- http://www.flounder.net/~mrsam/maildrop/ README.html README.html. Points to the maildrop mailing list, tells how to make the RPM, describes Maildrop's functionality and how to make it run from Sendmail. (This page appears in the frame at http://www.flounder.net/~mrsam/maildrop/ )
- http://www.flounder.net/~mrsam/maildrop/maildrop.html "maildrop.html" man page - the fullest description of Maildrop.
- http://www.flounder.net/~mrsam/maildrop/ INSTALL.html Installation guide.
- http://www.qmail.org/man/man5/maildir.html Man page for the maildir mailbox format.
- http://www.flounder.net/~mrsam/maildrop/maildropfilter.html Documents the filtering language. Also documents environment variables for purposes such as controlling where Maildrop delivers messages to..
- http://www.flounder.net/~mrsam/maildrop/maildroptips.html Hints on using the filtering language.
As user root, I un-tar-gzipped the tarball and followed the instructions in INSTALL.html - for a non-RPM compilation.
Earlier versions of Maildrop required that it find one of two database systems to use, but the current (May 2003) version will compile without these if there are none available. The two systems are "GDBM" and (Berkeley) "DB". I don't know much about these. Red Hat has one or both by default. They are used by Maildrop for some of the more advanced filtering commands which I don't think I have used. There is an option --without-db which can be given to the ./configure command (see INSTALL.html) to disable the use of these database systems. This would remove certain fancy commands from the filtering language, but significantly reduce the size of the Maildrop executable. On a high-capacity system, this would probably be a good idea unless someone really needed those fancy commands.
From the main directory of the file tree, I ran :
./configure
There were no options I wanted to give to this process, but after it is done, I need to check and perhaps edit two files which are created by this process:
- /maildrop/config.h
- /maildrop/xconfig.h
xconfig.h had nothing I wanted to change. The config.h has one thing which needs defining: the DEFAULT_DEF parameter. I changed the line in config.h to:
/* Default mail delivery instruction */
#define DEFAULT_DEF "Maildir/"This tells Maildrop to deliver to a directory (for user "blah") /home/blah/Maildir/ which is what is needed for working with Courier IMAP.
As explained in INSTALL.html, the fact that DEFAULT_DEF does not start with a / means that the rest of it will be appended to the user's home directory.
The fact that it ends with a / tells Maildrop that this is the location of a maildir mailbox, rather than a place to write Mbox mailbox files.
This directory /home/blah/Maildir/ must be set up as a Maildir before Maildrop will deliver to it. There is a program "maildirmake" for this purpose. More on this below. That directory is where ordinary delivery will take place, and all the user's mailboxes will be sub-directories of this one.
If you want to implement my DELTAG modifications to Maildrop, now is the time to introduce then. See:
>>>../ Maildrop-mods-filtering / <<<Check the version they are for (1.5.3 in May 2003) and if you are compiling a later version of Maildrop than the one I made these patched files from, then its up to you to compare the two new files and the two ones you have in your source code to determine whether you can just drop in the new ones, or whether you need to adapt my changes to the source you are dealing with. My patches are very clearly commented - you can't miss them! The files are:/maildir/maildircreate.c
/maildrop/maildir.C
These changes alter the maildrop binary. They also affect the length of the maildirmake binary but as far as I know, do not affect its function.
Now I compile it (remember to get back the the main directory from /maildrop/ !):
make
make install-stripmake install-man
INSTALL.html documents where things are installed. This is the text from INSTALL.html, with the full path to the files with reference to the compilation directory and my notes in brackets for programs I found which did not correspond with what is in INSTALL.html.
/usr/local/bin- A number of binaries will be installed here, starting with the main binary, /maildrop/maildrop, as well as additional utilities:dotlock(Actually I couldn't find this but:/lockmail/lockmail/),/maildir/maildirmake,/rfc2046/makemime, /maildrop/reformail, and/rfc2046/reformime. If certain options are selected, some additional binaries may be installed here as well, such asdeliverquota. (Also: /maildrop/mailbot).
/usr/local/share/maildrop/scripts- A number of Perl and shell scripts will be installed here, and soft links will be created from the/usr/local/bindirectory. Because these are architecturally-independent text scripts, they are installed in the/usr/sharehierarchy, but since they intend to be executed from the command line, the installation script puts soft links in/usr/local/binfor each script.
/usr/local/man- manual pages.
/usr/local/include- C header files, for development, if the--with-develoption is specified to theconfigurescript.
/usr/local/lib- C libraries, for development, if the--with-develoption is specified to theconfigurescript.
/usr/local/share/maildrop/html- HTML versions of manual pages installed in/usr/local/man.
The executable /usr/local/bin/maildrop was user root and group mail . rwxr-xr-x (100755).
Now I need to figure out how to make Postfix use Maildrop . . .
Before trying to run Maildrop from Postfix, we need to make sure that there is a Maildir ready in the user account. From /home/blah/ as user blah NOT as root, give the command:
maildirmake MaildirThis generates a directory of this name, with three empty subdirectories cur, new and tmp.
Do this for all users.
Now to to this directory and do the same thing as root:
/etc/skel/Then, every new user (generated with useradd) will have a Maildir in their home directory. (Likewise, if you want each user to have a default .mailfilter file put it in this directory.)
As noted previously Postfix is capable of delivering mail directly to Maildir mailboxes. See: http://www.postfix.org/faq.html#maildir - however I want to use the filtering capabilities of Courier Maildrop. I could not find specific instructions in any doco on how to configure Postfix to use Maildrop, but I found a few things in the archives of the Postfix mailing list. According to one message (http://msgs.SecurePoint.com/cgi-bin/get/postfix0102/395/1.html) it is simply putting "maildrop" into either the mailbox_command or mailbox_transport parameter in /etc/postfix/main.cf. Here is the relevant section of that file, with my additions in purple:# The mailbox_command parameter specifies the optional external
# command to use instead of mailbox delivery. The command is run as
# the recipient with proper HOME, SHELL and LOGNAME environment settings.
# Exception: delivery for root is done as $default_user.
#
# Other environment variables of interest: USER (recipient username),
# EXTENSION (address extension), DOMAIN (domain part of address),
# and LOCAL (the address localpart).
#
# Unlike other Postfix configuration parameters, the mailbox_command
# parameter is not subjected to $parameter substitutions. This is to
# make it easier to specify shell syntax (see example below).
#
# Avoid shell meta characters because they will force Postfix to run
# an expensive shell process. Procmail alone is expensive enough.
#
# IF YOU USE THIS TO DELIVER MAIL SYSTEM-WIDE, YOU MUST SET UP AN
# ALIAS THAT FORWARDS MAIL FOR ROOT TO A REAL USER.
#
#mailbox_command = /some/where/procmail
#mailbox_command = /some/where/procmail -a "$EXTENSION"mailbox_command = /usr/local/bin/maildrop
# The mailbox_transport specifies the optional transport in master.cf
# to use after processing aliases and .forward files. This parameter
# has precedence over the mailbox_command, fallback_transport and
# luser_relay parameters.
#
# Specify a string of the form transport:nexthop, where transport is
# the name of a mail delivery transport defined in master.cf. The
# :nexthop part is optional. For more details see the sample transport
# configuration file.
#
#mailbox_transport = lmtp:unix:/file/name
#mailbox_transport = cyrus
Also in the main.cf file I made this change:
local_destination_concurrency_limit=1
Because Maildrop can only deliver one email at a time and so should not be asked to deliver to two or more local mailboxes at the same time.
(Restart Postfix after changing the config file!)
I haven't used mailbox_transport or fallback_transport. I don't understand enough about Postfix to know whether I should I set these to point to Maildrop too. I suspect they don't affect me in my simple setup of delivering to local users, because I have had no problems in 2 years. There are all sorts of complex delivery options, via various means, to files on different machines, which are discussed on the Postfix mailing list.
Then I sent a test message, from another machine, to user "blah" and I expected to see the message appear as a file in /home/blah/Maildir/new/ . If this doesn't happen, then its time to scrutinise /var/log.maillog . Make sure that Postfix has the right location for Maildrop - that you haven't missed out a teensy detail like "local" in its path . . . You can also get a bit of debugging information if it is working - so you probably don't need debugging . . . by creating a .mailfilter file in the user's directory. It must be set to the owner and group of the user and only have read and write by that user turned on. Then, in that file have the line:
logfile "mailfilter-log.txt"and that file should have something written to it every time Maildrop successfully handles a message for that account.
Log rotation for the log file
If this log file ever gets to 50,000,000 bytes then Maildrop will fail with and error, causing Postfix to write something like this to /var/log/maillog:
temporary failure. Command output: maildrop: signal 0x19My workaround for this is to add a file to /etc/logrotate.d/, with any name, and with contents such as this:
/home/blah/mailfilter-log.txt {
weekly
missingok
}
This doesn't save me in the event of a mail loop which generates huge log files, but perhaps it would be good to have the delivery fail under such conditions and hopefully break the limit. My verbose, weekly, log is around 3 Megabytes and it zips to much smaller as part of my daily backup. I hardly ever look at it, but it can be handy to trace mail problems.
I found this an infuriating program to compile and get running. Maybe its just me - but I found the documentation really hard to translate into an understanding of what I should do and in what order. What you read below is far simpler than the path I took to get it working.
The doco I read is:
http://www.inter7.com/courierimap/ What I will call the Intro page - actually it is a frameset which has the file http://www.inter7.com/courierimap/courierimap.html as the main body of text. There is important material here on configuring for building RPMs and especially for disabling xinetd's IMAP and POP3 settings, since Courier IMAP does not use xinetd. http://www.inter7.com/courierimap/INSTALL.html Installation instructions. http://www.inter7.com/courierimap/FAQ.html FAQ. Includes: An option I want to enable to get around a bug which exists in Netscape Communicator 4.x. (I am not sure if it exists in Mozilla - Netscape 7.) It needs either the GDBM or Berkeley DB database library. My installation has GDBM 1.8.0-20. I am not sure what the package name is for Berkeley DB - but I think I installed it. (INSTALL.html says that GDBM will be used in preference to Berkeley DB.) http://www.inter7.com/courierimap/README.imap.html Notes on configuring clients, such as Netscape 4.7x Communicator and Pine. This is vital, because without proper attention to configuring the client, all sorts of things will go wrong. http://www.inter7.com/courierimap/BUGS.txt Bugs in the way certain user agents behave - such as flaky behaviour by Netscape Communicator (4.7x - what about Mozilla and Netscape 7?) when creating a folder and immediately accessing it, or loop endlessly on attachments with backslashes in the filename . . . StarOffice and some versions of Outlook Express being non-standards compliant . . . . But this material looks rather old and may not be entirely up-to-date with current versions of these programs.
First we need to get rid of the old UW IMAP server. As root:
rpm -e imapWe also need to get rid of xinetd's management of the IMAP port - so delete any file such as /etc/xinet.d/imap and restart xinetd with /etc/rc.d/init.d/xinetd restart . Removing the UW IMAP package may have already done this.
An IMAP server needs to authenticate the request from the remote user's IMAP client. The IMAP protocol involves the server sending a pseudo-random challenge to the client, which transforms it cryptographically with its password and sends the response back. No passwords are sent in the clear. But at he server end, the IMAP server needs to integrate with some external system for determining whether the response will be accepted as proof that this is from the real user, not someone trying to hack into the system.
There are various approaches to user authentication, such as LDAP and an SQL database (such as MySQL). These are typically required for large systems, but not in mine. I want to use the standard PAM (Pluggable Authentication Module) approach used by a standard RH 9.0 system. PAM is a standardised, configurable, interface to various actual authentication schemes. As INSTALL.html says:
An authentication module does a few things besides checking if a userid and password are valid. It's job also includes specifying the location of the primary maildir, and its system user and group id.
However later, it also says that PAM is only used for authentication and that the user and group IDs are still taken from the system password file. (So maybe it would be easier to use the authshadow approach . . and not use authpam at all?)
Whenever PAM is used, it needs to be configured to handle requests from Courier IMAP, so its not just a question of configuring Courier IMAP, it also requires modifying the installed PAM configuration.
My use of PAM for IMAP (and POP3 if I was using it) will be configured to use the standard (for Red Hat Linux) shadow password system. So the only user names and passwords by which IMAP can be used are those of local user accounts. (Courier IMAP can use external database libraries to save all messages in mailboxes which are implemented with the database system. This means there is no direct reliance on Unix/Linux user accounts. But this is not what I am doing.)
I found that if I did not properly constrain the compilation of Courier IMAP with various ./configure parameters, then it would create an LDAP-based authentication system which would cause the IMAP daemon not to start.
Courier IMAP can work with OpenSSL to provide encrypted IMAP over SSL. (xinetd would need to be configured to accept this - the IMAPS protocol I think.) I would be interested in using this later.
As noted above, the source tarball I downloaded on 2003 May 26 was:courier-imap-1.7.3.tar.bz2 (Latest files seem to be 2003 May 16)Do not un-tar.bzip it as root - you must be a mortal user to compile and install Courier IMAP.
There is a file named 00README.NOW.OR.SUFFER which urges me read INSTALL and not to do a quickie job of this . . .
I decided to compile and install without making an RPM at all. Here's why:
Last time I made an RPM, and now I prefer not to. Not using an RPM means there is no instant way of removing the installation. but that's not a concern for me.
The RPM approach, as I understand it, does not allow for any changes to the options which can be - and I thought needed to be - built into the program(s). Last time, I had to change one of these, so I had to do a compilation quite separate from this initial RPM installation. The Intro file describes a way of changing the options for an RPM build, but from what I can see, this only changes how the package is built (by modifying the file courier-imap.spec, but not the things we want to change as per the INSTALL.html file. Also, there are some things to do if the local rpm program does not like running as a non-root user. The RPM installation process (if PAM is used) does its own thing to /etc/pam/imap and /etc/pam/pop3 . I would rather do these things manually.The INSTALL.html file does not mention anything about RPMs - so as far as I can see, if you want to change something, such as the option to work around IMAP client bugs (such as if you want to make it work with Netscape 4.7x) - --enable-workarounds-for-imap-client-bugs - then you can't do this via a compilation which produces RPMs.
INSTALL.html has all the things to configure before compilation, and then how to compile, install do some more configuration and then get it running. The first part is done as a mortal and the second as root.
So it seems that I need to:So I read INSTALL.html . . .
- "Disable" any xinetd entries for imap and pop3 . (Courier IMAP does not use xinetd.) This is already done. Removing the UW IMAP package also removed these files in /etc/xinetd/ . Actually, I only had an imap file there and it was renames to imap.rpmsave. I moved it somewhere else so as to get rid of it from this directory entirely. Do I need to start xinetd? For Justin (just in case) I will: /etc/rc.d/init.d/xinetd restart .
- Read INSTALL.html to see if there is anything I need to change about how Courier IMAP will be built. (This is what I find really hard to do properly.)
- Compile and install with the usual ./configure and make procedures, not the RPM approach.
I am not using LDAP or MySQL - just ordinary PAM user authentication.There is a worrying looking section on how I will have to tell the PAM library how to authenticate the "imap" service . . "What you need to tell your PAM library is something that you will have to figure out by yourself, because it depends on the version of your PAM library, and your operating system." (But later I find that the installation process does this just fine. In 2001/02 I recall I had to do some careful hand-tweaking.)
Another post-installation thing is that I will have to ensure that Courier IMAP runs when the system boots up.
There is some configuration to do after installation to make IMAP-SSL work with openSSL. Likewise some start and shutdown scripts for this too. I will leave SSL out of this for the time being.
There are similar config files and start / stop scripts for the POP3 daemon and its SSL sibling. I won't bother with this.
Options to configure: The one which looks relevant to me are . . . only one:, at first . . .
--enable-workarounds-for-imap-client-bugs - there are a number of various bugs in certain IMAP clients. The current list of broken IMAP clients consists of Netscape Messenger and Sun's StarOffice. This option enables some workarounds for some bugs in these clients, however, note that this may break compatibility with software that correctly implements IMAP4rev1. Additionally, "make check" will fail when this option is used. See imap/BUGS.(html|txt) for more information. NOTE - if this option is used, make check WILL FAIL. You should first configure Courier-IMAP without this option, run make check, then reconfigure Courier-IMAP with this option.Some other options can be set at the ./configure stage, but I choose not to - these relate to authentication and it is possible to alter them in a config file after compilation.
INSTALL.html instructs me to do this:
"
make check" performs some internal sanity checks. Ifmake checkfails, something is wrong, and Courier-IMAP may not work for you reliably. Certain options are documented to causemake checkto fail, due to different IMAP protocol behavior. If you need to use those options, first compile Courier-IMAP without them, run make check, and if all goes well extract the source code again in a different directory, then build it for the second time using your options.
Building things twice just to incorporate a commonly used option is a pest - but this seems to be the price of a complex program with rigorous checking. So the first compilation is not installed - it is just a way of checking that the second compilation would be OK apart from its options which cause the test to fail, in this case: --enable-workarounds-for-imap-client-bugs .
If you accidentally run ./configure as root, delete the directory and start again! Failure to do so may clobber the ./configure process and you may find a missing autoresponsequota.h (and a bunch of others) at make check.
So the full procedure seems to be:
- As non-root user:
- Unzip the tarball into directory A.
- ./configure
- make.
- make check.
- If all is well, forget about this directory A and:
- As non-root user:
- Unzip the tarball into directory B
- ./configure --enable-workarounds-for-imap-client-bugs
- make.
- make check. (This will fail.)
- As root:
- make install-strip .
- make install-configure. (Install some config files, maybe the /etc/pam ones?)
- Tweak configuration files as noted in INSTALL.html
- Various things to make sure that the authentication and imap deamons will start and stop automatically.
- Start the authdaemond process and then start the IMAP daemon.
- Test it via an IMAP client.
1 As a non-root user, after unzipping the tarball into directory A:
./configure
There were no errors. There is a message about not being able to change passwords in "webmail" - referring to the Courier web-mail client which I am not installing.
make
No problems!
make check
No problems!!
Now for the real thing . . .
2 As a non-root user, after unzipping the tarball into directory B:
./configure --enable-workarounds-for-imap-client-bugs
The results were the same as in the A directory.
make
No problems.
make check
This ends with an error message about the --enable-workarounds-for-imap-client-bugs option.
Making check in imap
make[1]: Entering directory `/audio/Courier-IMAP/courier-imap-1.7.3/imap'
Error: --with-trashquota or the --enable-workarounds-for-imap-client-bugs
option was specified to the configure script.
As INSTALL told you, make check fails if these options are used, and I wasn't
kidding when I wrote it. Reconfigure and rebuild without these options, then
rerun make and make check. If make check passes, reconfigure again with your
original options, and proceed with installing this server. Have fun!
make[1]: *** [check] Error 1
make[1]: Leaving directory `/audio/Courier-IMAP/courier-imap-1.7.3/imap'
make: *** [check-recursive] Error 1
So all is well so far!
3 Now, as root Make sure that we are "umask 022":
umask
This should result in 0022 . As per the instructions in INSTALL.html "set your umask to 022 before running
make installormake install-strip.". "umask" is documented in man bash . The command umask displays the current state and will change it if necessary.)make install-strip
This generated a bunch of stuff, including:
You must now set up the following command to run at system boot:
/usr/lib/courier-imap/libexec/authlib/authdaemond start
Do not forget to run make install-configure
This, I think is unnecessary, since /etc/rc.d/init.d/courier-imap starts it .
I found two files installed in /etc/pam.d/ - imap and pop3 . Both were the same:
#%PAM-1.0
#
# $Id: system-auth.authpam,v 1.1 2001/02/02 05:42:57 mrsam Exp $
#
# Copyright 1998-2001 Double Precision, Inc. See COPYING for
# distribution information.
#
# This is a sample authpam configuration file that uses pam_stack
# (circa linux-pam 0.72).
auth required pam_nologin.so
auth required pam_stack.so service=system-auth
account required pam_stack.so service=system-auth
session required pam_stack.so service=system-authThe active lines are identical to other files in that directory, such as ppp , though some, such as for login were more complex:
(In 2001 or 2002 when I installed Courier IMAP in RH 7.1, as documented in the page you can reach from the parent directory, a file like the above was installed by the RPM process. On 2003 June1 when I compiled Courier IMAP 1.7.3 on that RH 7.2 system, there were no such such files before or after the compilation - so that system was operating without them since early 2002.)
Now . . .
make install-configureI am not sure what this did.
The INSTALL.html doco has instructions on starting and stopping Courier IMAPD at bootup and shutdown. This would involve adding:
/usr/lib/courier-imap/libexec/imapd.rc startto the end of my /etc/rc.d/rc.local file. It also instructs on adding a similar line with "stop" instead to my shutdown script, but I am not sure what that is. But I did not do this, because it later says there is a file to copy to the System V init system.
If your system uses System-V style startup scripts, take a look at
courier-imap.sysvinit- this is a sample/etc/init.dscript.courier-imap.sysvinitis created byconfigure. In most cases it can be merely copied to/etc/init.dand/etc/rc?.ddirectories (with the execute permission bit turned on).The
courier-imap.sysvinitscript is created in the main compilation directory. I copied it to /etc/rc.d/init.d/courier-imap and set it to user/group root and permissions 755.Then I ran this:
chkconfig --add courier-imap
This generated a bunch of files in the various /etc/rc.d/rc.0 etc. directories .
To see what state things were in I ran::
chkconfig --list > /root/init.d.txt
and found that this new courier-imap entry was added with ON for run states 2, 3, 4 and 5 - just as I want.
Now I need to look at Courier IMAP's config files, as mentioned in INSTALL.html (search for "After installation".)
I want to cut down the number of authentication methods it uses - since I believe this stopped it running in my previous attempts where I did not do this.
Courier IMAP's config files are at:
/usr/lib/courier-imap/etc/
authdaemonrc } Most of the authentication configuration is here.
authdaemonrc.dist
authldaprc
authldaprc.dist
imapd } The main config file.
imapd.dist
imapd.cnf } Minor conf files.
pop3d.cnf }
imapd-ssl
imapd-ssl.dist
pop3d
pop3d.dist
pop3d-ssl
pop3d-ssl.dist
quotawarnmsg.example
Because, in /usr/lib/courier-imap/etc/imapd, AUTHMODULES="authdaemon"I need to make some authentication changes in /usr/lib/courier-imap/etc/authdaemonrc :Because I don't want to create and use virtual mailboxes, I need to remove "authuserdb" from the "authmodulelist" list. (INSTALL.html doesn't say this precisely, but this seems to be the intent of its meaning.)
Because I want to use PAM with shadow passwords, I also want to get rid of "authldap" and two others I don't think I need: "authcustom" and "authcram" either. So I change:
authmodulelist="authcustom authcram authuserdb authldap authpam"
to:
authmodulelist="authpam"
(When installing this Courier IMAP 1.73 on my RH7.2 system, I had been running just authshadow previously, and with the new one I selected only authshadow for the authmodulelist . So maybe version 1.73 decides to include authshadow on RH7.2 but not on RH9.0.)
There are various options in /usr/lib/courier-imap/etc/imapd , including one to cause expunged messages to be written to the Trash mailbox. The one which needs to be changed is this:
In order to make the daemon start at bootup, change:
IMAPDSTART=NO
to:
IMAPDSTART=YES <<<< This will avoid much frustration with the program not starting!!!This is looked at by a script in the System V init directory: /etc/rc.d/init.d/courier-imap .
(On 2003 June 1 when I compiled Courier IMAP 1.7.3 on a Red Hat 7.2 machine, for some reason there were these two lines missing from /usr/lib/courier-imap/etc/imapd :
MAXDAEMONS=40Trying to start the IMAP server was unsuccessful, and generated messages: Invalid -maxprocsarg option. or, when that was fized: Invalid -maxperip option. in /var/log/maillog .)
MAXPERIP=4
The following action seems to be unnecessary, since /etc/rc.d/init.d/courier-imap starts it:
/usr/lib/courier-imap/libexec/authlib/authdaemond start
Now I started Courier IMAPD:/etc/rc.d/init.d/courier-imap restartThis generated:
Stopping Courier-IMAP server: imap imap-ssl pop3 pop3-sslThen I ran my standard little script to produce listings of all running processes, in two different ways:
Starting Courier-IMAP server: imap
#!/bin/bashWith a bit of reformatting, this is what I found:
ps -faxww > 0ps-fax.txt
ps -laxww > 0ps-lax.txt
1156 ? S 0:00 /usr/lib/courier-imap/libexec/authlib/authdaemond.ldap startSo much for getting rid of LDAP . . .
1157 ? S 0:00 \_ /usr/lib/courier-imap/libexec/authlib/authdaemond.ldap start
1163 ? S 0:00 \_ /usr/lib/courier-imap/libexec/authlib/authdaemond.ldap start
1166 ? S 0:00 /usr/lib/courier-imap/libexec/couriertcpd
-address=0
-stderrlogger=/usr/lib/courier-imap/libexec/courierlogger
-stderrloggername=imapd
-maxprocs=40
-maxperip=4
-pid=/var/run/imapd.pid
-nodnslookup
-noidentlookup 143
/usr/lib/courier-imap/sbin/imaplogin
/usr/lib/courier-imap/libexec/authlib/authdaemon
/usr/lib/courier-imap/bin/imapd Maildir
1169 ? S 0:00 /usr/lib/courier-imap/libexec/courierlogger imapd
Now Courier IMAP is working fine!
Courier IMAP, from the user's point of view, makes all its mailboxes as subfolders of the Inbox. As I installed it on my Red Hat system has the following arrangements for user blah :/home/blah/Maildir/
This needs to be created by some means (say by the useradd program and a suitable prototype directory in the /etc/skel/ system). To make it, one can use Courier's maildirmake binary:
as the user blah, to make this directory. This will be found at /usr/lib/courier-imap/bin/ . From /home/blah/ simply:maildirmake Maildir
This /home/blah/Maildir/ directory is a Maildir format directory. Its owner is blah and it contains three subdirectories which, like itself, are chmod 700.
/home/blah/Maildir/cur
/home/blah/Maildir/new
/home/blah/Maildir/tmpTogether, these directories constitute a Maildir format "Inbox" which I can access via IMAP with Netscape 7.02 or Mozilla. (See the parent directory for my notes on configuring these properly, for instance to get rid of sending with the "format=flowed" arrangement, which corrupts plain-text messages by the addition of a space to every line which starts with a space or a ">".)
Here is an example of user-created mailboxes. Each of these of these mailboxes is a Maildir directory with its three sub-directories.To the client they appear as a tree of directories and sub-directories, such as:
Inbox
|
|--aaa
|
|--Trash
|
|--xxx
|
|--yyy
|
zzz
zzz is sub-mailbox of yyy which is a sub-mailbox of Inbox.
Physically in the file system, they are as follows, not counting each Maildir directory's three subdirectories and two or three files which Courier IMAP puts in each one.
/home/blah/Maildir/.aaa
/home/blah/Maildir/.Trash
/home/blah/Maildir/.yyy
/home/blah/Maildir/.yyy.zzzEach can contain both messages and mailboxes.
I have not figured out the concept of shared mailboxes. This is an advanced approach suitable for having several users access common mailboxes, with the ability to mix their mails in those mailboxes and only being able to delete their own emails. Courier IMAP now can work with FAM, the File Alteration Monitor so that multiple users can be notified instantly of all changes made by others to the shared mailboxes.
With my previous University of Washington IMAP server, the arrangement was that all mailboxes were simple linear files.The Inbox for user blah was:
/var/spool/mail/blahThe user-generated mailboxes, at least the way I set them up, using the same example mailboxes as above, are at:/home/blah/mail/aaaIn this arrangement, "yyy" is a folder for containing mailboxes, it is not a mailbox itself.
/home/blah/mail/Trash
/home/blah/mail/yyy/
/home/blah/mail/yyy/zzz
I have added two features to Maildrop's filtering abilities:The full details and doco on these is in a separate directory. This also discusses my exploration of Maildrop filtering.
- The ability to deliver an email so that it is tagged for deletion. (Source code changes to Maildrop.)
- The ability to add a header, such as [Postfix] to the subject line of an email. (External filter program called subjadd .)
>>>../ Maildrop-mods-filtering / <<<Here is a brief discussion of why I want the two additional features.The ability to add a label to the subject line is useful, for reasons including:
Here is a short version of why I want to be able to put messages in the Inbox tagged for deletion. Please see the above-mentioned directory for a full discussion of why I want to do this and move mail sorting onto the server and so not depend on the client.
- Labelling all emails from a mailing list, or some other source, for easy visibility and for alphabetic sorting in the Inbox.
- Potentially the labelling of SPAM, if the Maildrop filtering scheme, or any external programs it invokes, can figure out what is SPAM and what is not.
I want to replicate the functionality of Netscape's mail filtering system - but to do it entirely at the server. I am on several dozen mailing lists and so filtering is vital. I have Netscape set up to copy messages to the particular mailbox for that mailing list, and to leave the message in the Inbox, but flagged for deletion. The filtering happens when I "Alt-F M" Netscape to check for new mail. The headers come to Netscape and it sends commands back to the IMAP server about where to put the messages and whether to set the "T" tagged for deletion flag for messages which also remain in the Inbox.The great benefit of this is that I can see all the activity on my lists without having to look into their various mailboxes. When I have scanned the "tagged for deletion" messages, (which I can read, print, reply to or untag with the Del key) then I use "Alt-F T" to expunge the mailbox. Netscape tells the IMAP server to delete them all.
(Later . . . I have set this up and it is an excellent thing!)
I wrote a Perl script which solved my problems of converting old Mbox mailboxes to the new Maildir format. This is "mb2md" and it has its own Web-abode:>>> mb2md <<<For a discussion of previous ways of doing this and exactly how I converted from the old UW IMAP arrangement, please see the end of the 2001 documentation of Redhat 7.1, Postfix, Maildrop and Courier IMAP, which you can find in the parent directory: ../ .
(Added 2002 October 23.)
I run a nightly cron job to back all my user directories and their mail to another Linux machine on the LAN. When I was using Mbox mailboxes, which could be altered at any time with incoming mail, I found I could not simply tar-gzip the directories themselves, but had to copy them to a temporary directory and tar-gzip them there. Probably, with Maildir mailboxes, I don't need to do this, but it is part of the backup script and I still use it.
As time goes on and mail accumulates, mailboxes grow and this poses some problems:
- Storage limits on the server's hard drive.
- Time taken to copy and tar-gzip for backup purposes.
- Total size of the entire backup file - I want to keep it under 700 Megs so it will go on a CD-R.
Therefore, mailboxes need to be emptied or deleted - but which ones to get rid of? In order to decide, I wanted to know how much storage each mailbox was taking in the tar-gzip file. Here is a shell script - my slight modification of an excellent script by Michael Carmak on the Courier Users list on 23 September 2002. Thanks Michael!
maildir-report-3.sh.txt maildir-report-3.sh.txt.gzAn example of its output follows. It does not report the final size of the entire backup file, since it does not create any such file. It generates temporary tarballs of each mailbox, and reports on the size of that, amongst other things. In the final backup file, the size would be different, but not by much. So with this, I can see which mailboxes contribute most to the backup file size.
This script reports on one user only, and it reports on all the mailboxes except for the Inbox. To do that, I would need to modify it to look into /cur as well. But I have a pretty good idea of what is in the Inbox since I am using it all the time. What I want to know about is the mailing list mailboxes, the virus and spam pits, and also any mailboxes I had forgotten about which are growing fat, such as ones to keep messages prior to filtering, the Drafts mailbox etc.
These are a selection of my 210 or so mailboxes.
Mailbox: .0-Inbox-Old.Inbox-02-03
Number of files: 1447
Total size of all files: 21.0592 MB
Disk space used: 25M
Approx tar.gz size: 13M
Mailbox: .0-Inbox-Old.Post-Filter-Inbox
Number of files: 9888
Total size of all files: 294.157 MB
Disk space used: 326M
Approx tar.gz size: 107M
Mailbox: .0-SPAM-etc.0SPAM-HTML-refresh
Number of files: 7
Total size of all files: 56.7617 kB
Disk space used: 88k
Approx tar.gz size: 29k
Mailbox: .Drafts
Number of files: 1397
Total size of all files: 22.4293 MB
Disk space used: 26M
Approx tar.gz size: 6.1M
Mailbox: .Lists.Mail-Courier-users
Number of files: 5697
Total size of all files: 21.7214 MB
Disk space used: 30M
Approx tar.gz size: &n