The below error message in /var/log/syslog can have two popular causes:
Jul 7 15:01:27 myserver imapd-ssl: Failed to connect to socket /tmp/fam--
Jul 7 15:01:39 myserver imapd-ssl: Failed to connect to socket /tmp/fam--
Jul 7 15:02:28 myserver imapd-ssl: Failed to connect to socket /tmp/fam--
Jul 7 15:02:43 myserver imapd-ssl: Failed to connect to socket /tmp/fam--
One possibility is that the package "gamin" is not installed. If this is the case for you then you can merely run:
$ sudo apt-get install gamin
As most people will be using a Debian or a Debian based distibution such as Ubuntu, it is unlikely that gamin will not be installed as many packages require it as a dependency. You may have noticed that the service crying out in syslog is imapd-ssl (or courier-imap-ssl in this case) for which gamin is a dependency and hence is already installed.
So if gamin's already installed, what's giving us this grief? The problem is that the users connecting via imapd-ssl are being authenticated against PAM correctly but are not being found by the nsswitch services. In this particular example, the nsswitch services were not finding the LDAP users because of a permissions problem with /etc/libnss-ldap.conf.
See here for the instructions on correcting libnss-ldap.conf