OK. NO TCO OR ROI HERE. WE LIED.
Posted on February 7th, 2007 by Peter | Permalink

Ever since the Active Directory-based recipient validation was implemented in ORF, we receive requests to make it exempt from whitelisting, i.e. to reject the invalid recipient even when e.g. the sender is whitelisted.

The request is perfectly reasonable, yet it does not play very well with ORF.

Here is the problem: ORF is primarily a spam filter, but some of its features have email security role more than anti-spam. These tests are the AD Test, Recipient Blacklist (for recipient validation), Attachment Filtering and External Agents (agents filtering for viruses). Regardless role, these are blacklisting tests, so whitelists take precedence over them. This results in hardly reasonable scenarios like letting a virus-infected email enter to your system, just because the sender is (auto) whitelisted.

Obviously, we have to deal with this problem somehow. We went through to quite a few possible solutions, but unfortunately none of these were without tradeoffs.

First, we cannot just re-prioritize these tests to run without whitelist control. Remember that ORF monitors all incoming SMTP connections, including SMTP connections that are used to relay outbound emails. If no whitelisting mechanism is employed, ORF filters outbound email as well. That may sound acceptable, or maybe even great for Attachment Filtering and External Agents, but I am quite confident that you do not want the recipients of your outbound emails validated as local addresses. A few would pass.

Ok, so we still need to exclude outbound emails from these tests. Good news: ORF already does this. Bad news: it does it a weird (but otherwise practical) way.

The best way to tell whether the email is inbound or outbound is to check the recipient domain and if it is a domain that your server is responsible for, the email is inbound, otherwise outbound. Cool, but you might have noticed that ORF does not keep track of the local domains.

Instead, ORF uses two whitelists which almost accidentally eliminate the need for outgoing email detection. The first is the AUTH Whitelist that whitelists emails arriving via authenticated SMTP connections, thus it covers authenticated outbound relaying. The second whitelist is the IP Whitelist, which

  • Whitelists email from the local host
  • Whitelists emails coming from Intermediate Hosts, which always includes the private intranet address ranges.

These two IP Whitelist rules cover the typical outbound relaying cases, and when not, it is enough to add relaying server’s IP address to the Intermediate Host List.

Excellent, but there is a glitch: the Intermediate Host List is not for detecting outbound emails. Typically, you add your secondary MXs and front-end servers to the IHL, allowing ORF to “look behind” these in the delivery path and check the actual source IP (DNS Blacklists, SPF Test, etc). In other words, the IHL-based IP Whitelist rule tells you whether the email is coming from a trusted source and this covers your trusted outbound relay hosts, too.

So a design decision had to be made: either (A) we require the user to list the local domains for proper
inbound/outbound email detection or (B) run the AUTH Whitelist + IP Whitelist always, even when email security tests are exempt from whitelist scope.

After long consideration, we decided to use option (B). Here are the advanteges and disadvantages:

Advantages:

  • No administrative overhead
  • Exceptions for email security tests can be enabled by default. This is important because many ORF users actually think that ORF already works this way, so they will not seek a configuration option to enable exceptions.

Disadvantages:

  • If you have a front-end server, you will not have recipient validation at the Before Arrival filtering point. It is not that you had that before, but technically you could have it now.
  • Your front-ends, secondary MXs and hosts listed on the IP Whitelist or anyone who successfully authenticates with your SMTP server will be able to bypass these email security tests.

We believe this is still better than requiring to list all domains and maintaining that list (sure we can detect the domain list, but eventually you would have to at least confirm that list and maintain it later).