ORF has two major issues with the Greylisting and the Auto Sender Whitelist databases:

• Sharing: databases cannot be shared between ORF instances
• Reliability: the database engine is not rock solid on some systems

Sharing is a simple case. Database sharing would be needed on networks with more inbound email servers: it is no fun when an email gets temporarily rejected two times by Greylisting, just because the left hand doesn’t know what the right hand is doing.

Reliability is a bigger issue. Only few systems are affected by engine reliability issues (namely, database corruption) and we found no common points between them, also the issue never reproduced in our lab. Under unrealistically high direct engine load, we could get some database operations to fail, but we could never achieve anything as serious as a database corruption. Still, the fact that we could drive the database into failures indicates that something is wrong with the thread safety of the database engine.

In ORF 4.0, we will address both of these issues.

To fix the reliability problems, ORF 4.0 will limit all database operations to a single thread only. The engine is proven robust with single threaded use and although this limitation reduces the throughput, the database performance will be OK for most smaller systems. Typically, ORF operations on complete well within 1 second (even on a large database), so 60 * 60 * 24 = 86.400 operations can be performed a day, enough for approx. 50.000 emails/day. It is just rough estimation, but I think the actual figures will be better and for the question of scaling, the answer will be the same as for the other major issue.

Database sharing will be implemented using external databases, with support for the following servers: Microsoft SQL Server (MSSQL) 2005 Express Edition, MSSQL 2000 and MSSQL 2005 (initially). The benefits of this model are numerous:

• allows sharing the databases between ORF instances,
• provides high performance,
• can scale well (SQL Server Express -> SQL Server),
• allows programmatic access to Auto Sender Whitelist data.

The price of choosing SQL servers is that the databases have to be created manually. We will provide downloadable guides with setup instructions and the necessary SQL statements, but running database servers requires some field-specific knowledge (e.g. adding users, configuring network access, etc.) that we cannot fully eliminate by guides.

This has turned out to be a lot more interesting issue than I expected. I was thinking about contacting MS PSS and so I wrote an as-simple-as-possible test application to reproduce the issue. Surprisingly, the test app worked flawlessly on Vista, so started investigating the differences.

First, my test application was using the HTML Help API (hhctrl.ocx) directly and was statically linked (which means that Windows would not allow the test app to start if hhctrl.ocx is not available on the system path). ORF, on the other hand, uses a third-party library that loads hhctrl.ocx dynamically and uses some registry magic to determine the location of hhctrl.ocx, regardless the system path.

My initial thought was that the ORF Log Viewer and the test app was using different versions of hhctrl.ocx and the Log Viewer found the broken one. To test my theory, I started both applications and checked the loaded DLLs by Process Explorer just to find that Log Viewer doesn’t even have hhctrl.ocx loaded. Wow, that makes sense! Calls to hhctrl.ocx will hardly work if it’s not loaded. Actually, our calls to HtmlHelp() ended up in the middle of invalid memory.

I checked the library code again, now digging deeper in the loader code and found that it tries to determine the location of hhctrl.ocx by checking its COM registration under HKCR\CLSID\{adb880a6-d8ff-11cf-9377-00aa003b7a11}\InprocServer32. Under pre-Vista operating systems, the default value for this key used to be simple String Value (REG_SZ), containing e.g. “C:\WINDOWS\system32\hhctrl.ocx”. On Vista, however, this is an Expandable String Value, pointing to e.g. “%SystemRoot%\System32\hhctrl.ocx”. The library fails to check for this and to expand the the value into a real file system path, so it also fails to load hhctrl.ocx. Even worse, the library does not offer proper error handling, so the caller application is not aware of the failed load and calls the trusted HtmlHelp() call, resulting in some accidental endless loop.

End of story, library fixed.

If you can’t wait for ORF 4.0 to run the Log Viewer and the Reporting Tool on Vista, you can apply a manual fix as described below:

• Run regedt32.exe as Administrator (%SystemRoot%\System32\regedt32.exe)
• Open the HKEY_CLASSES_ROOT\CLSID\{adb880a6-d8ff-11cf-9377-00aa003b7a11}\InprocServer32 key
• Change the %SystemRoot% variable in the value to C:\Windows (or what is your local system root path)

Expect a little fun with permissions. Ah, and don’t forget to change it back to %SystemRoot% when ORF 4.0 ships—I am not aware of any problems with hardcoding the system path, but who knows.

No big deal because there is no point installing ORF on Windows Vista: the IIS SMTP component has been removed from IIS 7 of Vista (rant below). Yet even if you do not install ORF on Vista, you may try running the ORF Log Viewer on your brand new OS, but what you get is a program that eats 100% of the processor time and has to be shut down from Task Manager.

I’ve done some debugging and found that the ORF Log Viewer (and the ORF Reporting Tool) hangs on the HTML Help initialization:

HtmlHelp(0, nil, HH_INITIALIZE, dwCookie);
(see MSDN)

It is a Microsoft API that should just work, but it does not. Any ideas?

And now, the promised ranting (I love ranting! :). Why IIS SMTP had to be removed from IIS 7 on desktop Vistas? According to a Microsoft blog post , the whole point of shipping IIS 7 with Vista before Longhorn Server is released is to make life of developers easier. So why they actually make our life harder? As the blogger points out, developers typically use desktop Windows versions for development and testing. We do not run server operating systems on our desktop computers. Now if we decide to upgrade to Vista for developer desktop, we will have to install Windows 2000/2003 Server or XP into a virtualization solution, testing code in the virtual machines. That is big PITA, so Microsoft, I am not happy for your decision.

Ever since the Active Directory-based recipient validation was implemented in ORF, we receive requests to make it exempt from whitelisting, i.e. to reject the invalid recipient even when e.g. the sender is whitelisted.

The request is perfectly reasonable, yet it does not play very well with ORF.

Here is the problem: ORF is primarily a spam filter, but some of its features have email security role more than anti-spam. These tests are the AD Test, Recipient Blacklist (for recipient validation), Attachment Filtering and External Agents (agents filtering for viruses). Regardless role, these are blacklisting tests, so whitelists take precedence over them. This results in hardly reasonable scenarios like letting a virus-infected email enter to your system, just because the sender is (auto) whitelisted.

Obviously, we have to deal with this problem somehow. We went through to quite a few possible solutions, but unfortunately none of these were without tradeoffs.

First, we cannot just re-prioritize these tests to run without whitelist control. Remember that ORF monitors all incoming SMTP connections, including SMTP connections that are used to relay outbound emails. If no whitelisting mechanism is employed, ORF filters outbound email as well. That may sound acceptable, or maybe even great for Attachment Filtering and External Agents, but I am quite confident that you do not want the recipients of your outbound emails validated as local addresses. A few would pass.

Ok, so we still need to exclude outbound emails from these tests. Good news: ORF already does this. Bad news: it does it a weird (but otherwise practical) way.

The best way to tell whether the email is inbound or outbound is to check the recipient domain and if it is a domain that your server is responsible for, the email is inbound, otherwise outbound. Cool, but you might have noticed that ORF does not keep track of the local domains.

Instead, ORF uses two whitelists which almost accidentally eliminate the need for outgoing email detection. The first is the AUTH Whitelist that whitelists emails arriving via authenticated SMTP connections, thus it covers authenticated outbound relaying. The second whitelist is the IP Whitelist, which

• Whitelists email from the local host
• Whitelists emails coming from Intermediate Hosts, which always includes the private intranet address ranges.

These two IP Whitelist rules cover the typical outbound relaying cases, and when not, it is enough to add relaying server’s IP address to the Intermediate Host List.

Excellent, but there is a glitch: the Intermediate Host List is not for detecting outbound emails. Typically, you add your secondary MXs and front-end servers to the IHL, allowing ORF to “look behind” these in the delivery path and check the actual source IP (DNS Blacklists, SPF Test, etc). In other words, the IHL-based IP Whitelist rule tells you whether the email is coming from a trusted source and this covers your trusted outbound relay hosts, too.

So a design decision had to be made: either (A) we require the user to list the local domains for proper
inbound/outbound email detection or (B) run the AUTH Whitelist + IP Whitelist always, even when email security tests are exempt from whitelist scope.

After long consideration, we decided to use option (B). Here are the advanteges and disadvantages:

Advantages:

• No administrative overhead
• Exceptions for email security tests can be enabled by default. This is important because many ORF users actually think that ORF already works this way, so they will not seek a configuration option to enable exceptions.

Disadvantages:

• If you have a front-end server, you will not have recipient validation at the Before Arrival filtering point. It is not that you had that before, but technically you could have it now.
• Your front-ends, secondary MXs and hosts listed on the IP Whitelist or anyone who successfully authenticates with your SMTP server will be able to bypass these email security tests.

We believe this is still better than requiring to list all domains and maintaining that list (sure we can detect the domain list, but eventually you would have to at least confirm that list and maintain it later).

Since ORF 3.0, we sign every ORF binaries with Vamsoft’s Authenticode code signing certificate. This is great, because you can be sure that you have the original files from Vamsoft and no tampering was made with them. This practice is also encouraged by Microsoft and signed components probably will be a requirement for the Windows ‘Longhorn’ Server Certification Logo program.

When we released the Vamsoft Image Spam Agent extension for ORF, we signed the agent imgspamagent.exe binary as usually and let it go. Then we started receiving reports about timeouts running the agent and we did not have any idea why only specific servers were failing to run the agent and why the agent worked when they tried running it from command-line. And we were struggling to find an explanation until one of our clients (thanks!) found that the agent tries to connect to crl.verisign.com:80 every time it starts. Cool.

I mean, uh? Pardon me?

Obviously, the timeout was caused by the security settings of the affected servers—these servers blocked HTTP access to crl.verisign.com, thus the timeout error. But what makes the agent trying to visit VeriSign if we did not add such code to it?

Here is the trick: Windows was checking the Certificate Revocation List (CRL) of VeriSign to make sure that our certificate was not revoked, e.g. because our certificate was compromised and now bad people are trying to impersonate Vamsoft. The funny thing is that Windows does not check the CRL for regular ORF binaries, because they were written in Delphi, but the agent was written in .NET/C# and .NET programs behave differently.

According to this blog entry, if your .NET assembly is signed using Authenticode, every time the assembly is loaded, Windows performs the Certificate Revocation List check (depending on the user settings). This potentially means a couple of seconds wait to get your .NET program started or even more if the firewall prevents the program from accessing the CRL.

Now we know. If you sign it, it won’t run.