ORF Enterprise Edition was 5 years old yesterday. The first release of ORF was published on November 25, 2002, at 09:31AM (CET) with advanced features such as the ability to use more than one DNSBL (seriously, the Standard Edition could use only one at once), IP/Sender/Recipient whitelists and blacklists and the Reverse DNS Test. The first license was sold in 19 minutes, for 79 USD / EUR (yes, those days Euro and US Dollars were 1-to-1!).

Happy Birthday ORF!

Exchange Management Changes

One of the big changes in Exchange 2007 is less visible for the administrators: it was (almost) completely .NETified, i.e. rewritten in managed code. This change affects the management interface, too: previous IIS/Exchange versions could be managed using COM and ADSI (metabase), but Exchange 2007 offers a Windows PowerShell-based management interface instead.

In fact, PowerShell is the only way to manage Exchange 2007: the graphical MMC administration tools of Exchange also use PowerShell commands. For instance, when you add a new Accepted Domain using the Exchange Management Console, the console MMC actually runs the New-AcceptedDomain PowerShell command. Even refreshing the list of accepted domains is done using PowerShell, by submitting a Get-AcceptedDomain command and processing the command results.

Due to this dependence of Exchange on PowerShell, it is not surprising that ORF has to use PowerShell to manage the ORF 4.1 Exchange transport agents. For instance, to start using a new transport agent, you have to first install the agent using the Install-TransportAgent command (actually, cmdlet in the PowerShell terminology), then use Enable-TransportAgent to enable the newly installed agent and then Restart-Service MSExchangeTransport to apply the changes by restarting the MSExchangeTransport service.

And how does this affect ORF?

Well, among other things, it would look quite odd if you would have to start the Exchange Management Shell right after installing ORF and type five rather cryptic-looking commands to install the two transport agents used by ORF. To avoid this, we have to automatize the installation somehow. Another thing is that we need to query the status of the transport agents and display them in the ORF Admin Tool.

So, two ORF components are necessarily affected: the ORF Setup and the ORF Administration Tool.

We played with the idea of creating separate ORF versions for Exchange 2007 and IIS/Exchange 2000/Exchange 2003, but eventually we realized that it would be just pain in the ass for the ORF users, so any changes in the above components will be triggered by the ORF Connector Mode, i.e. ORF components will detect if Exchange 2007 is installed and act accordingly.

In IIS Connector Mode, the ORF Setup will install the ORF SMTP Module just like previous versions did. In Exchange 2007 Connector Mode, Setup will install the ORF Transport Agents (and ask the user whether restarting the MSExchangeTransport service is a good idea at the moment). Similarly, when updating, ORF will ask to stop the IISAdmin service in IIS Mode and to stop the MSExchangeTransport service in Exchange 2007 Mode. (Actually, it’s not that simple. See below).

The ORF Administration Tool will feature a reworked Status Information page for the Exchange 2007 Connector Mode. The new page will display the ORF Service status, the status of the ORF Transport Agents and a summarized status which tells whether ORF is ready to filter emails (based on the agent + service status).

To help the Setup and the Admin Tool, we will introduce a new ORF component called the ORF Agent Installer (orfainst.exe), a command-line tool written entirely in .NET (C#) that allows installing/uninstalling the ORF Transport Agents and also querying their status. A separate component was needed, because the ORF Administration Tool is written in a native language (Borland Delphi) and thus it cannot use directly use PowerShell, however, it can run the Agent Installer and process the output.

Once you will get ORF 4.1, you will notice that the ORF Administration Tool does not immediately show you the status of the ORF Transport Agents, it will take up to 10 seconds to get them. This is mostly because of PowerShell: it takes ages to load the base PowerShell environment, another year is taken by loading the Exchange PowerShell snap-in and then the Get-TransportAgent cmdlet runs reasonably fast.

Back to the installer for a short moment: one interesting question raised by the common Exchange 2007 and IIS/Exchange 2000/Exchange 2003 installer is what happens when someone runs ORF in IIS Mode and then upgrades to Exchange 2007, without uninstalling ORF? Obviously, ORF will stop working, because the ORF SMTP Module installed for IIS no longer connects ORF to the incoming emails. Our current plan to deal with this situation two ways:

• The ORF Administration Tool will detect the Connector Mode changes and will tell the user what to do to connect ORF to Exchange 2007
• The ORF uninstaller will take the above case into consideration and will stop services and uninstall components accordingly.

Some details of the above may change as we are still working on the necessary changes in ORF.

Transport Agents versus SMTP Event Sinks

Exchange Server 2000 and 2003 both used the IIS SMTP Service for email transport. ORF hooked specific events occuring the IIS SMTP Service using a programming interface called SMTP Event Sinks and this provided filtering both for IIS servers and Exchange 2000/2003.

Exchange 2007, however, no longer relies on IIS SMTP. Instead, it uses its own MSExchangeTransport service for email transport. Also, a new programming interface was introduced, called Transport Agents.

SMTP Event Sinks and Transport Agents are quite different beasts for programmers, because:

• SMTP event sinks are COM servers. ORF’s SMTP event sink is called the ORF SMTP Module, which is a lightweight component that connects IIS SMTP to the ORF Service. Event sinks can be written in any native programming language that supports creating COM servers—for instance, originally, the ORF SMTP Module was written in Borland Delphi and was just recently ported to C++ (for 64-bit compatibility).
• Transport Agents are .NET managed objects. This means that they have to be written in a .NET managed language, such as C# or Visual Basic.NET, but not in C++ or Borland Delphi.

…from this, it is quite obvious that ORF requires a new Exchange Transport Agent component, written in a managed language, but the list of changes does not stop here, because the Transport Agent API is different from the SMTP Event Sink API…

• Some events have no 1:1 mapping. The ORF SMTP Module hooks 3 events of the IIS SMTP. The first event is when the SMTP client issues the RCPT TO recipient specification command on an incoming SMTP connection. This is the Before Arrival filtering point. The second event hooked is when the email transfer was just finished on an inbound SMTP connection (On Arrival filtering point). The third, optional event is what ORF hooks when it has an outbound binding to an SMTP virtual server. This event is triggered when a MAIL FROM sender specification command is submitted by the local server on an outgoing connection. ORF uses this event to collect the outbound email sender / recipient addresses and to store this information in the Auto Sender Whitelist.
  
  Using Exchange 2007 Transport Agents, you can hook the first two events (using an agent subtype called SMTP Receive Agents), but not the third one, because there is no way to hook events occuring on outbound connections. For some reason, Microsoft has decided that this feature is not important to have in Exchange 2007.
  
  Consequently, ORF will have to hook a different type of event to monitor the outbound emails. This event type of called a Routing Event and it is invoked for both inbound and outbound emails. In this type of event, it is the responsibility of the caller to decide whether the email is outbound, which is not an obvious task.
• Some properties that ORF previously relied on do not exist. The ORF SMTP Module queries the SMTP authentication status and the authenticated user’s name from IIS to allow whitelisting of authenticated SMTP sessions. In Exchange 2007, the user name is not available and the authenticated status can be Anonymous, Organization and Partner. It is not entirely clear at the moment what these mean (yes, I did not do my homework :), but we suppose that anything other than Anonymous should be considered authenticated. It also looks possible to hook the authentication event itself to get the authenticated user’s name (where it makes sense), but most likely we will not do that—it would be too little gain for the price paid.
  
  There is another issue with the Pickup folder path that I will talk about in another article.
• And then there are new properties with no matching pair in SMTP event sinks, such as AntispamBypass, which “indicates whether the session is a connection with a partner configured for anti-spam bypass”. We are not sure yet how to categorize this, but this check will be probably part of the AUTH Whitelist.
  
  A very nice new property is called LastExternalIPAddress, which would have been terribly useful in the previous Exchange versions. It is basically what ORF determines on IIS using the Intermediate Host List. So far it seems we will not rely on this property, however, because it adds no extra value compared to the Intermediate Host List, which we will continue to use due to its other roles.
• Transport Agent changes are not as instant as SMTP Event Sink changes. IIS could install and uninstall SMTP Event Sinks on the fly, however Exchange 2007 requires changing the MSExchangeTransport service on virtually any transport agent change.

Of course, this above is only a limited list of the most visible changes only, but you will see more in the next article of the series when I will talk about the management changes.

From the ORF viewpoint, the most significant change that Exchange 2007 introduced is that it no longer relies on the Internet Information Services (IIS) SMTP Service for email delivery. Instead, it uses its own transport layer provided by the MSExchangeTransport service. As ORF integrates with IIS SMTP using a technology called SMTP Event Sinks, it cannot provide filtering for Exchange 2007 the same way it did previously.

Exchange 2007 uses a new integration technology called Transport Agents and ORF need to be changed significantly to be compatible with the new interface.

In this series series of articles I will talk about how the changes introduced by Exchange 2007 affect ORF and how do we deal with them.