Exchange 2007 Anti Spam Filtering

Many companies go out and buy expensive third party spam appliances that act as smart hosts in their DMZ before understanding what they have currently purchased when buying Exchange 2007 licenses. In this post I am going to highlight the full extent of Exchange 2007's new anti-spam technology and also provide better alternatives then spam appliances for companies that want to go beyond the functionality provided in Exchange Server. By reading this I hope you achieve a more strategic approach to your anti-spam technology and save money where possible.

People don't know that Exchange 2007 can do spam filtering because all the spam filtering functionality is disabled by default. To enable it you need to run a powershell script located in C:\Program Files\Microsoft\Exchange Server\Scripts called install-AntispamAgents.ps1. This can be enabled on a hub transport server or an edge transport server. Edge Transport is the ideal server to enable anti-spam on as this exchange server role is designed to sit out in the DMZ by itself and communicate with the outside world. For more information about enabling anti-spam agents see:

http://www.petri.co.il/install-anti-spam-exchange-2007.htm

Below we will be going through the various aspects of Exchange 2007's Anti Spam Technology:

Content filtering

Exchange 2007's Content Filter is called IMF (intelligent message filter). Content Filtering is the same principal regardless what anti-spam device your using. With content filtering, the anti-spam server downloads the entire email, then analyses the email and provides a SCL (Spam Confidence Level) rating from 0 to 9. A value of 9 is definitely spam, a value of 0 is not spam. Like any content filter you can make it stricter or looser... the stricter you make it the more false positives you get (emails that are legit but detected as spam).

You can configure the Content Filter agent to take the following actions on messages according to their SCL rating:

- Delete message
- Reject message
- Quarantine message

For example, you may determine that messages that have an SCL rating of 7 or higher must be deleted, messages that have an SCL rating of 6 must be rejected, and messages that have an SCL rating of 5 must be quarantined.

The Exchange 2007 intelligent message filter can customized configuring custom words or phrases to be either blocked or allowed to modify the SCL score by whatever value you seem fit.

Exchange 2007's content filter is more powerful than many others on the market including many spam appliances you need to pay for! When you finish reading this article you will understand why.

The Intelligent Message Filter gets updated every 2 weeks by a Microsoft update to ensure it keeps logic about all the new spam emails that are flying around the internet. If your IMF filter is missing lots of stuff, install your windows updates!

There is one disadvantage of IMF however, it cannot scan emails over 11MB in size. These emails will simply pass through unscanned. However the default maximum message size limit on Exchange is only 10MB so for many companies this will not be a problem. It is also very rare that spam emails are over 11MB in size, as spammers want to send as many out as possible to get their message out, they cannot do this if they are sending large emails.

For more on Content Filtering see:

http://technet.microsoft.com/en-us/library/bb124739.aspx

Connection filtering

Connection filters are the first thing that is used to check an incoming email. Connection filters look at the IP address of the sender. If the senders IP address is marked against a list, the connection is terminated before the server on the other end before it even gets to send its HELO or EHLO statement in result saving you on bandwidth and CPU by not having to download the spam email, analyses it with a content filter and then decide the action. If a IP is determined as bad the connection is simply dropped!

There are four types of Connection Filters you can configure:
- Administrator-defined IP Allow List
- Administrator-defined IP Block List
- IP Block List providers (Real-time Spam Black Lists (RBL))
- IP Allow List providers

In the real world you’re mainly going to just use the IP Block List providers. You can configure as many RBL's as you want. However keep in mind when an email comes in, before your exchange server starts receiving the content of the email it needs to query each IP Block List provider on the internet to see if the senders IP address exists in the list – and if so block it. You can find out if a company has too many RBL providers by telneting their SMTP server, if the connection hangs for a while with a black screen before you get the SMTP Banner, that is because they have a fair few RBL providers that their email server is busy checking before accepting communication.

There are hundreds of RBL providers out there on the internet that you can use for free. My faverote RBL provider is spamhaus, its one of the large ones and has a huge list that is regulary kept up to date.

Spamhaus have 3 spam lists, SBL, XBL and PBL.

The SBL is a realtime database of IP addresses of verified spam sources and spam operations (including spammers, spam gangs and spam support services), maintained by the Spamhaus Project team and supplied as a free service to help email administrators better manage incoming email streams.

The Spamhaus Exploits Block List (XBL) is a realtime database of IP addresses of hijacked PCs infected by illegal 3rd party exploits, including open proxies (HTTP, socks, AnalogX, wingate, etc), worms/viruses with built-in spam engines, and other types of trojan-horse exploits.

The Spamhaus PBL is a DNSBL database of end-user IP address ranges which should not be delivering unauthenticated SMTP email to any Internet mail server except those provided for specifically by an ISP for that customer's use. The PBL helps networks enforce their Acceptable Use Policy for dynamic and non-MTA customer IP ranges.

If you want to use all three of these Spamhaus has a RBL list called ZEN that encompasses all three.

Microsoft has their own IP Reputation Service also exclusively to Exchange 2007/2010 customers that you may want to consider implementing as well.

For more about connection filtering see:

http://technet.microsoft.com/en-us/library/bb124320.aspx

Spam Quarantine

Spam Quarantine stores messages marked at spam by the Intelligence Message Filter. You can quarantine the email in a spam mailbox inside your organization, or deliver the spam email to the users junk email folder or both. For example you can have messages that are at a very high SCL rating to go straight to the spam quarantine which administrators have to review using an outlook client and release it to the user if necessary. You can then have messages that have a borderline SCL rating to get released to the users junk email folder in their outlook.

For more information about Spam Quarantine see:

http://technet.microsoft.com/en-us/library/aa997692.aspx

Recipient Filtering

This is required whenever you have an edge transport server out in your DMZ. The Edge Transport Server is a workgroup PC, it is not a member of your domain and does not have any direct access to active directory. When emails come in from the internet that are addressed to an internal recipient, the edge transport server needs to know if that recipient email address actually exists in the exchange organization. If it doesn’t know this, it may forward emails to your internal hub transport servers for addresses that do not actually exist inside your exchange organization.

Exchange 2007 sends this information to the Edge Transport server using EdgeSync. This is a subscription that is made between the hub transport servers and the edge transport servers that uses ADAM (Active Directory Application Mode) or AD LDS (Active Directory Lightweight Directory Services) is what its called now in server 2008. This is a portable copy of Active Directory. EdgeSync is one way replication and only replicates a couple of attributes that are required by the exchange 2007 edge transport server.

For more information about Recipient Filtering see:

http://technet.microsoft.com/en-us/library/bb123891.aspx

Sender Filtering

Sender filtering lets you block individual email addresses such as billy@hotmail.com. It also has an option to let you block any emails that come in that do not have a senders email address specified.

Sender ID

Sender ID varifys each email sent from a domain name such as @microsoft.com actually came from @microsoft.com by performing a reverse DNS lookup to ensure the address was not spoofed. It also goes further to verify if there is an SPF (Sender Policy Framework) record int he senders public DNS. SPF records are not defined by many companies when they should be as its part of the IEEE framework. SPF records are a type of custom DNS record such as an A record. You enter it into your public DNS zone file.

For more information about Sender ID see:

http://technet.microsoft.com/en-us/library/aa996295.aspx

Safelist Aggregation

Safelist Aggregation blows many anti-spam technologies out of the water as it integrates with the users Outlook client. This functionality collects data from the anti-spam Safe Recipients Lists or Safe Senders Lists and contact data that Outlook users configure and makes this data available to the anti-spam agents on the computer that has the Edge Transport server role installed using EdgeSync. Safelist aggregation can help reduce the instances of false-positives in anti-spam filtering that is performed by the Edge Transport server.

Safelist Aggregation is quite complex to setup and requires users to have entered data into their safe senders, or safe recipients list in outlook which no one ever does right? I find the best way to populate these fields is to enable by group policy "Automatically add people I send e-mail to the Safe Senders List". This enforces the option to be enabled on everyones outlook client for either Outlook 2003, 2007 or 2010.



The data in the Safe Senders is stored in Active Directory and is replicated via Edge Sync to the Edge Transport server. For exchange 2007 RTM you could have 1024 entries in AD for safe senders, with SP1 it went up to 3072. When these records get full, it starts removing the oldest records.

The information that Safelist Aggregation collects from outlook is:
- Safe Senders
- Safe Recipients
- Safe Domain
- External Contacts

This data is hashed using SHA-256 under the users attributes such as msExchangeSafeSenderHash and msExchangeSafeRecipientHash so its very secure.

When email comes in, the exchange content filter IMF looks to see if the sender is in the users safe senders list and if so greatly reduces the SCL rating applied to the email. This allows you to configure a very tight SCL quarantine rating on your organisation without recieving many false positives.

For more information on Safelist Aggregation see:

http://technet.microsoft.com/en-us/library/bb125168.aspx

Sender Reputation

Sender Reputation gathers statistical information about SMTP Sessions, IMF Content Filtering, Sender ID Verification and general sender behavior and creates a history of the sender’s characteristics. If the data gathered concludes that the sender is a spammer they are added to a block senders list. This means that the senders IP address will be blocked by the connection filter if the user tries repetitively tries to spam the domain. Because connection filters simply block the connection it also means that the Intelligence Message Filter doesn't have to rescan emails that are already going to be spam reducing server load.

You can also configure IP addresses blocked by Sender Reputation to be blocked temporarily for a time period such as 48 hours or whatever you want to define. This means you do not have to worry about removing blocked entries as they will automatically remove themselves. If the offender continues to send spam emails Sender Reputation will then automatically block them for another 48 hours and so on.

For more information on Sender Reputation see and to see how SRL calculates its statistics see:

http://technet.microsoft.com/en-us/library/bb124512.aspx

Virus Protection

Exchange 2007 has no built in virus filtering. However it has features such as attachment filtering where you can specify particular types of attachments that are not allowed through. This is known as Attachment Filtering, for more information see:

http://technet.microsoft.com/en-us/library/bb124399.aspx

Additionally using virus RBL lists such as Spamhaus's XBL list, you block all known IP's that have worms/viruses and are currently known to spam. Just with the integrated exchange filtering technology you can protect yourself against most virus threats.

If you do want to do a content level filtering of attachments using an antivirus engine you will need to install an exchange capable mail filtering solution on your edge transport server such as Microsoft Forefront Security or a third party vendor. Also with Forefront Security you get IMF updates every 24 hours instead of every 2 weeks like you do through the standard windows update process.

How does this go in the real world?

From my experiance Exchange 2007's anti-spam technology if setup correctly is extremely effective. Many companies do not use it due to it's complexity to setup.

On my home network I run Exchange 2007 spam filtering. To show you an example of how effective this spam filtering is... In the last 2 weeks I have not recieved any spam emails in my inbox. Looking on the my Exchange Server I have a total of 4263 emails blocked from my RBL providers. Notice I am only using the Spamhaus provider. I could add more in here if I wish. Please note that if an IP is detected by one RBL provider it does not check the others. This is why xbl has the biggest number. Many of these 4263 emails would have had viral attachments.



38 did make it through the connection filter, but IMF picked them up:



My email address is associated to a Microsoft .Net passport as well as being posted all over the internet. Out of all this spam sent to me in the past 2 weeks, not one hit my inbox, not one. This is not saying that the spam filter is bullet proof but it does a very effective job seeming its free technology that comes with exchange 2007 out of the box.

Going Beyond Exchange

If you do want to go beyond the capability integrated into Exchange Server and buy an appliance you need to make sure the appliance supports replication of data such as Safelist Aggregation and the internal Recipient list from Exchange, if you dont set this up correctly it could be less effective then an Edge Transport server.

However personally I would never buy an appliance, I would go to a third party such as messagelabs.com as I have had great success with them in the past or an Exchange Hosted Filtering provider. This requires you directing your company MX records to the filtering provider. The hosted provider then relays the clean emails to your organization. The benefit you get out of this is it reduces your internet bandwidth usage. If spam emails are sent directly to your organization, your servers process the request by checking your RBL providers - if negative, receive the email pass it through your content filter if positive then block it. For a large company this will consume a fair bit of bandwidth just dealing with "crap". Going through a hosted filtering provider eliminates this as they have to deal with spam emails internally.

Final Comments

I hope now you understand the full potential of Exchange 2007's spam filter and consider it before going and wasting money on expensive spam appliances that do pretty much the same thing.

Please provide me with any feedback either by leaving a comment or shooting me an email at clint@kbomb.com.au