Understanding Network Services and Active Directory Domain Controller Placement for Exchange Server 2013 (part 5)

SMTP Mail Security, Virus Checking, and Proxies

Spamming and security issues are daily concerns for email administrators. As the Internet grows, so too does the amount of spam that mail servers have to confront. Unwanted messages not only can take up a lot of space on mail servers, but can also carry dangerous payloads or viruses. Administrators have to maintain a multilayered defense against spam and viruses.

There are several security areas that have to be addressed:

• Gateway security to control access to the mail server delivering messages to/from the Internet

• Mail database security where messages are stored

• Client mail security where messages are opened and processed

Gateway security is a primary concern for administrators because a misconfigured gateway can become a gateway used by spammers to relay messages. Unauthenticated message relay is the mechanism spammers rely on to deliver their messages. When a server is used for unauthenticated message relay, it not only puts a huge load on server resources, but also might get the server placed on a spam list. Companies relying on spam lists to control their incoming mail traffic refuse mail delivered from servers listed in the database; therefore, controlling who can relay messages through the mail relay gateway is a major concern.

Application-level firewalls such as Microsoft ForeFront Threat Management Gateway (TMG) allow mail proxying on behalf of the internal mail server. Essentially, mail hosts trying to connect to the local mail server have to talk to the proxy gateway, which is responsible for relaying those messages to the internal server. Going one step further, these proxy gateways can also perform additional functions to check the message they are relaying to the internal host or to control the payload passed along to the internal server.

This configuration is also helpful in stopping dangerous viruses from being spread through email. For example, dangerous scripts could potentially be attached to email, which could execute as soon as the user opens the mail. A safe configuration allows only permitted attachment types to pass through. Even those attachments have to pass virus checking before they are passed to an internal mail server.

The following process describes how one server contacts another server to send email messages that include virus checking:

1. The sender contacts its SMTP gateway for message delivery.

2. The SMTP gateway looks up the MX record for the recipient domain, then looks up the A record, and establishes communication with the host at the IP address specified in the A record. The application proxy acting as the SMTP server for the recipient’s domain receives the message. Before the recipient gateway establishes communication with the sender gateway, it can check whether the sender SMTP gateway is listed on any known spam lists. If the server is not located on any spam lists, communication can resume and the message can be accepted by the proxy server.

3. The application proxy forwards the message for virus checking.

4. After virus checking, the mail is routed back to the application proxy.

5. Mail is delivered to the internal SMTP gateway.

6. The recipient picks up the mail message.

Third-party products can be used for virus checking not only at the gateway level, but also directly on an Exchange email database. Database-level scans can be scheduled to run at night when the load is lower on the server; real-time scans can perform virus checking in real time before any message is written to the database.

The final checkpoint for any multilayered virus protection is on the workstation. The file system and the email system can be protected by the same antivirus product. Messages can be scanned before a user is able to open the message or before a message is sent.

Protecting email communications and message integrity puts a large load on administrators. Threats are best dealt with using a multilayered approach from the client to the server to the gateway. When each step along the way is protected against malicious attacks, the global result is a secure, well-balanced email system.

The Edge Transport Server’s Role in Antivirus and Antispam Protection

In Exchange Server 2007, the introduction of the Edge Transport server role was brought about by the increased need to protect organizations from unwanted message traffic. The Edge Transport server is designed to provide improved antivirus and antispam protection for the Exchange environment. This server role also applies policies to messages in transport between organizations. The Edge Transport server role is deployed outside the Active Directory forest in the perimeter network and can be deployed as a smart host and SMTP relay server for an existing Exchange Server 2013 organization.

Actually, you can add an Edge Transport server to any existing Exchange environment without making any other organizational changes or upgrading the internal Exchange servers. There are no preparation steps needed in Active Directory to install the Edge Transport server.

For additional protection, Microsoft Forefront for Exchange Server 2010 can be used on the Edge Transport 2010 role to provide a more robust anti-malware defense.

SMTP Server Scalability and Load Balancing

In a larger environment, administrators might set up more than one SMTP server for inbound and/or outbound mail processing. Windows Server 2012 and Exchange Server 2013 provide a very flexible platform to scale and balance the load of SMTP mail services. DNS and Network Load Balancing (NLB) are key components for these tasks.

Administrators should not forget about hardware failover and scalability. Multinetwork interface cards are highly recommended. Two network cards can be teamed together for higher throughput, can be used in failover configuration, or can be load-balanced by using one network card for front-end communication and another for back-end services, such as backup.

Network design can also incorporate fault tolerance by creating redundant network routes and by using technologies that can group devices together for the purpose of load balancing and delivery failover. Load balancing is the process where requests can be spread across multiple devices to keep individual service load at an acceptable level.

Using NLB, Exchange Server SMTP processes can be handed off to a group of servers for processing, or incoming traffic can be handled by a group of servers before it gets routed to an Exchange server. The following example outlines a possible configuration for using NLB in conjunction with Exchange.

DNS, in this example, has been set up to point to the name of the NLB cluster IP address. Externally, the DNS MX record points to a single mail relay gateway for companyabc.com. Exchange Server uses smart host configuration to send all SMTP messages to the NLB cluster. The NLB cluster is configured in balanced mode where the servers share equal load. Only port 25 traffic is allowed on the cluster servers. This configuration would off-load SMTP mail processing from the Exchange servers because all they have to do is to pass the message along to the cluster for delivery. They do not need to contact any outside SMTP gateway to transfer the message. This configuration allows scalability because when the load increases, administrators can add more SMTP gateways to the cluster. This setup also addresses load balancing because the NLB cluster is smart enough to notice whether one of the cluster nodes has failed or is down for maintenance. An additional ramification of this configuration is that message tracking will not work beyond the Exchange servers.

A less used but possible configuration for SMTP mail load balancing uses DNS to distribute the load between multiple SMTP servers. This configuration, known as DNS round-robin, does not provide as robust a message routing environment as the NLB solution.

Configuring DNS to Support Exchange Servers

Because DNS is already required and integrated with Active Directory before Exchange Server is installed, most companies already have a robust DNS environment in place. Exchange by itself accesses DNS servers to find resources on the local network, such as global catalog servers and domain controllers. It also uses DNS to search for MX records of other domains.