By now you get the point that Hub Transport servers
are essential to the Exchange organization. You understand that without
transport servers, messages from any user to any other user in your
organization, regardless of where their mailboxes are located, will
simply not be delivered. Well, where are those messages stored on a Hub
Transport server? The trusted transport database of course! (Actually,
it's the same database file format and database engine as the one used
to store all mailboxes on your Mailbox servers.) The Hub Transport
servers store all messages in transit in a single database file, often
referred to as the queue database. When administrators open the Queue
Viewer, they are no longer looking at a flat NTFS file structure (such
as the one in Exchange Server 2003), but they are making a query into a
database.
You can imagine that for organizations that maintain
thousands of mailboxes, and therefore have very busy Hub Transport
servers, the queue database can become very large. Although in a
healthy transport scenario the messages stored in the database don't
reside there very long, the database likely contains sensitive messages
where message delivery must be guaranteed.
Backing up the queue database does not seem
practical, considering that the data it contains is highly dynamic and
will be quite irrelevant within a few minutes of its backed-up state.
But in a case of data loss or complete server failure, it is possible,
and likely, that the entire queue database may be lost, including all
those sensitive messages it contained.
The solution to this problem is shadow redundancy (or also known as shadow transport).
1. Understanding Shadow Redundancy
Yes, Shadow redundancy is a new technology
introduced in Exchange Server 2010, which protects organizations in the
event of a Transport server loss. Shadow redundancy is used by Exchange
Server Hub Transport servers and Edge Transport servers. The main
principle behind shadow redundancy is maintaining a copy of a message
on the previous hop until the server verifies that it has successfully
delivered it to all hops. In a shadow redundancy scenario, the
following occurs:
The Hub Transport server delivers a message to another Hub Transport server or an Edge Transport server.
An SMTP session is opened.
The receiving server advertises support for shadow redundancy.
The sending server instructs the receiving server to track the discard status.
The message is delivered to the receiving server.
The
receiving server acknowledges receipt of the message and records the
sending server name for sending discard information for the message.
The sending server moves the message to a shadow queue.
The message is delivered to its next hop (in this case, a third-party server).
The third-party mail server acknowledges receipt of the message. The delivery is marked as complete.
The sending server queries the receiving server for the status of the message.
The
receiving server checks the local discard status, sends back the list
of messages that have been delivered, and removes the discard
information.
The sending server deletes the list of messages from its shadow queue.
In a case of delivery failure, the sending server (in this case a Hub Transport server) would resubmit the message.
Several caveats exist. First, all servers that
participate in the process must support shadow redundancy. At this
point, that means all Exchange Server 2010 Hub Transport and Edge
Transport servers only. If your organization contains Hub Transport
servers from previous versions of Exchange Server, they can coexist in
the same organization. However, in the event that an Exchange Server
2010 Edge Transport server communicates with an Exchange Server 2007
Hub Transport server, the advertisement of shadow redundancy (XSHADOW)
will fall on deaf ears. The message will not be moved to a shadow queue
and a delivery attempt occurs immediately.
There are not too many actions that are configurable
for shadow redundancy, though you can disable shadow redundancy (it's
enabled by default) and you can modify the expiry timestamp on messages
stored in the shadow database (the default is two days, since that is
the default amount of time that a message is queued for delivery). For
example, the following EMS command will modify the expiry value for
messages in the queue to 1 hour.
Set-TransportConfig -ShadowMessageAutoDiscardInterval 01:00:00
We want to briefly mention that Exchange Server 2010
has provisions in place when communicating with servers other than
Exchange Server 2010 Transport servers. The delayed acknowledgment
process provides a "best effort" attempt at minimizing the possibility
of lost messages originating from the Internet. Delayed acknowledgment
is a receive connector setting that is designed for submitting SMTP
servers that are not configured for shadow redundancy. Delayed
acknowledgement delays the
acknowledgment that a Hub Transport server provides to a sending
server, letting it know that the message was accepted. The benefit here
is that if the server fails and it had not yet delivered the message
internally (and therefore benefited from shadow redundancy), the
sending server would resubmit the
message. As you can imagine, Exchange servers relinquish most of the
control in this scenario. The responsibility for redelivering the
message to the Exchange organization, when no acknowledgment is sent
out, lies solely with the submitting server on the Internet.
2. Transport Dumpster
With a similar purpose as shadow redundancy,
Exchange Server 2007 introduced the Transport Dumpster feature. The
Transport Dumpster is a special cache maintained on all Hub Transport
servers. The purpose of the Transport Dumpster is to maintain a copy of
the recently delivered messages. This feature is used in conjunction
with the database replication technologies introduced in Exchange
Server 2010 the database availability group (DAG).
In the event that a message was delivered to a
Mailbox server that is a member of a DAG and that server fails, the
Mailbox server that will become active will then contact the Hub
Transport server to verify which messages in its Transport Dumpster
have already been replicated to members of the DAG.
During normal mailbox database copy replication,
each Hub Transport server obtains the timestamp of the last transaction
that is considered to be consistent across all Mailbox database copies.
This value tells the Hub Transport server exactly which transactions
have been properly replicated to and applied to Mailbox databases
within the DAG. (The value that is queried for is the value for the
LastLogInspected time.) At that point, the Transport Dumpster knows
that it is now safe to delete any messages that have been delivered
prior to this timestamp, maintaining only messages that have not been
replicated to all Mailbox database copies. In the event that
replication status cannot be obtained for one or more mailbox database
copies, the Transport Dumpster does not truncate (delete messages)
until the replication status for all copies is successfully retrieved.
The Transport Dumpster size, by default, is set to
18 MB per Mailbox database. An administrator can modify the dumpster
size by using the Set-TransportConfig cmdlet.
