Exchange 2013 Mail Flow

Part 1
Introduction
As you probably already know, Microsoft reduced the number of server roles in Exchange
2013 to just two in order to “increase simplicity of scale, hardware utilization and failure
isolation”:
 Mailbox Server role: which includes the Client Access protocols, Hub Transport
service, Mailbox databases and Unified Messaging, and also handles all activity for a
given mailbox;
 Client Access Server role: which provides authentication, redirection and proxy
services (it doesn't do any data rendering). The Client Access server is a thin and
stateless server and there is never anything queued or stored on it. The Client Access
server offers the usual client access protocols: HTTP (Outlook Anywhere, ActiveSync
and Web Services), POP, IMAP and SMTP, but not MAPI (all MAPI connections are
encapsulated using RPC-over-HTTPS)!
Note that the Edge Server is not present in Exchange 2013. This is because its services are
now provided by the Client Access Server (although you can still use an Exchange 2007/2010
Edge server as a gateway with your Exchange 2013 environment).
In this new architecture, the Client Access server and the Mailbox server roles are not as
dependent on one another as in previous version of Exchange because all processing and
activity for mailboxes occurs on the Mailbox server where the active database copy of a
mailbox resides. All data rendering and data transformation is performed local to the active
database copy, eliminating concerns of version compatibility between the Client Access
server and the Mailbox server.
This architecture change brings many other benefits, but in this article we are going to focus
solely on how it affects the flow of e-mail messages.

Transport Pipeline
Mail flow takes place through the Transport Pipeline which is a collection of services,
connections, components and queues that work together to route messages. With Exchange
2013, the transport pipeline is now made up of 3 different services:
 Front End Transport service, which runs on all Client Access servers, acts as a
stateless proxy for all inbound and outbound external SMTP traffic. This service does
not inspect message content but it can perform basic e-mail filtering based on
connections, domains, senders and recipients. It only communicates with the
Transport service on a Mailbox server and does not queue any messages locally;
  Transport service runs on all Mailbox servers and is almost identical to the Hub
Transport server role in previous versions of Exchange. It handles all SMTP mail flow
for the organization, performs message categorization and content inspection.
Unlike previous versions of Exchange, the Transport service never communicates
directly with a mailbox database, which is now handled by the Mailbox Transport
service. The Transport service routes messages between the Mailbox Transport
service, the Transport service and the Front End Transport service. This service does
queue messages locally.
 Mailbox Transport service also runs on all Mailbox servers and is made of two
separate services:
o The Mailbox Transport Delivery service receives SMTP messages from the
Transport service and connects to the mailbox database using an Exchange
Remote Procedure Call [RPC] to deliver the message.
o The Mailbox Transport Submission service connects to the mailbox database
using RPC to retrieve messages and submits them over SMTP to the
Transport service. The Mailbox Transport service also does not queue any
messages locally.
As Exchange 2013 no longer has an Edge Server role, e-mail messages from the Internet are
received and sent to Exchange using a third-party e-mail gateway, an Exchange 2007/2010
Edge server or through the Exchange 2013 Client Access server as Microsoft intends it to be.
In this last scenario, these e-mails enter the Exchange organization through a Receive
connector in the Front End Transport service and are then routed to the Transport service
on a Mailbox server.
While Exchange 2007/2010 Hub Transport servers were not configured out of the box to
accept e-mails from the internet, the new Client Access server comes with a Receive
Connector named “Default Frontend <server_name>” that is already configured to allow
“Anonymous Users” to connect to it:
Figure 1.1: Default Frontend Receive Connector
As to messages within the organization, they enter the Transport service on a Mailbox
server through one of four ways:
1. Through a Receive connector;
2. Through the Pickup and Replay directories;
3. Through the Mailbox Transport service;
4. Through agent submission.
Before putting everything together in a nice diagram, we just need to have a deeper look
into the Transport Service, which consists of the following components and processes:
 SMTP Receive: when e-mails are received by the Transport service, content
inspection is performed, transport rules are applied and anti-spam/anti-malware
inspection is performed (if enabled). If the e-mail is not rejected after passing
through SMTP Receive, it is put in the Submission queue;
 Submission is the process of putting e-mails into the actual Submission queue and
can happen in three different ways:
o Through an SMTP Receive connector;
o Through the Pickup and Replay directories which now reside on Mailbox
servers;
o Through an agent.
  Categorizer picks up a message at a time from the Submission queue and completes
the following steps:
o Recipient Resolution: the recipient's e-mail address is resolved to determine
whether the recipient has a mailbox in the Exchange organization or an
external e-mail address;
o Routing Resolution: after information regarding the recipient is resolved, the
ultimate destination for the message, the route to that destination and the
next hop are determined;
o Content Conversion: occurs so that the message is sent in a format that is
readable by the recipient by transforming e-mail messages from one format
to another format such as MAPI to MIME, UUENCODE to base64 encoded, or
for appropriate rendering that is specific to an e-mail client like HTML, rich
text format or plain text.
Additionally, mail flow rules are applied. After messages have been categorized, they are
put into a delivery queue that is based on the destination of the message (mailbox database,
Database Availability Group [DAG], Active Directory [AD] site, AD forest or external domain).
 SMTP Send: depending on the location of the message recipient, the message is
routed to:
o the Mailbox Transport service on the same Mailbox server;
o the Mailbox Transport service on a different Mailbox server that is part of the
same DAG;
o the Transport service on a Mailbox server in a different DAG, AD site, or AD
forest; or
o the Front End Transport service on a Client Access server for delivery to the
Internet.
As a picture is worth a thousand words, the following diagram (taken from TechNet) puts
everything we talked so far together and shows the relationships between the components
in the Exchange 2013 transport pipeline:
Figure 1.2: Exchange 2013 Transport Pipeline
In the next article, we will analyze the flow of an e-mail message and trace its path using the
diagram above!
Transport Agents
Twice now we mentioned Transport Agents without going into detail on what these are.
Transport Agents, as with previous versions of Exchange, allows administrators to install
custom software on an Exchange server in order to process e-mail messages that pass
through the transport pipeline. This software might be created by Microsoft, by third-party
vendors or by your organization.
Once more Exchange provides extensibility through the Exchange Transport Agents SDK.
This SDK is .NET based and allows third-parties to implement the
classes SmtpReceiveAgent, RoutingAgent and DeliveryAgent. When compiled against
libraries in the SDK, the resulting assemblies are registered with Exchange 2013, which loads
the agents and invokes their event handlers during specific stages of SMTP sessions or
message processing.
The Transport service fully supports all the predefined classes in the SDK, which means that
any transport agents written for Hub Transport server role in Exchange 2010 should work in
the Transport service in Exchange 2013. On the other hand, the Front End Transport service
only supports the SmtpReceiveAgent class while the Mailbox Transport service does not
support the SDK at all, so you can't use any agents in the Mailbox Transport service.
Exchange 2013 comes with built-in transport agents, with most of them invisible and
unmanageable by the transport agent cmdlets. However, a few of the built-in transport
agents in the Transport service are visible using the Get-TransportAgent cmdlet:

Figure 1.3: Exchange 2013 Transport Agents

Conclusion
So far we looked at how the new design of Exchange 2013 changed the Mail Flow in general.
In the second article of this series, we will see how message routing happens in the new
version of Exchange and in the third and final article, we will finish by exploring how each
service routes messages and we will track an e-mail to see its exact flow across all the
different services.
Part 2
Message Routing
The main purpose of the Transport service (present on all Mailbox servers) is to route
messages to their final destination. Routing decisions are made during message
categorization, with the categorizer being the component of the Transport service that
processes all incoming messages and determines what to do with them based on their
destination.
Because in Exchange 2013 all Mailbox servers host the Transport service, the routing
process is now fully aware of Database Availability Groups [DAGs] and uses DAG
membership as a routing boundary. This way, if a Mailbox server belongs to a DAG, the
primary mechanism for routing messages is closely aligned with the DAG because if the DAG
spans multiple Active Directory [AD] sites, using the AD site as the primary routing boundary
is inefficient.
Obviously, Exchange 2013 continues to use AD site membership as a routing boundary for
Mailbox servers that do not belong to a DAG and for routing with previous versions of
Exchange.
Due to the new architecture design of Exchange 2013, there are also other changes to the
routing process. The diagram below helps to understand some of these changes:
Figure 2.1: Exchange 2013 Transport Pipeline
 The Transport service on a Mailbox server never communicates directly with a
mailbox database. Instead, it communicates with the Mailbox Transport service
which then communicates with the mailbox database on the local Mailbox server.
When the Mailbox server is a member of a DAG, only the Mailbox Transport service
on the Mailbox server that holds the active copy of the mailbox database accepts the
message for the destination recipient;
 Remote Procedure Calls [RPCs] are only used by the Mailbox Transport service when
sending messages to or receiving messages from the local mailbox database, they
are never used for cross-server communication. Instead, the Mailbox Transport
service and the Transport service on different Mailbox servers communicate using
SMTP;
 Instead of using one queue for all destinations in a remote AD site, Exchange 2013
queues messages for specific destinations within the AD site, such as individual Send
connectors;
 Linked connectors (a Receive connector linked to a Send connector) have been
deprecated.
As explained previously, when an e-mail is received by the Transport service it has to be
categorized. First, recipient resolution is performed after which the ultimate destination can
be determined. Now comes routing, where Exchange determines how to best reach that
destination. As with previous versions of Exchange, the ultimate destination for a message is
called a Routing Destination (note that the following also existed before):
 Mailbox database: any recipient with a mailbox in the Exchange organization has
this as the routing destination. Because public folders are now also a type of
mailbox, routing messages to public folder recipients is the same as routing
messages to mailbox recipients;
 Connector: this can be a Send connector for SMTP messages, a Delivery Agent
connector or a Foreign connector used as a routing destination for non-SMTP
messages;
 Distribution Group expansion server: when a distribution group has a designated
expansion server (no longer a Transport server but a Mailbox server in Exchange
2013) that is responsible for expanding the membership list of the group, this is the
routing destination.
What is different in Exchange 2013 is the notion of Delivery Groups. Each routing
destination has a collection of one or more transport servers (either an Exchange 2013
Mailbox server or an Exchange 2010 Hub Transport server) that are responsible for
delivering messages to that routing destination, known as a delivery group.
If the routing destination is a mailbox database, the transport servers in the delivery group
are the same version of Exchange as the mailbox database. If the routing destination is a
connector or a distribution group expansion server, the delivery group may contain a
mixture of Exchange 2013 Mailbox servers and Exchange 2010 Hub Transport servers.
There are 5 types of delivery groups in Exchange 2013:
 Routable DAG: a collection of mailbox servers that belong to a DAG. The mailbox
databases in the DAG are the routing destinations that are serviced by this delivery
group. After the message arrives at the Transport service on a Mailbox server that
belongs to the DAG, the Transport service routes the message to the Mailbox
Transport service on the Mailbox server in the DAG that currently holds the active
copy of the destination mailbox database. The Mailbox Transport service on the
destination Mailbox server then delivers the message to the local mailbox database.
Although a DAG may contain Mailbox servers located in different AD sites, the DAG is
the delivery group boundary;
 Mailbox delivery group: a collection of Exchange servers of the same version located
in one AD site. The AD site is the delivery group boundary. The mailbox databases
located on Exchange 2010 Mailbox servers are serviced by the Exchange 2010 Hub
Transport servers located in the AD site; the mailbox databases located on Exchange
2013 Mailbox servers in an AD site that don't belong to a DAG are serviced by the
Transport service on Exchange 2013 Mailbox servers in the AD site.
 How the message is delivered to the mailbox database depends on version of
Exchange:
o Exchange 2013: after the message arrives at the destination Mailbox server in
the destination AD site, the Transport service uses SMTP to transfer the
message to the Mailbox Transport service which then delivers the message to
the local mailbox database using RPC;
o Exchange 2010: after the message arrives at a random Exchange 2010 Hub
Transport server in the destination AD site, the store driver on the Transport
server uses RPC to write the message to the mailbox database;
 Connector source servers: a mixed collection of Exchange 2010 Hub Transport
servers or Exchange 2013 Mailbox servers scoped as the source server for a Send
connector, a Delivery Agent connector or a Foreign connector. The connector is the
routing destination that is serviced by this routing group. When a connector is
scoped to a specific server, only that server is allowed to route messages to the
destination defined by the connector. This delivery group may contain Exchange
2010 Hub Transport servers or Exchange 2013 Mailbox servers located in different
AD sites;
 AD site: in some circumstances, an AD site isn't the ultimate destination of a
message, but the message must pass through an Exchange 2010 Hub Transport
server or Exchange 2013 Mailbox server in that site before it reaches its destination;
 Server list: a collection of one or more Exchange 2010 Hub Transport servers or
Exchange 2013 Mailbox servers that are configured as distribution group expansion
servers. The distribution group expansion server is the routing destination serviced
by this delivery group.
Delivery group membership isn't mutually exclusive. For example, an Exchange 2013
Mailbox server that is a member of a DAG can also be the source server of a scoped Send
connector. This Mailbox server would belong to the routable DAG delivery group for
mailbox databases in the DAG, and also a connector source server delivery group for the
scoped Send connector.
When a message is to be delivered to a remote delivery group, a routing path must be
determined for the message. Exchange 2013 uses the same logic as Exchange 2010 to select
the routing path for a message:
1. Calculate the least-cost routing path by adding the cost of the IP site links that must
be traversed to reach the destination. If the destination is a connector, the cost
assigned to the address space is added to the cost to reach the selected connector. If
multiple routing paths are possible, the routing path with the lowest aggregate cost
is used;
2. If more than one routing path has the same cost, the routing path with the least
number of hops is used;
 3. If more than one routing path is still available, the name assigned to the AD sites
before the destination is considered. The routing path where the AD site nearest the
destination is lowest in alphanumeric order is used. If the site nearest the
destination is the same for all routing paths being evaluated, an earlier site name is
considered.
Conclusion
This article was all about theory and more text than what I would prefer... However, it is
necessary to understand how Exchange 2013 uses Routing Destination and Delivery Groups
to route messages to their correct destination.
The third and final article of this series will be more interesting (I promise!) as we will be
looking at how the Front End Transport and Mailbox Transport services route messages; we
will trace an e-mail message through all the steps and processes discussed so far and finally
we will have a look at Queues.

Part 3
Front End Transport service Routing
As explained in the previous articles of this series, this service runs on all Client Access
servers and acts as a stateless proxy for all inbound and outbound external SMTP traffic.
For outgoing messages, the Transport service on Mailbox servers uses Send connectors to
communicate with the Front End Transport service. Specifically, outgoing messages are
proxied through the Front End Transport service when
the FrontEndProxyEnabled parameter on an applicable Send connector is set to $True in the
Exchange Management Shell, or the Proxy through client access server option is selected in
the Exchange Administration Center:
Figure 3.1: Exchange 2013 Send Connector Proxy Enabled
For incoming messages, the Front End Transport service looks for a single healthy Transport
service on a Mailbox server to receive the message. Failure to do so results in the e-mail
service being perceived as unavailable by the external senders as this service does not queue
messages.
The Front End Transport service resolves message recipients to mailbox databases (unless
the recipient is a distribution group, for example) by querying AD. Then, for each mailbox
database, it looks up the delivery group (which can be a Routable DAG, a Mailbox delivery
group or an AD site) and the associated routing information.
Based on the number and type of recipients, the Front End Transport service performs one
of the following actions:
 For messages with a single mailbox recipient, select a Mailbox server in the target
delivery group and give preference to the Mailbox server based on the proximity of
the AD site;
 For messages with multiple mailbox recipients, use the first 20 recipients to select a
Mailbox server in the closest delivery group, based on the proximity of the AD site.
Because message bifurcation doesn't occur in Front-End Transport, only one Mailbox
server is selected;
 If the message has no mailbox recipients, select a random Mailbox server in the local
AD site.
Mailbox Transport service Routing
For incoming messages, the Mailbox Transport Delivery service receives SMTP messages
from the Transport service and connects to the local mailbox database using Remote
Procedure Call [RPC] to deliver the message. For outgoing messages, the Mailbox Transport
Submission service connects to the local mailbox database using RPC to retrieve messages
and submits the messages over SMTP to the Transport service. The Mailbox Transport
service is stateless, and does not queue any messages locally.
When a user sends an e-mail, the Mailbox Transport Submission service resolves the
message recipients to mailbox databases (unless the recipient is a distribution group, for
example) by querying AD. Then, for each mailbox database, it looks up the delivery group
(which can be a Routable DAG, a Mailbox delivery group or an AD site) and the associated
routing information.
Based on the number and type of recipients, the Mailbox Transport Submission service
performs one of the following actions:
 For messages with a single mailbox recipient, select a Mailbox server in the target
delivery group and give preference to the Mailbox server based on the proximity of
the AD site;
 For messages with multiple mailbox recipients, use the first 20 recipients to select a
Mailbox server in the closest delivery group, based on the proximity of the AD site;
 If the message has no mailbox recipients, select a Mailbox server in the local delivery
group.
When the Mailbox Transport Delivery service receives a message from the Transport
service, it accepts or rejects the message for delivery to a local mailbox database as it can
only deliver the message if the recipient resides in an active copy of a local mailbox
database. If not, then the Mailbox Transport Delivery service can't deliver the message and
must provide a non-delivery response to the Transport service. This might happen if, for
example, the active copy of the mailbox database recently moved to another server and the
Transport service incorrectly transmitted a message to a Mailbox server that now holds an
inactive copy of the mailbox database. In this case, the non-delivery response that the
Mailbox Transport Delivery service returns to the Transport service is one of the following:
 Retry delivery;
 Generate an NDR;
 Reroute the message.
Tracking Message Flow
Similar to Exchange 2010, we can use Delivery Reports to track messages sent to or from
users in an Exchange 2013 organization (sent using Outlook or Outlook Web App only and
not POP or IMAP!) and check their delivery status.
However, to better understand the flow of messages in Exchange 2013, we need to turn to
our good old friends Message Tracking Logs. There is only one difference in this new version
of Exchange: because we no longer have a dedicated Hub Transport Server role, the Get-
TransportServer cmdlet is getting replaced by two new cmdlets:
  the Get-TransportService cmdlet which returns transport configuration information
for the Transport service on Mailbox servers or for Edge Transport servers;
 the Get-MailboxTransportService cmdlet which returns transport configuration
information for the Mailbox Transport service on Mailbox servers
So, let’s look at the flow of an e-mail sent from the internal user Nuno (whose mailbox is on
server MBX1) to the internal user Mota (whose mailbox is on server MBX2). Using the
Exchange Management Shell, we run the following cmdlet to get information regarding this
particular e-mail (in this case it doesn’t matter which of the above cmdlets we run as all we
need are the names of the Mailbox servers possibly involved in the mail flow, which both
cmdlets return):
Get-TransportService | Get-MessageTrackingLog –ResultSize Unlimited –Start “11/11/2012”
–Sender [email protected] -Recipients [email protected] –MessageSubject
“Testing Mail Flow” | Sort TimeStamp | Select ClientHostname, ServerHostname,
ConnectorId, Source, EventId
Analyzing the output of this search, we obtain the following which we can map into the
following diagram:

Figure 3.2: Exchange 2013 Transport Pipeline

1. The Mailbox Transport Submission service uses the Store Driver to connect to the mailbox
database using RPC and retrieves the e-mail:
ClientHostname : MBX1.letsexchange.com
ServerHostname : MBX1
ConnectorId :
Source : STOREDRIVER
EventId : RECEIVE
2. After resolving the message recipient to its mailbox database, the Mailbox Transport
Submission service looks up the delivery group, a Mailbox delivery group in this case. So the
Store Driver passes the e-mail to the Hub Selector for this to be sent over SMTP to the
appropriate server.
In this case the e-mail is not passed to the Transport service. You can test this by stopping
the Microsoft Exchange Transport service on your source mailbox server and you will see
the e-mail will still be sent.
ClientHostname : MBX1
ServerHostname : MBX2.letsexchange.com
ConnectorId :
Source : STOREDRIVER
EventId : SUBMIT
3. The Transport service on the MBX2 mailbox server receives the e-mail sent over SMTP
from the Mailbox Transport Submission service of server MBX1 using its default receive
connector. Content inspection is performed, transport rules are applied and anti-spam/anti-
malware inspection is performed (if enabled). As the e-mail is not rejected by any of these,
it is put in the Submission queue.
ClientHostname : MBX1.letsexchange.com
ServerHostname : MBX2
ConnectorId : MBX2\Default MBX2
Source : SMTP
EventId : RECEIVE
4. The Categorizer picks up the e-mail from the Submission Queue, processes it and puts
into a delivery queue based on the destination of the message (in this case, a mailbox
database):
ClientHostname : MBX2
ServerHostname :
ConnectorId :
Source : AGENT
EventId : AGENTINFO
5. The e-mail is sent over SMTP from the Transport service to the Mailbox Transport
Delivery service:
ClientHostname : MBX2
ServerHostname : MBX2.letsexchange.com
ConnectorId : Intra-Organization SMTP Send Connector
Source : SMTP
EventId : SEND
6. The e-mail is received over SMTP by the Mailbox Transport Delivery service from the
Transport service:
ClientHostname : MBX2.letsexchange.com
ServerHostname : MBX2
ConnectorId : MBX2\Default Mailbox Delivery MBX2
Source : SMTP
EventId : RECEIVE
7. The Mailbox Transport Delivery service uses the Store Driver to connect to the mailbox
database using RPC and writes the e-mail to the mailbox database:
ClientHostname : MBX2
ServerHostname : MBX2
ConnectorId :
Source : STOREDRIVER
EventId : DELIVER
Simplifying the diagram in Figure 3.2 for this example, we end up with:
Figure 3.3: Mail Flow Trace
Queues
As we saw several times now, out of the three services involved in Mail Flow, only the
Transport service provides local queuing. The Front End Transport service and the Mailbox
Transport service, do not provide local queuing.
This is different from earlier versions of Exchange as we do have queues in the Exchange
2007/2010 Edge servers. I do not know the reason behind this decision but if you use a
Client Access server as the e-mail gateway to your organization, be very careful! If there are
no mailbox servers available to receive e-mails (either the one where the recipient’s mailbox
resides or any other), a failure notification will be issued to the sender’s server
immediately... Although most servers will keep trying to deliver the e-mail to your
organization for 24, 48 or even 72h, this might lead to message failures especially with
applications or other devices other than mail servers.
Therefore, if you have a custom Receive Connector for applications, printers, or other
devices, with Exchange 2013 you might want to create it on your Mailbox servers and not on
the Client Access servers for this exact reason.
Conclusion
In this article series we explored Mail Flow and the changes introduced to the Transport
Pipeline by the new architecture of Exchange 2013.
In a future article, we will look at the High Availability improvements to the Transport
Pipeline introduced in Exchange 2013.