BizTalk 2009 : Implementing a Scatter or Gather Pattern

The Scatter/Gather pattern is a method for broadcasting and processing messages in parallel. The "scatter" portion distributes a series messages all at the same time, and then an aggregator "gathers" the messages back into the main response before continuing. Figure 1 shows an example.

Figure 1. Example of data flow within a Scatter/Gather pattern

Figure 1 shows a Get Data Request being received. An orchestration "scatters" the request to multiple solicit-response send ports in parallel. The orchestration waits for all responses to be received and aggregates them back into a single Get Data Response message, which is sent back to the requestor.

The key to having BizTalk scatter the requests to multiple backend systems is its ability for send ports to be bound by subscription filters. This allows an orchestration to send one message to the Messagebox via direct binding and have one or more send ports process the request in parallel. With BizTalk's ability to scale across multiple hosts, this pattern can also provide significant performance benefits.

This type of pattern is especially useful for executing identical requests to multiple backend systems. For example, say a customer calls into a call center to retrieve information about all their loans at a bank. The following scenario shows how this pattern could be leveraged:

Figure 6-28 shows a collapsed view of the orchestration that implements the pattern to support this scenario.

The orchestration shown in Figure 2 has the following major sections:

• A request shape to receive the request from the calling IVR application

• A Scatter scope that submits a direct "GetLoanInfo" message to the Messagebox where three send ports, one for each backend system, are subscribed to this message schema

• A construct message shape to create the initial response message

• A Gather scope that receives the responses from the backend systems and aggregates them into the response template message

• A send shape to send the aggregated content back to the IVR system

Figure 2. Example collapsed Scatter/Gather orchestration

Figure 3 shows the expanded Scatter scope.

Figure 3. Scatter scope

The Scatter scope is quite simple. It creates a new backend message type and sends the message to a send port via direct binding to the Messagebox. Then, based upon send port subscriptions, there are many recipients of the message, therefore causing multiple messages to be processed. In addition, each send port could have a map associated with it to further transform the message to a schema format required by the backend application.

Once the message has been sent, an empty response message is created, shown in Figure 4, that will ultimately be used to hold all the aggregated result data.

Figure 4. Response message creation

Now that the messages have been sent and the response message is ready, you can start gathering the responses and aggregating them together. Figure 5 shows the expanded Gather scope.

Figure 5. Gather scope

The Gather scope performs the following functions:

• Loops until all messages are received.

• Listens for a correlated message to be received.

• Appends the received message to the response message.

• If all messages aren't received by the time the delay time occurs, the listen shape is exited.

Once the Gather scope has completed, a send shape is used to send the aggregated messages back to the calling application to complete the pattern.

As you can see from the sample orchestration, the pattern's concept fits seamlessly with one of BizTalk's core features: content-based routing. By using content-based routing, the orchestration's implementation of the pattern is kept quite simple and straightforward.