BizTalk 2010 Recipes : Creating Envelopes to Split Inbound Data

3/26/2011 2:48:33 PM

1. Problem

You are receiving a message that contains multiple records in a batch fashion. In order to import this information into the appropriate systems, each record must be handled individually, as opposed to processing them all in a single batch.

2. Solution

Envelopes allow you to define a container schema that wraps a number of child schemas or subschemas. By defining which child records it contains, the envelope allows BizTalk Server to access the subrecords individually (a process commonly known as debatching) and process them as distinct messages. For this solution, it is assumed that the schema defining the child record has already been created . In this case, our child record schema is Order – the incoming document contains one or more "Orders," and we want to split these into individual orders using an envelope.

To create an envelope schema, follow these steps:

Open the project that contains the child record schema. In this case, "Order" would be the child record.
Right-click the project, and select Add => New Item to open the Add New Item dialog box.
Select Schema Files from the BizTalk Project Items category and Schema from the list of templates. Enter a descriptive name for your new schema and then click Add. In this case, OrderEnvelope will be used.
Right-click the Root node of the new schema, and change it to the root name of the incoming schema containing the batch of records (OrderEnvelope in this example).
Click the Schema node (directly above the newly renamed root node), and change the Envelope property of the schema to Yes (in the Properties window), as shown in Figure 1.
Right-click the newly renamed root node, and select Insert Schema Node => Child Record, Child Field Attribute, or Child Field Element, to add the appropriate envelope elements. In this example, a child field element named BatchID and a child record named Orders, which will contain the child records, are added.
Right-click the record that will contain the child records (Orders in this example) and select Insert Schema Node => Child Record to add a container for the child records. In this example, a child record named Order is added.

Figure 1. Configuring the schema to be an envelope
Right-click the container child record (Order in this example), and select Insert Schema Node Any Element to add a placeholder for the child elements. The resulting envelope schema has the structure shown in Figure 2.

Figure 2. Envelope schema structure
Click the root node (OrderEnvelope in this example), and click the Body XPath property of the schema (in the Properties window), which will open the Body XPath dialog box.
Navigate through the schema structure displayed in the tree view, and select the container record for the child records you wish to process individually (the Orders record in this example), as shown in Figure 3. Then click OK.

Figure 3. Specifying a child record
Build and deploy the solution.
The next step is to create the receive port, receive location, and send port. The basics are as follows:
1. Create a Receive Port and an associated File Receive Location. Set the Receive pipeline property as XMLReceive. See Figure 4.
  
  Figure 4. Configuring the Receive Port for Splitting
2. Create a File Send Port. Set the filter to subscribe to the Receive Port (BTS.ReceivePortName) and the Send Pipeline property to XMLTransmit. See Figure 5.
  
  Figure 5. Configuring the Send Port for Splitting
3. Enlist the Send Port and enable the Receive Location.
Drop an instance of the Orders XML document on the File Receive Location. The split of the inbound Orders document will occur automatically in the pipelines.

NOTE

You can create a sample instance to work from by right-clicking the OrderEnvelope schema and selecting Generate Instance. Paste this into Notepad or some other text editor. Next, right-click the Order schema, and select Generate Instance. Paste this into the correct node in the Orders document in the text editor. Save this manually created document and use it as a test input document.

3. How It Works

Envelopes provide the means to group multiple messages into one XML document. In the preceding example, the envelope was used to disassemble individual order documents from a single enveloped document containing a batch of orders. Once the envelope and document schemas (OrderEnvelope and Order in the example) have been deployed, BizTalk Server has the ability to leverage the two in message processing. The following XML represents one possible instance of the envelope schema:

<ns0:OrderEnvelope xmlns:ns0="http://EnvelopeSolution.OrderEnvelope">
  <BatchID>BatchID_0</BatchID>
  <Orders>
    <ns0:Order xmlns:ns0="http://EnvelopeSolution.Order">
      <OrderID>1</OrderID>
      <OrderAmount>1.00</OrderAmount>
    </ns0:Order>
    <ns0:Order xmlns:ns0="http://EnvelopeSolution.Order">
      <OrderID>2</OrderID>
      <OrderAmount>2.00</OrderAmount>
    </ns0:Order>
  </Orders>
</ns0:OrderEnvelope>

When passed through an XML disassembler pipeline component in BizTalk Server, the preceding XML message will produce the following two XML documents, which can be processed individually:

<ns0:Order xmlns:ns0="http://EnvelopeSolution.Order">
  <OrderID>1</OrderID>
  <OrderAmount>1.00</OrderAmount>
</ns0:Order>

<ns0:Order xmlns:ns0="http://EnvelopeSolution.Order">
  <OrderID>2</OrderID>
  <OrderAmount>2.00</OrderAmount>
</ns0:Order>

As these order documents are split up into separate messages within BizTalk Server, they can undergo different forms of business logic or be delivered to a different destination, depending on their properties.

The key data element that allows envelope processing in BizTalk server is the Body XPath property. This data element acts as a pointer to the container record that encapsulates or surrounds the child records. On the OrderEnvelope root node, the Body XPath property was set to the Orders record, which contains the individual Order records. In envelope schemas containing a single root node, the Body XPath property must be set to that root node.

For envelope schemas with multiple root nodes, the following apply:

If the Root Reference property is not set on the envelope schema, the Body XPath property must be set for all root nodes.
If the Root Reference property is set on the envelope schema, the Body XPath property must be set for the root node configured in the Root ReferenceBody XPath property can be set for the other root nodes, but it is not required. property. The
It is not required to set the Root Reference property, but the Body XPath property must always be set on at least one root node.

In addition to encapsulating multiple messages, envelopes can also supply header information that applies to all the records it contains. In the example, the BatchID element is defined once at the envelope level, but applies to all the individual order documents. This functionality of supplementing the individual message data with header data or metadata can be seen in the common example of SOAP envelopes .

Within BizTalk Server, envelope processing for received messages occurs in the disassemble stage of a receive pipeline. The out-of-the-box XML receive pipeline handles the removing and parsing of any envelope data contained within the inbound document. The subschemas are dynamically determined at runtime by the envelope's properties (specifically, the Body XPath), and used to validate the child documents' structure and split each out into an individual message.

A custom receive pipeline can also be used to more precisely process inbound enveloped documents. By leveraging the XML disassembler pipeline component, the envelope and document (child document) schemas can be set explicitly in the properties of the component. All envelope and document schemas explicitly set should have unique target namespaces.

NOTE

If envelope or document schemas are explicitly set on an XML disassembler component, only documents matching those schemas will be processed within the pipeline. The order of envelope schemas is enforced based on the configuration within the Envelope property on the XML disassembler component. In contrast, the order of the document schemas is not enforced.

After the inbound enveloped document has been debatched into individual messages, the disassembler promotes any properties of the envelope to each individual message. In the example, if the BatchID had been configured as a promoted property, it would have been attached to each individual message during disassembly.

Implementing envelopes for outbound messages is also possible within BizTalk Server. This process is handled in the assembling stage of a send pipeline. By leveraging the XML assembler pipeline component in a custom send pipeline, the envelope schemas can explicitly be set in the properties of the component. As it passes through the custom pipeline, the message will be wrapped in the specified envelope(s) and have the appropriate message properties demoted to the envelope(s). If a batch of outbound messages is sent through the custom send pipeline, the batch of messages will all be combined into a single document and wrapped in the specified envelope.

In addition to having a single envelope, a group of messages can be wrapped in a series of nested envelopes. Using nested envelopes provides a flexible way for transferring message batches that have a complex structure or relationship.

While the preceding solution used BizTalk Server's pipeline capabilities to handle inbound envelope processing, there are other implementation options to consider when designing a solution. A loop can be used within an orchestration to iterate over child records within an enveloped document, splitting out each submessage individually using XPath queries or a node list object. An orchestration can also be used to call out to an external assembly to handle message debatching. When determining which method to use, it is important to consider the following:

Does the entire batch need to be handled transactionally (for example, if one of the messages fails to process correctly, should the entire batch be canceled)?
• Do the individual records within the batch need to be processed in a specific order (for example, in the same order in which they appear in the original document)?
Does there need to be any event or notification after all messages in the batch are successfully processed?
Is business logic (orchestration) implemented on each individual message after it has been debatched?

If your business scenario would lead you to answer yes to any of these questions, using an orchestration to handle the parsing of an enveloped document may be required. The main benefit of using an orchestration is enhanced control over the processing of the individual messages. The order of the messages can be maintained; scopes can be used to implement transactionality across the entire message batch; compensating and error handling are more robust; and it is simple to implement logic required when all messages within a batch have completed processing. The major downsides to using orchestrations for debatching are performance and ease of modification to the solution (as changing the orchestration requires you to redeploy the BizTalk Server solution).

If the requirements of your business scenario allow for envelope processing to be handled via pipelines, you will realize performance gains. Additionally, the solution will be simplified by minimizing the number of implemented artifacts.