BizTalk 2006 : Getting Started with Pipeline Development (part 2) - Understanding Pipeline Execution, Understanding Interchanges

2. Understanding Pipeline Execution

Pipeline components execute in a sequential order starting from the first stages and working down to the last. The exception to this rule is the Disassembler component, which will execute sequentially within the Disassemble stage of the receive pipeline until a component is found that is able to process the message. The first Disassembler component that accepts the message is executed, and then the resulting messages are passed to the remaining stages of the pipeline. When pipeline components execute sequentially, the output of one pipeline component is used as input to the next one in the sequence. Note the Disassemble stage in Figure 3. These components will fire sequentially until one of them determines that they can handle the incoming message. In this scenario, the first component to do this executes.

Figure 3. Pipeline execution

3. Understanding Interchanges

Most BizTalk developers really do not understand or appreciate fully what an Interchange is. An Interchange is a message or series of messages that flow through the Messagebox. Generally one message = one Interchange. The Interchange is uniquely identified by the InterchangeID promoted property. In many cases, the Interchange contains more than one message. This is often the case when processing flat-file documents or XML documents that contain envelopes. In this case, there will be one Interchange with as many messages as were contained in the original envelope. Each message would have the same InterchangeID; however, they would all have unique MessageIDs. In pipeline development, only Disassembler and Assembler components need to be concerned about this, since all other components receive one message and return one message. Disassemblers will receive one message and de-batch it into many messages, and the reverse is the case with Assemblers.

What Is an Interchange?

In terms of BizTalk processing, an Interchange is the message or series of messages that are received by a receive port and run through a receive pipeline. Interchanges with multiple documents generally relate to batch messages that are de-batched by a Disassembler component within a pipeline. If an envelope message was received with 20 documents inside, the Interchange would contain 20 messages. Each message would contain the same InterchangeIDs within the message context and distinct MessageIDs as shown in Figure 4.

In BizTalk Server 2004, should a document within the Interchange or "batch" be considered invalid either due to a "subscription not found" error or because the data within the message does not conform to the schema specified, every message within the batch would fail. This had dramatic repercussions for systems that did not want this functionality. In BizTalk 2006, there is anꌡoption on the pipeline to allow the pipeline to not fail should one message be bad, continuing processing of the batch and only failing the individual messages that are in error. The remaining messages in the batch would be allowed to proceed. The message that failed can be subscribed to and routed to a generic error-handling mechanism such as an orchestration or a port.

4. Overview of Recoverable Interchange Handling

Recoverable Interchange Processing (RIP) is a mechanism whereby the Messaging Engine will allow you to subscribe to special properties that are promoted by BizTalk when an error condition occurs. Typically, this is used in de-batching scenarios where a pipeline will be creating multiple messages in an Interchange. Should an error occur, the recoverable Interchange setting will allow the pipeline to throw an error only on the documents that failed and permit the remaining documents to be passed through the pipeline.

Message Suspension

Recoverable Interchange Processing is a function of the Disassembler component configured within the pipeline. If a message is in error and the component is using recoverable Interchanges, the message in error will be suspended and placed in the suspended queue; other messages will be propagated down the pipeline for further processing.

Once the messages within an Interchange are propagated down the pipeline or placed in the suspended queue, the further processing of those messages is treated transactionally as before. If a message fails at any point during its processing path, except in its routing (for example, no matching subscriber), all of the messages within the Interchange are thrown away and the originating Interchange is placed in the suspended queue.

Message Failure Points

Failur猸s in the message stages defined here result in the entire Interchange being suspended:

• Validate

• Resolve Party

• Map

Interchanges that fail processing due to one of these errors will become suspended. These Interchanges can then be resumed from within the BizTalk Administration Console. However, the Interchange will still likely fail unless the underlying cause of the failure is addressed.

Receiving the following errors will not cause the XMLDisassembler component to stop processing messages from an Interchange:

• Schema not found

• Schema ambiguity (more than one schema exists for the same message type)

• XML validation failed

• Flat-file parsing failed

Interchange processing will stop within the XMLDisassembler component if the message data is not well-formed XML. Since the data is read as a stream, a good check is to see whether document properties that would cause System.Xml.XmlReader to error are present. If they are, the XMLDisassembler will fail as well.

Messages that are extracted from Interchanges but fail because of a "No matching subscription found" error can be successfully resumed. All that is needed in this case is to ensure the BizTalk port or orchestration that has a subscription for the message is enlisted. The message can then be successfully resumed.

The two examples that follow illustrate the differences in Interchange processing as affected by RIP.

Example. Example 1: Standard Interchange Processing Behavior for the XMLDisassembler Component

<MyBatch>
      <SubDoc1>MyDataValue1</SubDoc1> //No Error
      <SubDoc2>MyDataValue2</SubDoc2> //Routing Error
      <SubDoc3>MyDataValue3</SubDoc3> //No Error
      <SubDoc4>MyDataValue4</SubDoc4> //Pipeline Failure - will be recoverable
      <SubDoc5>MyDataValue5</SubDoc5> //No Error
</MyBatch>

					  

Example. Example 2: Recoverable Interchange Processing Behavior for the XMLDisassembler Component

Using the same inbound batch document as shown in Example 1, if we set the XMLDisassembler to use RIP, the execution that will occur in the pipeline processing will be dramatically different. SubDoc1, SubDoc2, and SubDoc3 successfully pass through the pipeline and are ready to be published to the Messagebox. SubDoc4 generates an error in the XMLDisassembler and will be suspended. SubDoc5 passes through the pipeline and is able to be published to the Messagebox.

Once the entire Interchange is processed, SubDoc1, SubDoc2, SubDoc3, and SubDoc5 are 콫uccessfully published to the Messagebox. SubDoc4 is placed in the suspended queue.