BizTalk 2006 : Creating More Complex Pipeline Components (part 1) - Dynamically Promoting Properties and Manipulating the Message Context

Once you master the basics of creating a component, looking at the message, and figuring out what you can do with the data, you may ask yourself, "Okay, so what else can these things do?"We usually answer this with "Pipeline components are much like the Matrix in that you cannot be told what they can do; you must see it for yourself." In this respect, it is much easier to think about components in terms of specific problems they can solve.

Dynamically Promoting Properties and Manipulating the Message Context

The simplest and most common use of a pipeline component is to promote custom properties into the message context for a message. Often the data for the property is not available in the schema or is not easily accessible—so a simple promotion in the schema editor is not an option. and promote properties to the context.

Dealing with "Out of Order" Message Sequences

When dealing with Interchanges, most often it would be great to know just how many messages are actually in the Interchange and which one is the last one. Preserving order is a major pain for any type of application that needs to do so and potentially resequence messages to an outbound system. Normally selecting ordered delivery is good enough for this task, but two major problems still exist:

• Ordered delivery only works if you receive the messages in the proper order.

• Selecting a port as ordered has a huge performance hit, because messages will essentially be single-threaded as they are dequeued from the Messagebox.

What is needed is a resequencer pattern to reorder the messages into the proper order once they are received by an orchestration. The following is a definition and example of a resequencer as given by the Enterprise Integration Patterns site:

^[] Enterprise Integration Patterns site, Resequencer page (www.enterpriseintegrationpatterns.com/Resequencer.html). This site is an excellent resource for all types of patterns that can easily be created using BizTalk Server.

A Message Router can route messages from one channel to different channels based on message content or other criteria. Because individual messages may follow different routes, some messages are likely to pass through the processing steps sooner than others, resulting in the messages getting out of order. However, some subsequent processing steps do require in-sequence processing of messages, for example to maintain referential integrity.

How can we get a stream of related but out-of-sequence messages back into the correct order? The figure below logically demonstrates how this concept works.

Use a stateful filter, a Resequencer, to collect and re-order messages so that they can be published to the output channel in a specified order.

The Resequencer can receive a stream of messages that may not arrive in order. The Resequencer contains in internal buffer to store out-of-sequence messages until a complete sequence is obtained. The in-sequence messages are then published to the output channel.It is important that the output channel is order-preserving so messages are guaranteed to arrive in order at the next component. Like most other routers, a Resequencer usually does not modify the message contents.

We will give a complete example of how to implement custom resequencers using BizTalk orchestrations, convoys, and custom pipeline components. But the most fundamental part of the equation is storing the order. To do that, all you need is a simple pipeline component that can promote what the sequence number is for the current message along with a trigger for determining whether or not it is the last message in the Interchange. Resequencing is most commonly done using a Disassembler component to actually create messages, but can also be implemented using a simple Encoding component if the messages arrive out of order and within different Interchanges.

Custom Distinguished Fields

When you are writing a custom Disassembler, and you want to have distinguished fields that are accessible from within an orchestration, you will need to ensure that they are written to the context of the message before it is stored in the Messagebox. This is because normally when you use an XMLReceive pipeline, it automatically stores these values for you, but when you are writing a custom component, that storing of values doesn't happen auto-magically, so you need to deal with it yourself.

^[] People often do not understand what the runtime engine is doing when it processes a distinguished field. Most think that the distinguished field is simply a shortcut to the XPath expression that maps to the field in the document. Actually this is not true. The distinguished field defines what the XPath expression is, but the runtime engine prefetches this data when it processes the message. This is basically done because it is far more performant to do this, and the product team wrote their own XPath navigator component to use streams, making it very efficient to do this as the document is being processed. This is why there is very little performance overhead when you access a distinguished field in an orchestration—it is already loaded and cached. If you used XPath to get the value, the orchestration engine would have to load the entire document and evaluate the XPath expression manually— which is slow and very expensive when dealing with large documents.

The solution to the problem is simple—just write the value to the message context using code as shown in the following snippet. Note the format of the property name and the namespace.In order to be processed by the orchestration engine, they must be named as follows:

Name

The distinguished field location in XPath: "/*[local-name()='PurchaseOrder' and namespace-uri()='http://ABC.FullFillment']/*[local-name()='UnitPrice' and namespace-uri()='']"

Namespace URI

"http://schemas.microsoft.com/BizTalk/2003/btsDistinguishedFields"

//BizTalk System Properties Namespace
Private Const BTSFieldXPathLocation As String = "/*[local-name()='PurchaseOrder' _
and namespace-uri()='http://ABC.FullFillment']/*[local-name()='UnitPrice' and _
namespace-uri()='']"

Private Const BTSDistinguishedFieldsPropertiesNamespace As String = "_
http://schemas.microsoft.com/BizTalk/2003/btsDistinguishedFields "

//Write a distinguished property
message.Context.Write(BTSFieldXPathLocation, _
BTSDistinguishedFieldsPropertiesNamespace, 10);

					  

Checking for Schema Types in Components

As was stated previously, pipeline components that are expecting incoming documents to conformto a particular schema should do two things:

• They should probe the incoming document and determine whether they can process it based on the schema's namespace and root node.

• They should allow the developer to choose the allowed incoming schemas at design time using the Pipeline Designer.

Validating, also called probing the message, checks the incoming schema and simply indicates to the runtime that the component will be able to handle the schema—essentially a Boolean value. Ensuring that components validate against the schema is critical, as it often allows the same component code to be reused for multiple applications and also allows for per-instance pipeline configuration. This is done using the IProbeMessage interface.

IProbeMessage

The IProbeMessage interface has only one method?—Probe. This method checks whether the incoming message is in a recognizable format. The Probe method passes in the pipeline context object as an IPipelineContext interface along with the message represented as an IBaseMessage Interface and returns a Boolean. If the method returns False, the pipeline component cannot process the message, the current component instance stops executing, and the pipeline component execution sequence continues as defined in Figure 1. If it returns True, then the pipeline component executes as normal with its primary operation (Encode, Execute, Disassemble, etc.).

Following is an example of a simple IProbeMessage implementation:

Public Class MyProber Implements Microsoft.BizTalk.Component.Interop.IProbeMessage
Private _MyDocSpec As Microsoft.BizTalk.Component.Utilities.SchemaWithNone = New_
Microsoft.BizTalk.Component.Utilities.SchemaWithNone("")

'<summary>
'This property is the document specification for the inbound document. Only
'documents of this type will be accepted. The SchemaWithNone allows the developer to
'select the inbound document type from a pick list.
'</summary>
<Description("The inbound request document specification. Only messages of this _
type will be accepted by the component.")> _
Public Property MyDocSpec() As Microsoft.BizTalk.Component.Utilities.SchemaWithNone
            Get
                Return _MyDocSpec
            End Get
            Set(ByVal Value As Microsoft.BizTalk.Component.Utilities.SchemaWithNone)
                _MyDocSpec = Value
          End Set
End Property

Public Function Probe(ByVal pc As _
Microsoft.BizTalk.Component.Interop.IPipelineContext, ByVal inmsg As _
Microsoft.BizTalk.Message.Interop.IBaseMessage) As Boolean Implements _
Microsoft.BizTalk.Component.Interop.IProbeMessage.Probe

            Dim streamReader As New streamReader(inmsg.BodyPart.Data)
            Dim xmlreader As New Xml.XmlTextReader(inmsg.BodyPart.Data)
            xmlreader.MoveToContent()

            If (_MyDocSpec.DocSpecName = xmlreader.NamespaceURI.Replace("http://", _
"")) Then
                Return True
            Else
                Return False
            End If
End Function
End Class