BizTalk 2009 : The Enterprise Service Bus Toolkit 2.0 - The Architecture (part 2) - Adapter Providers & Mediation Policies

3. Adapter Providers

Dynamic ports are the key component of ESB off-ramps; however, on their own they do not give you everything you need to achieve the highly reusable, generic pattern that the off-ramps should be providing. Dynamic ports read context properties from the messages that they are sending and then use the settings in these properties to configure themselves as needed.

In a traditional BizTalk project, a developer would figure out what properties needed to be set and would then assign values to them from within an orchestration or pipeline component. However, this does not work in the ESB Toolkit since a developer is not going to know what properties need to be set. This information is going to be available only at runtime after the resolver framework has executed.

You'll run into a problem when you realize that the resolver mechanism returns data using a dictionary object that send ports are not able to understand. For the send port to route your message for you, the information contained in the dictionary object must be extracted and placed into BizTalk context properties. That is what the adapter provider framework is for—to extract the data contained in the resolver's dictionary object and write it into the message's context properties so that the dynamic port can read it. Figure 6 shows this process.

Just like the resolver framework, the adapter provider framework follows a factory pattern. When configuration information is received from a resolver, the adapter manager reads it and determines which adapter needs to be used. Once the adapter manager figures this out, it instantiates the adapter provider associated with that adapter. If you refer to Figure 5, you will recall that the ActivateService service has a binding called TransportType://WCF-BasicHTTP. This binding is what the adapter manager uses to determine which adapter needs to be used. In this case, it is the WCF-BasicHTTP adapter.

Once the adapter manager loads the required adapter, it passes in the dictionary object and the message that needs to be sent out. Each adapter provider knows which context properties need to be set in order for a dynamic send port to use a specific adapter. The adapter provider works by extracting data from the dictionary object and writing it into the required context properties. Once it is finished, the adapter provider returns the BizTalk message, which can now be handed over to an off-ramp to be sent out.

Figure 6. The file adapter provider updating a message with data provided by a resolver

Currently, the ESB Toolkit provides seven adapter providers, listed in Table 1.

One thing to keep in mind is that although BizTalk server ships with a large number of adapters out of the box, you will not be able to use every adapter as a dynamic ESB off-ramp. You will be able to use only those adapters that have an adapter provider available. However, you can continue to use any adapter as long as you are setting up a static or dynamic port that does not need to retrieve its configuration information using the resolver framework. In this manner, you'll see a number of developers creating hybrid solutions that use elements of the ESB Toolkit alongside traditional BizTalk practices. Of course, the ESB Toolkit consists of a very extensible framework that allows you to extend almost every feature of it. Therefore, you can easily create your own custom adapter provider if needed.

4. Mediation Policies (Itineraries)

So, you have seen that the ESB gives you on-ramps, off-ramps, reusable services, and resolvers to use. The next thing you need to know is how to use them. How do you hook them all together and use them to process your message?

This is where the concept of mediation policies comes in. Mediation policies allow you to define how your message should be processed by all of the mediation components. They also allow you to specify which resolvers should be used by each of those mediation components.

To avoid too much confusion, there is one thing that we should clear up before you proceed any further. Mediation policy is a conceptual term that we use when explaining the high-level architectural components within the ESB. Once we get down into the details of the ESB Toolkit, you will see that we change terminology and refer to these policies as itineraries. We will use the term mediation policy and itinerary interchange-ably. Do not panic—they mean exactly the same thing. Mediation policy is simply the conceptual term, whereas itinerary is the lower-level technical term.

We have found that the best way to understand the itinerary concept is to start with a real-world scenario, so let's define one. You have a basic web application that needs to route a message to a back-end web service. This service is defined in a UDDI 3 server, and all of the details about the service's endpoint are stored in this server. Now, the data structure that the web application uses is slightly different from the structure that the web service uses.

Fortunately, the ESB already has several maps in place to handle the kind of transformation just described. However, you do not know which map is needed, so you need the ESB to use the Business Rule Engine to figure that out. Essentially, you need the ESB to transform the message and route it to the back-end service. Let's assume for this sample that this is a one-way, fire-and-forget service.

Looking at the mediation components that are in the ESB, you know that this should be straightforward to do. All you need to do is follow these steps:

What you have just defined here at a high level is a mediation policy. It defines how your message should be processed by the various mediation components and where those components should retrieve their configuration information from.

4.1. How Mediation Policies Are Implemented

In a traditional BizTalk environment, you might use an orchestration to implement the process just described. It would be fairly easy to use an orchestration to hook all of the receive and send ports together. You could add the Business Rule Engine shape to access the rules engine, drop in a map shape to execute the map, drop in an expression shape, and write .NET code to call the UDDI server. However, as we have said multiple times already, the ESB is designed to be generic and reusable. Therefore, you do not want to create a heavy custom orchestration that would be able to handle only one specific process. You also want to avoid having to modify the BizTalk environment at all if possible. This is why you are going to use a mediation policy instead of an orchestration to configure this process.

What makes a mediation policy fundamentally different from an orchestration is that it is nothing more than XML data that travels along with the message as it passes through the ESB instead of being a static artifact that is deployed into the ESB. Unlike orchestrations, mediation policies do not need to be compiled into a DLL, and they don't need to be registered in the global assembly cache or even in BizTalk. The XML is stored in the message's context properties and can be accessed by any component as the message flows through the ESB.

Before digging too far into the implementation of mediation policies, let's take a look at an actual itinerary for the scenario we just put forth. The itinerary is in the following code example. Please note that the line numbers are not part of the itinerary. We have simply added them to make it easier to refer to specific lines.

1. <?xml version="1.0" encoding="utf-8"?>
2. <Itinerary xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" uuid="" beginTime="" completeTime=""
state="Pending" isRequestResponse="false" servicecount="0"
xmlns="http://schemas.microsoft.biztalk.practices.esb.com/itinerary">
3. <BizTalkSegment interchangeId="" epmRRCorrelationToken="" receiveInstanceId=""
messageId="" xmlns="" />
4. <ServiceInstance name="Microsoft.Practices.ESB.Services.Transform"
type="Messaging" state="Pending" position="0" isRequestResponse="false"
xmlns="" />
5. <Services xmlns="">
6. <Service uuid="cfbe36c5-d85c-44e9-9549-4a7abf2106c5" beginTime=""
completeTime="" name="Microsoft.Practices.ESB.Services.Transform"
type="Messaging" state="Pending" isRequestResponse="false"
position="0" serviceInstanceId="" />
7. </Services>
8. <Services xmlns="">
9. <Service uuid="6a594d80-91f7-4e10-a203-b3c999b0f55e" beginTime=""
completeTime="" name="Microsoft.Practices.ESB.Services.Routing"
type="Messaging" state="Pending" isRequestResponse="false"
position="1" serviceInstanceId="" />

					  

10. </Services>
11. <Services xmlns="">
12. <Service uuid="0e1d2b42c9564352b6878accc290fe52" beginTime=""
completeTime="" name="DynamicOffRamp" type="Messaging"
state="Pending" isRequestResponse="false" position="2" serviceInstanceId="" />
13. </Services>
14. <ResolverGroups xmlns="">
15. <Resolvers serviceId="Microsoft.Practices.ESB.Services.Transform0">&lt;![
CDATA[BRE:\\Policy=ResolveMap;Version=1.0;UseMsg=False;]]&gt;
</Resolvers>
16. <Resolvers serviceId="Microsoft.Practices.ESB.Services.Routing1">&lt;![
CDATA[UDDI3:\\ServerUrl=http://localhost/uddi;SearchQualifiers=andAllKeys;
CategorySearch=;BindingKey=uddi:esb:orderfileservicev3.1;]]&gt;
</Resolvers>
17. <Resolvers serviceId=" DynamicOffRamp " />
18. </ResolverGroups>
19. </Itinerary>

					  

4.2. Interpreting Itineraries

The XML that comprises this previous sample itinerary can look fairly scary the first time you see it. Fortunately, with version 2.0 of the ESB Toolkit, you will not need to work directly with this XML.

The first lines you should look at are lines 5 through 13. In these lines, you can see that we have a repeating node called <Services> that has a child node called <Service>. This <Services> node is where you define which services need to run and where you also define the order in which they need to run. For this example, we required three services: the Transformation Service, the Routing Service, and an off-ramp to transmit the message. This <Service> node requires the following structure:

<Services xmlns="">
<Service uuid="" beginTime="" completeTime="" name="" type="" state = ""
isRequestResponse="" position="" serviceInstanceId=""
</Services>

The key attributes to be aware of are name, type, isRequestResponse, state, and position. The name field is used to specify which service you want to run, that is, the map service, the Routing Service, an off-ramp, and soon. The type field is used to specify whether this service is a messaging-level service (meaning a pipeline will execute it) or an orchestration-level service. The isRequestResponse field is used to specify whether this is a one-way, fire-and-forget service or a two-way, request-response service. The state attribute is a marker that indicates whether this service has been processed yet by the ESB components. Its value is either Pending or Complete. Finally, the position field is used to define the order in which the services should run.

On line 6 in the sample itinerary, you can see that we have specified that the dynamic mapping service should execute first. We have also said this is a one-way service and that it should be executed by the ESB at the messaging level.

On line 9, we have declared that the dynamic Routing Service should be executed by ESB next. Again, this is a one-way service that is to be implemented at the messaging level.

Finally on line 12, we have said that the one-way dynamic off-ramp name DynamicOffRamp should be used by the ESB to transmit the message to the remote service.

Now, the ESB components are not set up to parse through these repeating <Services> nodes to find out which services need to be processed. Such parsing would represent too much overhead and would certainly be a slow process.

What we really need is some kind of pointer that would contain the information for the current service that needs to be implemented. All of the ESB components could then look to this one pointer to quickly figure out what work needed to be done next. The <ServiceInstance> node (which is on line 4) acts as this pointer. It contains the same information that the repeating <Service> nodes have. The only difference is that the <ServiceInstance> node holds only the information for the service that currently needs to be executed. When a message is first submitted into the ESB, the <ServiceInstance> node will have the same values as the first <Service> node. Each ESB component is designed so that once it completes its unit of work (whatever it is), it calls into the itinerary mechanism and asks that the itinerary be updated. This means that the XML is updated, and the current service has its state attribute set to Complete and its completeTime attribute set to the current time. The attributes in the <ServiceInstance> node are all updated to contain the values for the next service that needs to be run. At this point, the message is essentially thrown back into the ESB infrastructure, and whatever ESB agent is required to run next will execute.

The final lines that you should look at are 14 through 18. In these lines you can see where we are actually defining the resolver connection string that we talked about in the "Resolvers" section of this article. These are the connection strings that the resolver framework will use for each of the ESB services that we have defined. You can identify which resolver goes with which service by looking at the serviceId attribute in the <Resolvers> node. For example, look at line 15. You will see that this <Resolver> node has a serviceId equal to Microsoft.Practices.ESB.Services.Transform0. This value is a concatenation of the name and position attributes from the service defined on line 6. When each ESB service runs, it looks for a resolver that has a serviceId matching the name and position of the currently executing service. One line 15, you can see that we have said that the mapping service should use the Business Rule Engine to determine which map to run. You can determine this since a "BRE" connection string is defined for this resolver. For each service defined in the itinerary, there is a corresponding resolver defined for it.

Based on this itinerary, the ESB components would function as follows:

Figure 7 shows a visual representation of how this process would execute. From this example, you can see how an itinerary can link the various mediation components together for you.

Figure 7. Shows how process flows for an itinerary

To reiterate, the great thing about this approach is that we did not need to generate any new code to do this. We did not need to modify BizTalk and did not create a static, tightly coupled orchestration. This means that if we needed to make a change to this itinerary in the future (or add a new one), we could just edit the XML, and we would not need to make any change to BizTalk or recompile any low-level code.