BizTalk 2010 Recipes : Adapters - Receiving Messages with the SQL Adapter

4/24/2011 6:18:42 PM

1. Problem

You need to create an orchestration that retrieves messages from a SQL table via a SQL stored procedure. Specifically, you need to create an orchestration that will receive individual customer records from your SQL database via a SQL stored procedure.

2. Solution

The solution reviewed as part of this recipe outlines how to use the SQL receive adapter within the context of an orchestration. The solution has a sample table and stored procedure to demonstrate how the SQL receive adapter routes messages based on message content. The following instructions outline the tasks required to create and configure the physical SQL receive port as well as the orchestration that implements the SQL receive adapter.

3. Task 1: Create Supporting Artifacts

Perform the following steps to create the SQL support artifacts that are used by this recipe:

Create a new SQL database and table for which to insert the record. Compile the following table definition, and insert the statements listed in Listing 1.

NOTE

The table definition is different from the previous recipe. Make sure to use the table definition in Listing 6-6 or modify the previously created table definition.

Example 1. Example Customer Table Definition

CREATE TABLE [dbo].[Customer] (
    [CustomerID][int] IDENTITY (1,1) NOT NULL,
    [CustomerName][nvarchar](30) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
    [Address][nvarchar](60) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
    [City][nvarchar](30) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
    [Region][nvarchar](30) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
    [PostalCode][nvarchar](10) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
    [RecordStatus][tinyint] NOT NULL
) ON [PRIMARY]

Insert Customer (CustomerName, Address, City, Region, PostalCode, RecordStatus)
Values ('Shauna Marie', '9515 S Wally St', 'Bellevue', 'WA', '98004', 1)
Insert Customer (CustomerName, Address, City, Region, PostalCode, RecordStatus)
Values ('Pat Dean', '10034th 49th Ave NE', 'Redmond', 'WA', '95550', 1)
Insert Customer (CustomerName, Address, City, Region, PostalCode, RecordStatus)
Values ('Sherrie Long', '5015 48th Ave SW', 'Woodland Park', 'CO', '80863', 1)

If you do not have a stored procedure to retrieve individual customer records, compile the stored procedure listed in Listing 6-7.

NOTE

Notice that the select statement in Listing 2 contains a FOR XML AUTO, XMLDATA, ELEMENTS clause. The SQL adapter requires select statements returning records to return them in XML format. Additionally, the XMLDATA clause is required only when running the wizard to generate the schema for the result set from the stored procedure. When you test the solution, make sure to remove the XMLDATA clause from your stored procedure.

Example 2. Example Retrieve Customer Record Stored Procedure

if exists(select * from sysobjects where name = 'CustomerToBizTalk' and type = 'P')
    drop proc CustomerToBizTalk
go
CREATE procedure CustomerToBizTalk
AS
/*
**
** Object: CustomerToBizTalk

**
** Description: Retrieves records from the Customer table for the SQL receive
**                adapter recipe
**
**
*/
BEGIN
    /*Populate temp table with records that will be returned to caller*/
    Select top 1 CustomerID
        , CustomerName
        , Address
        , City
        , Region
        , PostalCode
        , RecordStatus
    Into    #TempCustomer
    From    Customer
    Where    RecordStatus = 1

    /*Update Status of retrieved record from the Customer table*/
    Update    Customer
    Set        RecordStatus = 0
    Where    CustomerID = (select CustomerID from #TempCustomer)

    /*Return records to caller*/
    Select CustomerID
        , CustomerName
        , Address
        , City
        , Region
        , PostalCode
        , RecordStatus
    from     #TempCustomer as Customer FOR XML AUTO, XMLData, Elements

    /*Clean up temp table*/
    drop table #TempCustomer
END

go
grant execute on dbo.CustomerToBizTalk to public
go

4. Task 2: Create BizTalk Solution Artifacts

Next, use the Add Generated Items Wizard to generate the BizTalk artifacts for this recipe. The wizard will prompt you to specify your BizTalk environment (including specifying the BizTalk MessageBox database, setting the connection string to the database you will be accessing, and identifying the type of SQL port you are creating such as send or receive). This recipe assumes you have created an empty BizTalk solution. Here are the steps to follow:

Choose Add Generated Items from the Solution Explorer's tool menu. Select the Add Adapter Metadata template.
On the Add Adapter Wizard page, select the SQL adapter, and verify the settings to your BizTalk MessageBox database.
NOTE

You can leave the Port field empty unless you are specifying an already created SQL send or receive port.
Click Next, and set the connection string for the database to which you are connecting (this is where the table and stored procedure created in the previous task reside). Then, in the SQL Transport Schema Generation Wizard, specify the target namespace as http://SQLReceiveAdapter and the document root element name as Customer_Response. The wizard should resemble Figure 1.

Figure 1. Using the SQL Transport Schema Generation Wizard
Click Next, and on the Statement type information page of the wizard, select Stored Procedure as the type of statement. On the Statement Information page of the wizard, select the CustomerToBizTalk stored procedure from the drop-down enumeration, and click the Generate button. The Generate button results in a generated execute script that is listed in the Generated Script field of the wizard. Figure 2 displays the completed wizard.
NOTE

If the stored procedure contained any input parameters, those parameters would display in the datagrid of the wizard. The wizard also depends on the select statement in the stored procedure to contain the XMLDATA clause in the FOR XML statement. The XMLDATA clause returns the schema layout for the structure returned by the select statement.

Figure 2. Completed wizard
Click Finish, completing the wizard. When you complete the wizard, notice that BizTalk creates two artifacts for you: the BizTalk orchestration and SQLService.xsd.
NOTE

The BizTalk orchestration allows you to perform additional processing of the inbound message, and the SQLService schema contains the physical representation of the data returned from the stored procedure.
Modify the BizTalk orchestration, configuring it based on the artifacts listed in Table 1. Configure the BizTalk orchestration for an inbound receive and an outbound send. See Figure 3.
Once your orchestration is complete, build and deploy your solution.

Table 1. SQL Receive Orchestration
ID	Name	Discussion
1	Receive Customer	Receive shape. Set the Activate property to True.
2	oprtReceiveCustomerSQL	Receive port
3	oprtReceiveCustomerSQLType	Receive port type
4	msgSQLCustomer	Orchestration message pointed to the schema generated from the Add Generated Items Wizard
5	Send Customer	Send shape
6	oprtSendCustomer	Send port
7	oprtSendCustomerType	Send port type

Figure 3. Viewing the complete orchestration

5. Task 3: Create Physical BizTalk Ports

Execute the following steps to create the SQL receive port that will call the stored procedure and route the message received to the deployed BizTalk solution:

To test this solution, you will need to create a physical SQL receive port to consume the inbound XML file and a physical file send port to send the message to a file share for examination.
From the BizTalk Administration Console, right-click the Receive Ports folder, and select Add Receive Port => One-way Port to open the One-Way Receive Port Properties dialog box. Name the new receive port Sample SQL Receive Port. Leave the rest of the settings with default values.

Right-click the Receive Locations folder, and select Add Receive Location. Select the receive port that was created in the previous step, and click OK. In the Receive Location Properties dialog box, configure the transport type, address, receive handler, and receive pipeline. Configure the dialog box settings specified in Table 2.

Table 2. SQL Receive Location Configuration Settings
Configuration Parameter	Value
Transport Type	SQL
Address (URI)	Location of your SQL server (See step 5 for more information.)
Receive Handler	BizTalk host instance (for example, BizTalkServerApplication)
Receive Pipeline	Microsoft.BizTalk.DefaultPipelines.XMLReceive

When configuring the Address (URI) property in the Receive Port dialog box, leave the Receive Configuration settings to the default values:
1. Set Document Root Element Name to Customer_Response.
2. Set Document Target Namespace to http://SQLReceiveAdapter.
3. On the SQL Command page, navigate to the deployed BizTalk project and schema to set the SQL command properly (as shown in Figure 4). The completed SQL Transport Properties dialog box should resemble Figure 5.
Figure 4. Selecting the Schema from the Deployed Project

NOTE

BizTalk uses the root element name and the namespace to route the inbound XML file. If the root element name and the namespace do not match the root element name and namespace of the SQLService.xsd file you created in the BizTalk solution, you will receive document routing errors and suspended messages when BizTalk tries to route the message from the SQL receive adapter.
Bind the deployed orchestration to the newly created send and receive ports.

Figure 5. Viewing the completed SQL Transport Properties dialog box

6. Task 4: Test the Solution

Your final task is to test your solution. Here are the steps to follow:

Modify and recompile the CustomerToBizTalk stored procedure. Remove the XMLDATA clause in the FOR XML AUTO statement and recompile it. The resulting stored procedure should resemble Listing 3.

Example 3. Revised Select Statement

/*Return records to caller*/
    Select CustomerID
        , CustomerName
        , Address
        , City
        , Region
        , PostalCode
        , RecordStatus
    from     #TempCustomer as Customer FOR XML AUTO, Elements

Start your orchestration and SQL receive port, and validate that a single XML file is created for each customer in your Customer table with the RecordStatus of 1. Also verify that the records in the Customer table have had their status set to 0 indicating that BizTalk has consumed the record. The resulting XML file should resemble Listing 4.

Example 4. Customer Sample XML

<?xml version="1.0" encoding="utf-16" ?>
<Customer_Response xmlns="http://SQLReceiveAdapter">
  <Customer>
      <CustomerID>1</CustomerID>
      <CustomerName>Shauna Marie</CustomerName>
      <Address>9515 S Wally St</Address>
      <City>Bellevue</City>
      <Region>WA</Region>
      <PostalCode>98004</PostalCode>
      <RecordStatus>1</RecordStatus>
  </Customer>
 </Customer_Response>

7. How It Works

Retrieving records via the SQL receive adapter is extremely useful and powerful when coupled with the use of stored procedures. Implementing the SQL receive adapter requires configuring and managing how records are retrieved from SQL. This recipe outlined how to use a stored procedure that selects records in a temporary table, updates the status of the individual record in the SQL table, and then returns the records to the SQL adapter. Additionally, this sample demonstrated how to use the FOR XML AUTO clause within a stored procedure.

8. Stored Procedure Configuration

The SQL receive adapter operates on the polling principle of retrieving records. The adapter will call the stored procedure on a configurable interval to retrieve records. However, the adapter does not maintain the state of which records it has consumed. Therefore, any stored procedure that the SQL receive adapter executes must maintain a level of state identifying the processed records. If you examine the stored procedure listed previously, you will notice that there is a RecordStatus column that is updated to 0 when the records are polled. You have many options for determining unprocessed records. For example, you can use the status flag, physically remove the record from the database, use the date range, and so on. If the stored procedure does not maintain state, BizTalk will continue consuming the same data.

In addition to maintaining state within the stored procedure, if you are returning a result via a select statement, then you must specify the FOR XML AUTO, XMLDATA clause for generating the SQLService.xsd schema. As explained, the XML AUTO clause instructs SQL Server to return the result set as XML, and the XMLDATA clause instructs SQL Server to return the schema layout for the returned result set. The SQL service adapter officially supports using the FOR XML AUTO clause.

You can also use the FOR XML Explicit clause, but it is not officially supported and should be used with care. Using FOR XML Explicit eliminates using the Add Generated Items Wizard to automatically generate the SQLService schema. However, you can create the SQLService schema manually. One benefit to using FOR XML Explicit over FOR XML AUTO is the greater level of control in the resulting XML structure that is returned to BizTalk. If you have a complicated XML message structure that is comprised of data from multiple tables, you can format the result set in the correct XML structure via the FOR XML Explicit clause. The select statement in Listing 5 demonstrates how to use the FOR XML Explicit clause.

Example 5. Return Result Set Using the FOR XML Explicit Clause

select    1    as Tag, NULL    as Parent
    , NULL    as [Customer!1!element]
    , CustomerID as [Customer!1!CustomerID!element]
    , CustomerName    as [Customer!1!CustomerName!element]
    , Address as [Customer!1!Address!element]
    , City as [Customer!1!City!element]
    , Region as [Customer!1!Region!element]
    , PostalCode as [Customer!1!PostalCode!element]
    , RecordStatus as [Customer!1!RecordStatus!element]
from    Customer
FOR XML EXPLICIT

9. Adapter Configuration

The SQL receive adapter contains many items for configuration. Table 3 lists the items worth noting. The SQL adapter receive location differs from other BizTalk receive locations in that each SQL receive location identifies a specific message type that is received from a database. If you expect to receive multiple types of messages, you must construct a SQL receive port for each unique message type. The SQL adapter is configured for each unique message type through the Document Root Element Name node and the Document Target Namespace node. Additionally, each SQL receive location is configured to execute a specific SQL stored procedure or command.

Table 3. SQL Receive Adapter Configuration
ID	Configuration Parameter	Description
1	Poll While Data Found	Indicates whether to continue polling for new records in additional batches or whether to submit a single query result for the entire polling window.
2	Polling Interval	Identifies how often to poll for records.
3	Polling Unit of Measure	Identifies the unit of measure for polling.
4	Connection String	Identifies the connection string for the database containing data to be polled.
5	Document Root Element Name	Identifies the Root node that BizTalk will assign the inbound message.
6	Document Target Namespace	Identifies the namespace that BizTalk will assign the inbound message.
7	SQL Command	Identifies the SQL command the SQL adapter will execute.
8	URI	Identifies a unique identifier for the receive location. This value is autogenerated.

In addition to the configuration items identified previously, you must verify that MSDTC is configured correctly on the target system that is hosting the SQL database because BizTalk enrolls every call from the SQL adapter into an MSDTC-styled transaction.