BizTalk 2010 Recipes : Document Mapping - Using the Database Lookup Functoid

1. Problem

You need to map an incoming node to a database table column to reference a specific set of data via a BizTalk Server map. Specifically, for an inbound author ID, you need to retrieve the person attributes that are stored in a SQL based table. Additionally, the data that is stored in the database table is dynamic, and coding the values within the BizTalk map is not possible.

2. Solution

BizTalk provides the Database Lookup functoid, which can retrieve a recordset. For example, suppose that the inbound message specifies an author's Social Security number but no personal information. The map must retrieve the author's information and map the information to the outbound message. The Database Lookup functoid can retrieve the information from a specific SQL table using the author's Social Security number as the value for which to search. The inbound XML message may have a format similar to the following and a table with data as show in Figure 1.

<ns0:PersonSearch xmlns:ns0="http://DatabaseLookupFunctoid.PersonSearch">
  <ID>172321176</ID>
</ns0:PersonSearch>

Figure 1. Record data

You can use the Database Lookup functoid by taking the following steps:

Figure 4. The final mapping

3. How It Works

The Database Lookup functoid requires four parameters as inputs, and it outputs an ActiveX Data Objects (ADO) recordset. Keep in mind that the recordset returns only the first row of data that matches the specific criteria provided to the Database Lookup functoid.

In addition to the input parameters, the Database Lookup functoid requires two helper functoids for optimal use:

• The Error Return functoid, which returns SQL-related errors or exceptions. The Error Return functoid requires the Database Lookup functoid as the input and a link to an output node in the target schema. To avoid runtime errors, verify that the only inbound connection to the Error Return functoid is that of the Database Lookup functoid and not any other database functoids. A generated error looks like that shown in Figure 5.

• The Value Extractor functoid, which retrieves a specific column from the returned recordset. The Value Extractor will retrieve the value based on the specific column specified in the input parameters.

Figure 5. An error has been generated

4. Security Considerations

Whenever you use SQL authentication (SQL username and password), there is potential for a security risk. Consider using trusted security for access to the database rather than specifying the username and password in the connection string used by the Database Lookup functoid. For example, here is a connection string that uses SQL security:

Provider=SQLOLEDB;Server=localhost;Database=pubs;User ID=sa;Password=password;
  Trusted_Connection=False

					  

And here is an example of a connection string that uses Trusted Security:

Provider=SQLOLEDB;Server=localhost;Database=pubs;Integrated Security=SSPI;

Keep in mind that if you choose Trusted Security for authentication, the account under which the BizTalk host instance is running must have appropriate access to the SQL Server, the SQL database, and the table in which the Database Lookup functoid is looking.

Another option to enclosing connection string information within the Database Lookup functoid is to make use of a Universal Data Link (UDL) file. A UDL is simply a text file with the file extension .udl. The connection string is included within the UDL file, and the connection string parameter in the Database Lookup functoid becomes a reference to that file, for example:

File Name=c:\BizTalkConfig\ConnectionString.UDL

Once the UDL file is created, it can be made available on a secure file share.

Additionally consider the use of a SQL view, versus direct table access, and having the Database Lookup functoid point to the database view. A SQL view offers the ability to manage table security permissions or the abstraction of the physical table structure.

5. Architecture Considerations

The Database Lookup functoid is convenient to implement in mappings. For straightforward data retrieval, this functoid performs adequately. However, the following items should be taken into consideration when evaluating when to use the Database Lookup functoid:

Database availability: If you cannot guarantee that the data source being queried will be available when BizTalk is available, using the Database Lookup functoid may not make sense.

Error management: Mapping will occur and not trap the SQL errors in the .NET exception style. Errors should be trapped and managed when mapping. When implementing the Database Lookup functoid, consider using the Error Return functoid. Additionally, after the mapping, it would be wise to query the Error Return node for an error message and implement error handling if one exists.

Performance: Evaluate your performance requirements and determine if accessing a SQL database will negatively affect your overall mapping performance. Implementing the Database Lookup functoid may not impact performance greatly, but consider the effect if you must run the Database Lookup functoid multiple times in a single map. Database Lookup functoids that are part of a looping structure will cause a level of performance degradation. Make sure that the latest BizTalk service packs are applied when using the Database Lookup functoid, as they include performance-enhancing features such as caching.

Database support: Evaluate if the database that you must access will support the necessary security requirements and also allow table (or at least view level) access.

6. Advanced Database Lookup Functoid Usage

The BizTalk map translates the Database Lookup functoid information into a dynamic SQL SELECT statement. If you run a SQL Profiler trace during testing of the BizTalk map, you will see the SELECT call with the dynamic SQL. Knowing that dynamic SQL is created by the Database Lookup functoid allows you to use it to perform some relatively powerful database lookups. The Database Lookup functoid allows only a single value and single column name to be referenced in the query. However, with a bit of extra mapping, you can use this functoid to query against multiple columns. The map in Figure 4 generates the following SQL query code:

exec sp_executesql N'SELECT * FROM people WHERE ID= @P1', N'@P1 nvarchar(9)',
N'172321176'

					  

This query performs a SELECT to retrieve all rows from the People table where the author ID is equal to the value in the inbound XML document (for example, 172321176).

Keep in mind that the Database Lookup functoid returns only the first row that it encounters in the recordset. If multiple authors had the same ID, you would potentially retrieve the incorrect author. For example, if the author ID is the last name of the author, you may retrieve multiple authors that share the same last name. One way to ensure uniqueness, aside from querying on a unique column, is to specify additional columns in the query. The Database Lookup functoid accepts only four parameters, so additional concatenation must occur before submitting the parameters to the Database Lookup functoid.

After configuring the inbound concatenated value, the next step is to specify multiple column names as the input parameter in the Database Lookup functoid. Figure 6 demonstrates a sample Database Lookup functoid configuration with multiple columns specified. The output from the Database Lookup functoid to the Value Extractor functoid does not change.

Figure 6. Database Lookup functoid with multiple columns

In this example, the inbound message specifies an author's first name and last name instead of a unique author ID. The map must still retrieve the author's information and map the information to the outbound message. The inbound XML message may have a format to the following message:

<ns0:PersonSearch xmlns:ns0="http://DatabaseLookupFunctoid.PersonSearch">
  <FirstName>Juan</FirstName>
  <LastName>Dos</LastName>
</ns0:PersonSearch>

The following is the dynamic SQL created in the map that accepts multiple columns:

exec sp_executesql N'SELECT * FROM authors WHERE FirstName+LastName= @P1', N'@P1
nvarchar(12)', N'JuanDos'

					  

The dynamic SQL created shows the inbound author's first name and last name parameters as a concatenated parameter. The SQL statement also shows a combination WHERE clause with FirstName + LastName.

There are some limitations to specifying multiple columns through the concatenation approach. Specifically, string data types are the only data types that work reliably due to the concatenation operation that occurs in SQL. Integer data types may also be used, but in the case of integer (or other numeric data types), SQL will perform an additive operation versus a concatenation operation. Adding two numbers together, as what would happen when specifying numeric data types, and comparing the result to another set of numbers being added together may yield multiple matches and may not achieve the desired results. The mix of varchar and numeric fields will not work with this approach, as you will receive a data casting exception from your data provider.