5. What to Keep in Mind About Your Code
A few simple things to keep in
mind before jumping into tuning your BizTalk solution as well are your
current coding practices. Coding is a constant improvement process.
While coding and performing code reviews, since the Assembly/C/C++ days,
we have constantly written and seen comments like "// —TBD: refactor or
rewrite in next rev." This is an obvious sign that no matter what stage
you are in, there is always some code in your application or library
that, given the time, you would revisit, rewrite, or repackage.
Some of the BizTalk coding
caveats that tend to apply to most BizTalk developers, and that
generally represent quick performance wins, are as follows:
Move your message transformations to the ports: This minimizes the number of message copies created in your orchestration.
Avoid using XmlDoc objects in your orchestration and use distinguished fields on a message: XmlDoc objects load the full message into a DOM and consume a considerable amount of memory resources. Each transformation from an XmlDoc
to a message object and back results in copies of these objects being
created in memory. Using distinguished fields simply references the
existing message and minimizes memory churn.
Move data validation to the pipeline or schema:
If you are performing any data validation on your input messages, for
example, validating that an order number is numeric and within a
prespecified range, you are better off specifying this form of
validation when defining the data types in the schema. If you require
other forms of contextual validation, you are better off doing that in a
pipeline. It is better to do this before persisting the message to the
Messagebox, running the routing logic to find the proper schedule
subscribed to the message instance, and then spawning an orchestration
instance and allocating the required resources for it to run, only to
realize that the data was not good enough to start with and issues an
exception or sends back an error. You can save a lot of system resources
and processing time by handling this in the pipeline and generating an
error message there that you can then route back to the sender using
content-based routing.
Avoid using orchestrations for routing:
If all your orchestration is doing is checking message fields to route
the message to the proper handler or another worker orchestration to
perform a particular task, strongly consider redesigning that piece of
your application as a set of port filters. Leverage receive ports with
multiple receive locations and send groups to route messages coming in
from multiple sources and send messages to multiple destinations instead
of relying on orchestrations to achieve that.
Avoid
calls to external assemblies that perform extensive processing,
especially if they call web services or make calls to a database: Avoid calling slow external assemblies from within your orchestrations.
This holds the processing host resources and valuable host threads from
servicing other orchestration instances while waiting for that external
logic to terminate and return. If these calls stall or take a
considerable amount of time, there is no way for the BizTalk engine to
dehydrate that orchestration instance and use the resources assigned to
it to service another one, because the engine sees the instance's state
as running while in fact the external code that it called is idle.
Leverage the messaging infrastructure to issue such calls that span
multiple processes boundaries.
Do not wrap calls to .NET objects in atomic transactions because they are nonserializable:
Do not create transaction scopes around an expression to simply get
around the shortcoming of an external object that you are using. If it
makes sense, make this object serializable, or if it is simply a
utility, use static methods instead of instantiating an object. Change
the class's implementation or implement a façade that provides you with
the required interface if needed.
Use Parallel shapes carefully: As illustrated in Figure 9,
Parallel shapes should be used to parallelize receives if the order of
incoming messages is unknown. The cost of persistence points associated
with Parallel shapes is high.
Differentiate between scopes and transactions:
Transaction scopes affect persistence points. Atomic transactions batch
state persistence points and write them to the database in a single
call. Long-running transactions, on the other hand, persist state at
different points along the process. If you are not really running a
transaction, do not assign a transaction type to your scope.
Use pass-through pipelines where possible:
The XMLSend or XMLReceive pipelines do a fair amount of work to
validate and assemble the data going through them. If you are sure that
outgoing and incoming messages are in a valid XML form, use a
pass-through pipeline to eliminate this unneeded overhead.
NOTE
The pass-through
pipeline does not promote any message properties. You might still want
to use the XMLReceive pipeline if you want to access promoted
properties.
Clean up suspended messages:
Suspended messages are held in the suspended queue and thus retain an
entry in the Messagebox spool table. An unexpected growth in the spool
table affects overall performance, because all instances added to,
changed, or removed from the system touch the spool table. It is
therefore wise to help maintain as small a size as possible for the
spool table. Suspended-resumable instances have a worse impact on the
system because they are considered by the engine to be dehydrated
messages on a server recovery from an unexpected failure and are
rehydrated back to memory, consuming valuable threads and resources.
Leverage exception handlers in orchestrations and subscribe to negative
acknowledgments or use a cleanup script to clear your system from
suspended instances regularly only after verifying that the data is
nonrepairable or unnecessary in the target system.
Regular code reviews
during the development cycle should ensure that these guidelines are
being followed and save the development team valuable time spent in
performance testing and bug fixing to meet performance requirements.
The following script can be used
to periodically clean up suspended messages. Add it to your task
scheduler on the server to run periodically—every day or couple of
days—and clean up suspended messages. The script needs to be configured
with the different hosts to inspect and clean up. The approved way of
cleaning up suspended messages programmatically is using the WMI APIs
exposed by BizTalk, which is unfortunately a resource-intensive way to
perform this task. To avoid locking up the server, the script cleans up
instances in batches. The number of instances in a batch—nMaxInstancesToClean—as well as the hosts—aryHostNames—to clean up suspended instances for and the log file location—strDirectory and strFile—can
be easily modified. The script also generates and appends data to the
same log file with each run for administrators to track its progress and
keep track of cleaned-up messages. Enter your host names in order of
priority, such that the one that needs to be cleaned first will run
first.
dim objServices, objMsg, svcinsts, inst, msg, ndx, size, nHostCount
Dim aryClassIDs()
Dim aryTypeIDs()
Dim aryInstanceIDs()
Dim aryClassIDsTemp()
Dim aryTypeIDsTemp()
Dim aryInstanceIDsTemp()
'-----------------------------------------------
Dim strKey2Instance
Dim strQuery2Msg
Dim objQueue
'-----------------------------------------------
'-- Creating and opening log file to append info
'-----------------------------------------------
Dim strDirectory, strFile, objFSO, objTextFile
'Create the File System Object
Set objFSO = CreateObject("Scripting.FileSystemObject")
'Check that the folder and file exists
If objFSO.FileExists(strDirectory & strFile) Then
'OpenTextFile Method needs a Const value
'ForAppending = 8 ForReading = 1, ForWriting = 2
Const ForAppending = 8
Set objTextFile = objFSO.OpenTextFile(strDirectory & strFile,
ForAppending, True)
Else
Set objTextFile = objFSO.CreateTextFile(strDirectory & strFile)
End If
'---------------------------
'-- Set the object reference
'---------------------------
set objServices = GetObject("winmgmts:\\.\root\MicrosoftBizTalkServer")
'--------------------------------------------------------------------
'-- Create array of hostnames to loop over hosts
'-- increase the array's size,
'-- add as many hostnames as required to the array,
'-- and initialize the corresponding aryHostSuspendedMessages entries
'--------------------------------------------------------------------
'--------------------------------------------------------------------------
'-- The maximum number of suspended instances to clean up in a single batch
'--------------------------------------------------------------------------
Dim strHost
nHostCount = 0
objTextFile.WriteLine("------- SCRIPT EXECUTION STARTED -------")
'--------------------------------------------------------------------
'-- Terminate instance in each suspended Queue for the selected hosts
'--------------------------------------------------------------------
for each strHost in aryHostNames
'wscript.echo "Host: " & strHost
objTextFile.WriteLine("----------------------------------------")
objTextFile.WriteLine(Now() & " : Host: " & strHost)
'--------------------------------
'-- Query for Suspended instances
'--------------------------------
wbemFlagReturnImmediately = 16
wbemFlagForwardOnly = 32
IFlags = wbemFlagReturnImmediately + wbemFlagForwardOnly
set svcinsts = objServices.ExecQuery(
"select * from MSBTS_serviceinstance
where (servicestatus=32 or servicestatus=4)
and HostName="""&strHost&"""",, IFlags)
'Using a semi-synchronous therefore the count property doesn't
'work with wbemFlagForwardOnly
'size = svcinsts.Count
'wscript.echo "Suspended Message: " & size
strKey2Instance = "MSBTS_HostQueue.HostName=""" & strHost & """"
set objQueue = objServices.Get(strKey2Instance)
If Err <> 0 Then
wscript.echo Now() & " : Failed to get MSBTS_HostQueue instance"
wscript.echo Now() & " : " & Err.Description & Err.Number
objTextFile.WriteLine(Now() & " :
Failed to get MSBTS_HostQueue instance")
objTextFile.WriteLine(Now() & " : " + Err.Description & Err.Number)
Else
ndx = 0
redim aryClassIDs(nMaxInstancesToClean)
redim aryTypeIDs(nMaxInstancesToClean)
redim aryInstanceIDs(nMaxInstancesToClean)
'Loop through all instances and terminate nMaxInstancesToClean at a
'time.
'This number was choosen for optimization, so it can be changed if
'desired.
for each inst in svcinsts
If ndx > nMaxInstancesToClean Then
'Currently 500 entries are ready to be terminated
'wscript.echo "Attempting to terminate "
'& ndx & " suspended instances in host"
objTextFile.WriteLine(Now() &
" : Attempting to terminate "
& ndx & " suspended instances in host")
objQueue.TerminateServiceInstancesByID aryClassIDs,
aryTypeIDs, aryInstanceIDs
If Err <> 0 Then
wscript.echo Now() & " : Terminate failed"
wscript.echo Now() & " : " & Err.Description
& Err.Number
objTextFile.WriteLine(Now() & " : Terminate failed")
objTextFile.WriteLine(Now() & " : " + Err.Description
& Err.Number)
Else
'wscript.echo "SUCCESS> " & ndx &
'" Service instance terminated"
objTextFile.WriteLine(Now() &
" : SUCCESS> " & ndx &
" Service instance terminated")
End If
'Reinitialize the arrays and counter
'to ensure we store non-terminated
'Entries for the next round of termination
ndx = 0
redim aryClassIDs(nMaxInstancesToClean)
redim aryTypeIDs(nMaxInstancesToClean)
redim aryInstanceIDs(nMaxInstancesToClean)
'Suspends script execution for 30 seconds,
'then continues execution
'wscript.echo "Suspending script execution for 30 seconds"
objTextFile.WriteLine(Now() &
" : Suspending script execution for 30 seconds")
Wscript.Sleep 30000
End If
aryClassIDs(ndx) = inst.Properties_("ServiceClassId")
aryTypeIDs(ndx) = inst.Properties_("ServiceTypeId")
aryInstanceIDs(ndx) = inst.Properties_("InstanceId")
ndx = ndx + 1
next
'If count <> zero then the arrays are still populated
'and the messages need to be terminated one last time.
If ( ndx > 0 ) then
redim aryClassIDsTemp(ndx-1)
redim aryTypeIDsTemp(ndx-1)
redim aryInstanceIDsTemp(ndx-1)
for i=1 to ndx
aryClassIDsTemp(i-1) = aryClassIDs(i-1)
aryTypeIDsTemp(i-1) = aryTypeIDs(i-1)
aryInstanceIDsTemp(i-1) = aryInstanceIDs(i-1)
aryHostSuspsendedMessages(nHostCount) =
aryHostSuspsendedMessages(nHostCount) + 1
next
'wscript.echo "Attempting to terminate " &
'ndx & " suspended instances in host"
objTextFile.WriteLine(Now() &
" : Attempting to terminate " & ndx &
" suspended instances in host")
objQueue.TerminateServiceInstancesByID aryClassIDsTemp,
aryTypeIDsTemp, aryInstanceIDsTemp
If Err <> 0 Then
wscript.echo Now() & " : Terminate failed"
wscript.echo Now() & " : " & Err.Description & Err.Number
objTextFile.WriteLine(Now() & " : Terminate failed")
objTextFile.WriteLine(Now() &
" : " + Err.Description & Err.Number)
Else
'wscript.echo "SUCCESS> " & ndx &
'" Service instance terminated"
objTextFile.WriteLine(Now() &
" : SUCCESS " & ndx & " Service instance terminated")
End If
Else
'wscript.echo "No suspended instances in this host"
objTextFile.WriteLine(Now() &
" : No suspended instances in this host")
End If
End If
nHostCount = nHostCount + 1
Next
objTextFile.WriteLine("")
objTextFile.WriteLine("------- SUMMARY START -------")
nHostCount = 0
for each strHost in aryHostNames
wscript.echo Now() & " : Total of " &
aryHostSuspsendedMessages(nHostCount) &
" suspended messages were terminated in " &
strHost & " host"
objTextFile.WriteLine(Now() & " : Total of " &
aryHostSuspsendedMessages(nHostCount) &
" suspended messages were terminated in " &
strHost & " host")
nHostCount = nHostCount + 1
Next
objTextFile.WriteLine("------- SUMMARY END -------")
objTextFile.WriteLine("")
wscript.echo Now() & " : Script Execution Completed"
objTextFile.WriteLine(Now() & " : Script Execution Completed")
objTextFile.WriteLine("------- SCRIPT EXECUTION ENDED -------")
objTextFile.Close
Set objTextFile = nothing
A sample log file output with multiple batch runs to clean the hosts:
------- SCRIPT EXECUTION STARTED -------
----------------------------------------
08/11/2009 3:42:31 PM : Host: BizTalkServerApplication
08/11/2009 3:42:37 PM : Attempting to terminate 500 suspended instances in
host
08/11/2009 3:42:43 PM : SUCCESS> 500 Service instance terminated
08/11/2009 3:42:43 PM : Suspending script execution for 30 seconds
08/11/2009 3:43:13 PM : Attempting to terminate 162 suspended instances in
host
08/11/2009 3:43:14 PM : SUCCESS> 162 Service instance terminated
----------------------------------------
08/11/2009 3:43:14 PM : Host: BizTalkServerIsolatedHost
08/11/2009 3:43:15 PM : No suspended instances in this host
------- SUMMARY START -------
08/11/2009 3:43:25 PM : Total of 662 suspended messages were terminated in
BizTalkServerApplication host
08/11/2009 3:43:30 PM : Total of 0 suspended messages were terminated in
BizTalkServerIsolatedHost host
------- SUMMARY END -------
08/11/2009 3:43:32 PM : Script Execution Completed
------- SCRIPT EXECUTION ENDED -------
