Dynamic WCF endpoint in BizTalk 2006 R2 with SSL and client certs

I had a requirement to submit a message to a WCF web service, but the target endpoint was dependent upon a value supplied in the initiating message. This work was done within an orchestration. The solution required the web service request to be sent over SSL, using client certificates for message authorisation.

The target URL was promoted as a distinguished property (called Url) in a message "CallbackUrl".

I had already consumed the target WCF service in my BizTalk project, so in my orchestration I added a new request-response port (MyPort), using the port type generated during that process. However, I configured the port to be dynamic.

In a message assignment shape, I created the message (MyMsg) to submit to the service. Then I configured the dynamic WCF bindings as below:

-------------------------------------------------------------------------------------
MyMsg(WCF.BindingType) = "basicHttpBinding";
MyMsg(WCF.EndpointBehaviorConfiguration) = "<behavior name=\"EndpointBehavior\"><clientCredentials><clientCertificate findValue=\"requiredCertName\" storeLocation=\"CurrentUser\" storeName=\"My\" x509FindType=\"FindBySubjectName\" /></clientCredentials></behavior>";
MyMsg(WCF.BindingConfiguration) = "<binding name=\"basicHttpBinding\" closeTimeout=\"00:05:00\" openTimeout=\"00:05:00\" sendTimeout=\"00:05:00\" maxBufferSize=\"650536\" maxReceivedMessageSize=\"650536\"><security mode=\"Transport\"><transport clientCredentialType=\"Certificate\" /><message clientCredentialType=\"Certificate\" /></security></binding>";
MyMsg(WCF.OutboundBodyLocation) = "UseBodyElement";
MyMsg(WCF.InboundBodyLocation) = "UseBodyElement";
MyMsg(WCF.PropagateFaultMessage) = true;
MyMsg(WCF.SuspendMessageOnFailure) = true;


MyPort(Microsoft.XLANGs.BaseTypes.Address) = CallbackUrl.Url;
MyPort(Microsoft.XLANGs.BaseTypes.TransportType) = "WCF-Custom";
------------------------------------------------------------------------------------
If you start with a working static port, you can export the bindings for the port and use the XML to populate the WCF.EndpointBehaviorConfiguration and WCF.BindingConfiguration properties.

In reality, the client certificates required were different for each target URL, so the certificate name was held against the URL in a config file and read in using a helper class. But to make the sample code above more straightforward, I've just hard-coded the certificate name.

The key to getting this working is that the public and private keys of the client certificate must be accessible by the account the BizTalk host instance runs under - if you're using a self-signed cert for testing. For production, the BizTalk host instance just needs access to the public key. Log on to the computer as the host account and import the certs into the "personal" folder in MMC.

Getting the font right in HTML emails (for fussy clients)

It's been a while since I wrote any HTML in anger, so I'm probably a bit behind the times... The other day a client insisted that all comms sent to them were in verdana, 10pt. This included a notification email sent from BizTalk. To force Outlook to display the font as verdana 10pt, setting the font in an old-school font tag didn't do the trick. Using a span tag instead solved the problem:


span style='font-family:\"Verdana\",\"sans-serif\";font-size:\"10pt\"'

Unit Testing Peskiness

Long time, no post, but I'm back. At least for this one note to self that I know I'll forget later.

So, let's say you're creating unit tests in VS2010 with MSTest. Let's also say that you're checking your code coverage along the way to watch your progress, and let's also say that there's always a pesky percentage point or 2 that comes up as Not Covered and just greatly inhibits the joy you'd like to be experiencing by seeing 100% code coverage.

What's more, let's say that those uncovered portions are always in Visual Studio generated code for 'MySettings', and you're not using 'MySettings', so you have no need to test 'MySettings'.

There is a handy-dandy attribute that can exclude bits from code coverage calculations, called logically enough: System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute().

Idle Friday afternoon ranting...

We did mention this blog would feature food, grief management and various assorted oddities as well as software development so...

Curiously (and it's okay, I am planning to see a doctor), I am in no mood for chocolate. Even after a 13 hour marathon yesterday related to an expired root certificate:

Customer: Send me the certificate you're talking about
Me:
Customer: Try this one
Me: Er, this looks the same as the one I just sent you that doesn't work
Customer: Yes
Me: Errrr.....
(2000 unsent messages, 13 hours, 10 people and some hysterical laughter - on my part - later... workaround agreed on and implemented until root cert renewed)

Anyways, I can attest to the fact that Kokako bliss balls are quite a tasty, healthy alternative to chocolately treats. They have dates, figs, maple syrup, cocoa, sunflower seeds... The kind of stuff that would make Ross roll his eyes and reach for a two litre bottle of Coke (although his eyes did unglaze briefly at the mention of maple syrup and cocoa... well, they would have... if he was around to amuse me for the remainder of this work day... Probably too busy drinking tequila in Mexico - the world IS unfair, kids...) Roll on the weekend!

XSLT namespace fun in scripting functoids

Another thing I frequently forget...

To find the namespace prefix to use in a scripting functoid in a map, right click on the map, select "validate map" and view the resulting xsl file. The opening stylesheet tag should show you which namespace prefixes apply to each input message (eg. s0 / s1).

If you're using aggregate schemas (particularly if you changed an existing map to use an aggregate input schema), you may now need to include the root node name in your XSLT. For example, if you have a message

<?xml:namespace prefix="ns0" /><ns0:request ns0="http://myschema">
<ns0:order>
<ns0:id>1234</ns0:id>
</ns0:order>
</ns0:request>

and you previously referenced the ID element using s0:order/so:id

you may now need to reference it using //s0:Request/s0:order/so:id

VB IsDate equivalent in C#

Now I'm living in a C# world... and my memory fails me every time I need to do this (which frankly, says worrying things about my memory)... DateTime.TryParse returns a boolean indicating whether the input string can be converted to a datetime or not. So you can write a wee function like the following:

public Boolean checkDate(string inputDate)
{
Boolean retValue = false;
DateTime outputDate;
if (DateTime.TryParse(inputDate, out outputDate))
{
retValue = true;
}
return retValue;
}

ESB Toolkit and UDDI: Binding a UDDI service to SQL

Maybe I was just blind or stupid when I needed to do this, but it took me a while to find some info on how to set up bindings for SQL in UDDI. So I'm noting these down for next time...

Context: Building a solution using BizTalk 2009 and the ESB Toolkit. I need to call a SQL stored procedure to return price data for a given product. Schemas have already been created using the WCF LOB adapter. Now I want to use a UDDI resolver to configure my off-ramp.

1. Browse to the publish tab in UDDI
2. Right click on the desired provider and choose "add service"
3. Either specify a key or allow the system to generate one
4. Edit the service name (eg. GetPriceData)
5. Right click on the service (or click on the bindings tab) and add a binding
6. Configure the endpoint - instead of an http service, we have an mssql service, so the endpoint will be in this format: mssql://databaseservername//databasename?
7. Configure the transport type: Add a category and select the scheme microsoft-com:esb:runtimeresolution:transporttype. Choose a value of WCF Custom Binding.
8. Configure the BizTalk application: Add a custom category and search for tModels using %biztalk%. Select microsoft-com:esb:runtimeresolution:biztalkapplication. Type "BizTalk Application" for the key name and the name of the your BizTalk application container (this is in your admin console) for the key value.
9. Add instance info (search for tModels containing %esb%) - you need to add the following:

• microsoft-com:esb:runtimeresolution:transporttype (use WCF-Custom, by which I mean set the instance parameter to WCF-Custom)
• microsoft-com:esb:runtimeresolution:targetnamespace (target namespace of your SQL schema - in my case http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo )
• microsoft-com:esb:runtimeresolution:action (the correct action is in your schema, if you created it using the WCF LOB adapter - in my case it was {TypedProcedure/dbo/GetPrice} - note that you need to enclose the action in curly braces)
• microsoft-com:esb:runtimeresolution:messageexchangepattern (use "Two-Way", if you're sending a request and expecting a response)
• microsoft-com:esb:runtimeresolution:jaxrpcresponse (set to "false")
• microsoft-com:esb:runtimeresolution:cachetimeout (set to "-1")
• microsoft-com:esb:runtimeresolution:endpointconfig (set to "BindingType=sqlBinding")

To get the key to use in your resolver, click on the service name. On the details tab, click the "more details" link on the right. This brings up the UDDI keys. Copy the value for the version 3 key into the "service key" property for the UDDI resolver in your itinerary.

Document structure validation and routing failed messages

A nice way of validating the document structure of an incoming message is to use a receive pipeline with both a disassembler component (which validates the structure) and a validation component (which apparently validates the message against any restrictions in the schema).

Configure your receive location to use this pipeline and check "enable routing for failed messages" on the receive port.

Now you can set up a send port (or whatever you need to handle invalid messages), which subscribes to messages with an ErrorReport.ReceivePortName = your receive port, ErrorReport.ErrorType = 'FailedMessage' and ErrorReport.FailureCategory = (you may need to do some testing to see what the appropriate categories are - the ErrorReport options are promoted properties on the failed message) and send an email alert or set up an error-handling orchestration etc.

Sending flat file attachments with an HTML email from BizTalk

I got this working in two different ways, both from within orchestrations. In both examples, I only ever needed to attach one file per email. However, I think the second example could be easily modified to work with multiple attachments by adding more parts to the multi-part message type. I haven't yet tried using SMTP.Attachments to add multiple attachments (in the first example), but it could be worth a try.

Example 1:
The requirement here was to email a file to the client upon dispatch of an order batch. To do this, I archived the dispatch file (in my case, this was an Excel file) out to a specific file directory using a FILE send port early in the orchestration (as file archiving was a requirement anyway). I used a dynamic send port, so I could set the file name and use this to reference the file later. Note that in the orchestration, I also set up a loop with a delay shape to check for the existence of the archived file before creating the email (in case the send port had been disabled or something else went wrong). To do this, I used a helper class which returned the boolean value of System.IO.File.Exists(path)- where path is a string variable equal to the archive file directory path and file name.

I created a multi-part message type for the email. It only had one part of type RawString (Microsoft.Samples.BizTalk.XlangCustomFormatters.RawString - sample code for this is in the SDK).

In a message assignment shape, I built the body of the email (usually I'd have HTML tags in the string variable for the body, but it's a pain to publish them here) using a string variable responseText and message ProvDispatch (an instance of the multi-part message type) eg.


responseText = "This is to confirm an order requiring provisioning has been dispatched.";

ProvDispatch.ResponseBody = new Microsoft.Samples.BizTalk.XlangCustomFormatters.RawString(responseText);
ProvDispatch.ResponseBody(Microsoft.XLANGs.BaseTypes.ContentType) = "text/html";
ProvDispatch(SMTP.EmailBodyFileCharset) = "UTF-8";

In the same shape, I then added the Excel file created earlier in the orchestration as an attachment. The file path was stored in a string variable "reportPath" and the file name was stored in a string variable "attachmentName".

ProvDispatch(SMTP.Attachments) = reportPath + attachmentName + ".xlsx";

Then set the subject and email addresses (if required):

ProvDispatch(SMTP.Subject) = "Dispatch Notification";
ProvDispatch(SMTP.From) = emailVariable;

I needed to use dynamic addressing for the outbound email, so I then configured my send port (DispatchResponse_Port) with the email to address, stored earlier in the orchestration in a string variable "provEmail":

DispatchResponse_Port(Microsoft.XLANGs.BaseTypes.Address) = "mailto:" + provEmail;

Your port should be configured to use the PassThru pipeline.

This example produces an email with a formatted HTML body "This is to confirm an order requiring provisioning has been dispatched" with the required file attached in the correct format with the correct name. For example, if the variable "attachmentName" above had a value "TestFile", the attachment on the email would be "TestFile.xlsx".

Example 2:
In this example, the client was sending an order file which needed to undergo some validation checks. If the file failed validation, it needed to be emailed back to the client as an attachment, with a list of validation errors in the body of the email.

The order file was in CSV format, which was being mapped to a common order schema on the receive port. So, if data validation checks failed, I needed to recreate the CSV file the customer sent in order to email it back to them.

To do this, I:

1. Created a map, which mapped the contents of the common order message to the CSV order message
2. Created a send pipeline "MyFF_SPipe", using a Flat File Assembler (document schema = CSV order file schema)
3. Created a receive pipeline "MyXDoc_RPipe", with nothing in the disassemble stage of the pipeline. This will be used to receive any document in a message variable of type System.XML.XmlDocument (which preserves the format of the inbound message - ie. does not convert it to XML)
4. In the orchestration, created a message FFRequest (type = CSV order file schema). Constructed this message using the map file created in step 1.
5. Created a message FFRequestOut (type = CSV order file schema)
6. Created an atomic scope and defined a variable "inputMsgs" of type Microsoft.XLANGs.Pipeline.SendPipelineInputMessages (add a reference to Microsoft.XLANGS.Pipeline.dll to the project) and a variable "pipelineMsgs" of type Microsoft.XLANGs.Pipeline.ReceivePipelineOutputMessages
7. To create the CSV file, in an atomic scope, I executed the send pipeline created in step 2 from a message assignment shape in a construct block (message constructed = FFRequestOut), using the following code:
   FFRequestOut = null; inputMsgs.Add(FFRequest); Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteSendPipeline (typeof(MySolution.pipelines.MyFF_SPipe) ,inputMsgs ,FFRequestOut);
8. Called the receive pipeline MyXDoc_RPipe created in step 3 from an expression shape using the following code:
   pipelineMsgs = Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteReceivePipeline(typeof(MySolution.pipelines.MyXDoc_RPipe),FFRequestOut);
9. In a construct block, assigned the first message (there should only be one) in pipelineMsgs to a message called "Attachment" of type System.Xml.XmlDocument using the following code:
   pipelineMsgs.MoveNext();
   Attachment = new System.Xml.XmlDocument();
   pipelineMsgs.GetCurrent(Attachment);
10. Created a multi-part message type for the outbound email with two parts - BodyPart of type Microsoft.Samples.BizTalk.XlangCustomFormatters.RawString and Attachment of type System.Xml.XmlDocument
11. In a message assignment shape in a construct block (constructing a message "FailResp" with the multi-part message type created in step 10), build the message:
    responseText = "File rejected for the following reasons...."; FailResp.BodyPart = new Microsoft.Samples.BizTalk.XlangCustomFormatters.RawString(responseText); FailResp.BodyPart(Microsoft.XLANGs.BaseTypes.ContentType) = "text/html"; FailResp.Attachment = new System.Xml.XmlDocument(); FailResp.Attachment = Attachment; FailResp.Attachment(MIME.FileName) = fileName; FailResp(SMTP.MessagePartsAttachments) = 2; FailResp(SMTP.Subject) = "Order file REJECTED"; FailResp(SMTP.EmailBodyFileCharset) = "UTF-8"; FailResp(SMTP.From) = emailFrom;
    * emailFrom is a string variable, as is fileName (which simply sets the attachment name to something sensible eg. RejectedOrderFile.csv)
12. I'm using a dynamic send port again, so I need to also configure the email to address (see example 1). The send port is also configured to use the PassThru pipeline.

Setting up Host Names

Again, this is just one of those things that I feel I should remember, but know I never will:
To set up host names, edit the file C:\WINDOWS\system32\drivers\etc\hosts.

It's a simple text file that maps IP addresses to host server names.

XML Editor Just Wasted My Day

I can't believe this happened. And yet I can.

Try this:
Copy the following xml snippet into a file and save it with an ".xml" extension:

<rowset>
<row>
<blah>OPN                                     </BLAH>
</ROW>
</ROWSET>

If you open that file in e.g. Notepad++, then you see it as-is.

If you right-click on that file, select "Open With" -> "XML Editor". Now what do you see?

- <rowset>
- <row>
<blah>OPN</BLAH>
</ROW>
</ROWSET>

No padding spaces. 37 missing spaces to be exact.

Now, let's say one is troubleshooting a Biztalk map that it throwing output validation errors because a field exceeds the defined 20 character MaxLength. The input file looks fine (in XML Editor). "Why would Biztalk just pad this thing out to 40 characters?"

Would you say that it would be useful to know that the input file has 37 trailing spaces after the text?

How much time, would you say, one could waste Googling and trying various oddball things trying to get rid of this mysterious series of spaces? My answer to that question is embarrassing.

Not a happy developer today.

Excel Solutions - Gross, but Occasionally Useful

So, once again maybe I'm behind the curve a bit, but I just found the Excel project types in VS2008. So in those cases where I need to hack something in Excel, I can now do it with managed code instead of VBA.

There were, however, a few things to note.
1) Security. The code needs to be Fully Trusted to run. In Admin Tools-> Microsoft .NET Framework 2.0 Configuration, go to Configure Code Access Security Policy. Go to Runtime Security Policy->User->Code Groups->All_Code, and add a new code group. I named mine "Ross Dev", based it on URL (specified my dev folder), and gave it Full Trust. Office projects are automatically given Full Trust, but at some point I had to do this, not sure why-- maybe when I tried to move it out of the original location.

This can also be done with the Caspol.exe commandline tool, but I like the GUI.

b) My app had to make a call to a web service, and it took a while to return data. I got a message about "blah blah blah pumping Windows messages blah blah" and mentioned an MDA ("Managed Debugging Assistant"). To turn this @#$% off, note the name of the MDA, go to the Debug->Exceptions... in VS, and expand "Managed Debugging Assistants"-- untick the trouble maker.

3c) The issue I haven't resolved, and therefore have just accepted, is that even though I created the project as an "Excel 2003" app (because the customer has not upgraded to Excel 2007 yet), when I close VS and re-open, it automatically updates the project to Excel 2007. No questions, prompts, or options-- it just does it. I don't know what will happen if it's put on machine with only Excel 2003 though. And I don't particularly care.

Valrhona Abinao 85% dark chocolate

Now ordinarily I'd consider 85% cocoa well beyond my chocolatey limits, but turns out this Valrhona one is pretty smooth and is doing a fine job of keeping me awake this first day back at work after a long Easter weekend.

Thanks for the Easter gift, boss!

Using xpath in BizTalk orchestrations

Because I always forget the syntax... Below is how to get a count of the number of records from within an expression shape:

{variable name}= xpath({MsgName}, "count(//{MsgNode})");

You can use other xpath functions in place of "count".

You can also write values using this method, but you'd need to do this within a construct block shape.

Biztalk Promoted Properties

Another lovely issue that had me scratching my head: in a schema in Biztalk, I believe that elements that are defined as type "xs:string" have essentially an unlimited length, or at least "unlimited" for any typical purpose that I would come across.

HOWEVER, if that element is specified as a promoted property, then it becomes limited to 256 characters. See "Limitations for Promoting Properties" here.

Sigh.

Re: BizTalk Config Settings

Try creating a helper class that reads the settings from your config file. You should be able to call that from within a message assignment shape in a construct block with no problems.

Ross' Inaugural Post: Biztalk Config Settings

Note to Selves: evidently code that retrieves settings from the Biztalk config file (BTSNTSvc.exe.config) such as:

ConfigSettings = (System.Collections.Specialized.NameValueCollection) System.Configuration.ConfigurationSettings.GetConfig("MySchedule");
My_Dynamic_SP(Microsoft.XLANGs.BaseTypes.Address)= "mailto:" + ConfigSettings.Get("ValidationErrorsEmailTo");

... cannot be used in a Construct Message shape because the NameValueCollection is somehow retrieved as a ReadOnlyNameValueCollection, and that class is not serializable, and that is unacceptable (the transaction type of everything is set to 'None'). Nor, does it seem, can it be done in an Expression shape in a Decision tree for (most likely) the same reason.

I put it in an Expression shape near the beginning of the orchestration, saved the value to a variable, i.e.:

errorEmailTo = ConfigSettings.Get("ValidationErrorsEmailTo");

... and used the variable later on in the Decision tree, and everything works. For now.