Hybrid integration is a hot topic. With the rise of Software as a Service (SaaS), businesses and their current infrastructure will have to rise and change with the API-enabled and connected enterprises of today.
The setup
We are going to build a cloud hosted API that is going to connect to BizTalk and offer a REST-endpoint that is securely accessible over internet. To manage this, we will set up an API Management Service that will expose an endpoint. The messages received on this endpoint will be sent over a Service Bus Queue where BizTalk will pick them up with the SB-Message adapter, authenticated by SAS (Shared Access Signature).
The Service Bus Queue will pose a problem if we want to extend this feature. The maximum size of messages is based on the chosen pricing tier. The Standard (and Basic) tier allow you to send messages of over 256 KB while the Premium tier allows messages of up to 1024 KB. Double the size, 60 times the price (give or take) and still a limitation.
To keep our solution compatible with larger messages, we are going to determine the size of the message in the policy and change routing dependent on that.
Now, as we can’t send the message over another Service Bus Queue, we will use Logic Apps, enhanced by Azure Functions, to store the message on Blob Storage and send a pointer over another Service Bus Queue.
We will start by setting up the upper flow. Creating an endpoint that is accessible from outside and will flow a message over our queue to BizTalk.
As our virtual API will be forwarding the requests towards the queue and BizTalk will be listening to this queue, we will set this up first. Then we will implement the SB-adapter to receive these messages and FILE drop them. Finally, we will implement the API management service by adding a new API, pointing towards the already generated queue.
Header: Content
The queue
From the Azure portal, add a new Service Bus.
This will prompt you to create a new Service Bus namespace. Choose a name, pricing tier, resource group and location for the new namespace and click Create. Provisioning will take about a minute.
Go to your newly created Service Bus and add a new queue for it, either by clicking the plus symbol or by navigating to the Queues tab.
Give the queue a name and be sure to disable partitioning for BizTalk as they don’t play nicely with each other.
On the newly generated tab, you are able to manage the newly created bus. Go to Shared Access Policies and add a new one to enable posting to and from the queue.
You can manage the roles here to enable sending, listening and/or managing (which will automatically enable listening and sending as well). Create a policy to enable listening and one to enable posting.
The detail view of these policies allows you to update them by adding or removing claims. They also hold the connection information in the following form:
Endpoint=sb://vandooren.servicebus.windows.net/;SharedAccessKeyName=API_Send;SharedAccessKey=xxxxx =;EntityPath=biztalkbroker
You will need this information later on for both policies.
The Bus Service Queue is now completely set up and ready to receive and send messages.
Listening to a Bus Service Queue from BizTalk
In this section, we will set up a listener on BizTalk. We will set up one but you could set one up for every API-endpoint you have. As Queues have the At-Most-Once delivery guarantee, at least and at most one of your listeners will receive the message and process it.
To start, create a new Receive Port and accompanying Receive Location in BizTalk. Pick “SB-Messaging” as the Transport Type and click Configure.
In the Queue URL, paste your endpoint and entity path from the connection string we saved earlier from the Shared Access Policies. Proceed to the Authentication tab.
Click the Shared Access Signature radio button and provide the Shared Access Key Name and Shared Access Key from the connection string.
Now create a send port that, for the sake of convenience, will drop the message as a file. Enable your receive location and send port and BizTalk is ready.
Testing the queue connection
For posting to the queue, we will need to pass some information in the headers. Keep this information close by as the APIM will also need this information to be able to post to the Queue.
| Header Key | Value |
| Authorization | See below |
| Content-Type | application/atom+xml;type=entry;charset=utf-8 |
| BrokerProperties | {} |
The “Content-Type” header must have the value of displayed above.
A BrokerProperties JSON-encoded string is also required. This Microsoft article shows an overview of the properties you can pass along.
The Authorization header requires a Shared Access Signature (SAS) based upon the earlier created Write Shared Access Policy.
Rene Brauwers has covered this in one of his blog posts while providing a small forms application that will generate the Authentication token. The Microsoft documentation offers more information on how to generate the token yourself.
The generated value will look something like this:
SharedAccessSignature sig=xxxxxxxxxxxx&se=xxxxx&skn=API_Send&sr=https%3a%2f%2fvandooren.servicebus.windows.net%2fbiztalkbroker
Now use your favorite API-tool to send a POST to your endpoint. Important side note about the endpoint is that your Queue has to be accessed by its URL/messages.
In this example it would be: https://vandooren.servicebus.windows.net/biztalkbroker/messages
If all is successful, you will receive a 201 Created Status Response. The message itself should now be on your FILE drop location.
If not, check your response code. A 400 status code means that there is an error in your request itself, the URI, Content-Type header or BrokerProperties are most likely. A 401 could mean that your token is invalid or that you have the wrong token.
If the response code is fine, check your Windows Application Log on your BizTalk Server.
Tips
- Do not enable partitioning on your queues. Partitioning enables Azure Service Bus to spread the messages out over several brokers instead of one, enabling higher throughput of messages. As of now, BizTalk has not been updated to integrate with this feature.
Conclusion
We have laid out the structure of our hybrid integration service and set up the first part of brokered messaging between the cloud infrastructure and the on premise infrastructure. This endpoint is secured by a SAS-key which only we have access to.
The next part will look at setting up the API Management Service and leveraging it to correctly send incoming requests to the Queue.
Happy coding!
Jochim
