Partitioning the BizTalk solution
The structure of a BizTalk solution can be critical to its success and ease the development, deployment, and maintenance efforts required over its lifetime. In the years I have been developing BizTalk solutions, I have happened upon many tips that make the process much easier and many are related to solution structure and the build/deployment process. This section will describe some of these tips in detail and provide guidance for architects and developers.
In the context of a BizTalk solution, structure carries the dual role of enforcing the isolation of components through separation of concerns/loose coupling and organizing artifacts into manageable pieces that can allow for varying rates of change. A poor solution structure will allow changes to propagate uncontrolled throughout the solution and result in a more haphazard spaghetti code approach; this is a bad thing for any project on any platform. It will also slow development and the rate of change while increasing costs for both development and maintenance. Ultimately it can even lower morale and result in a less successful solution outcome and a perception that BizTalk and the team have fallen short of expectations.
The guidance for solution structure that follows will help you create a successful BizTalk solution by avoiding the common pitfalls many make with the product. (I know. I made them. This book exists to help you avoid them.)
Specifying the requirements of solution structure
Before we delve into how to structure your solutions, we should outline the basic goals we are trying to achieve from a development perspective. The solution structure for all but the most trivial BizTalk applications must support:
- Multiple concurrent developers (and robust source control)
- Building on any developer workstation
- Testing on any developer workstation
- Test-driven development (TDD)
- Automated functionality testing
- Continuous integration
- Automated performance testing
These bullets are all important for different reasons and all manifest themselves in different ways in the solution structure. If your application centers around one large orchestration with all the logic enclosed in a single giant file then only one developer at a time will be able to make changes to it safely. You could try shared checkout, but I'm not sure what results you'll receive when it comes time to merge. Newer tools are meant to merge these files better, but I am still not convinced. BizTalk wasn't meant to be used that way anyhow, good BizTalk solutions have more smaller parts rather than fewer larger ones. Furthermore, as in traditional development, large monolithic structures are not very easy to work with. They are one of the code smells that Martin Fowler wrote about nearly a decade ago. This is certainly an area where BizTalk and traditional software development have much in common.
Being able to easily build the entire solution on any developer workstation, and test it, is also critical. Every developer should be able to see the solution work, on their own, in a private environment where they can feel free to break things or tear things apart, as well as simply debug issues. There are several other reasons for this, including support, loss of the build server, and emergency fixes.
Importantly, component and functionality testing must be automated and should be done early. After only a few manual test runs you would already have saved time by creating automated tests at the outset. Further, automated functional testing reinforces the idea that first you identify the problem, then you devise the solution. This is the core of TDD and is a great fit with BizTalk. These tests should provide self-documentation for the solution so that a new developer can see what the solution is meant to do and then watch it work. If you take TDD to its zenith then you will build your tests first and they will simply fail until the solution works.
Once these tests are in place and working with your first pass solution, you are free to go change the internal implementation with the confidence that you are still solving the problem at hand in a predictable and expected manner. As solutions get more complex, this becomes even more imperative, as it is easy to lose details. We'll cover unit testing later, but I'll leave it now by saying all expected behavior, including error handling, should have automated coverage. It is simply unacceptable in BizTalk to do this any other way. Manual testing is a recipe for disaster as changes in one part of the solution may introduce errors in another. Manual testing makes finding these errors almost impossible and requires far more time than simply automating tests.
Continuous integration (CI) aids us further by allowing full test suites to be run after every developer checks in. This becomes crucial in larger solutions as the decoupling BizTalk encourages carrying a price that some errors will not be found until runtime. I have worked on solutions where one developer's changes in a completely separate application have caused failures across many applications. Using the same tests from TDD on your CI environment can greatly reduce the chance of side effect issues.
Finally, automated performance or stress testing provides insurance that your solution will scale and run as expected with realistic loads. Many who are new to BizTalk create solutions that will work perfectly well on their developer machine running a few transactions, but fall apart when they encounter production loads. Sadly this is often in production environments. This embarrassment is unnecessary and completely avoidable.
Poorly planned solution structure will restrict all of the abilities listed previously and can make the platform feel more like a prison than a liberator.