BizTalk SOA and Business Process - Day (2)

Sessions

Customer Panel: SOA Success Stories Panel: Ryan Garner, JetBlue Airways, Justin Myrick, Clear Channel Communications & Rodney Turpin, Hewlett Packard Ryan Garner, JetBlue Airways
JetBlue's SOA evolved out of a business driver to standardize partner integration and take advantage of additional inventory channels.

Part of their success was attributed to creating a standards body for service creation. Stressed the importance of tackling areas such as:

• Security
• Health Checking
• Monitoring
• Logging

While JetBlue uses BizTalk it is not heavily used in their SOA implementation. They do use it to orchestrate the composition of multiple services to expose a single service.

They use a purchased product called Service Manager from SOA Software (yes, agreeably confusing) for their web service monitoring and Service Level Agreement (SLA) compliance.

They also were they only member of the panel to discuss setting up a forum for partner discussion and feedback using Community Server. While they seemed to have more partner integration that others of the panel, there was no discussion of partner management (ran out of time). If I can catch Ryan at the conference later I will try to get that question in front of him.

They followed a contract-first, or schema first, approach to their service creation and contribute that as a success factor.

Justin Myrick, Clear Channel Communications
Their SOA also evolved out of a business requirement. Evidently there was a proliferation of ERP systems; essentially one per market....and Clear Channel is in a lot of markets.

Stated Microsoft Technologies:

• BizTalk 2006
  
• Microsoft .NET 2.0
• ASP.NET
  
• Sharepoint
• Reporting Services
• SQL Server 2005
  
• K2
• Peoplesoft
• UDDI
  

They were also given the direction to "portalize Peoplesoft" by the CIO. Effectively targeting a view, or views, of work and business functions to specific users and user groups. Justin never came out and stated it, but this is really composition, or aggregation, of multiple services to present a view to a user. Reminds me a lot of Microsofts efforts around the Composite Application Block (CAB). This portal was developed and rendered using Sharepoint technologies.

Clear Channel used a WSE filter to log message request/response data to a database that was used as a debugging and operational aid.

Rodney Turpin, Hewlett Packard
While the other two panelists had much more recent SOA implementations (less than a year), HP has had an ongoing SOA initiative that has been evolving for several years. HP has something on the order of ~11,000 diseperate applications that needed to be integrated, or their functionality composed into services easily consumed by new systems.

HP is the only panelist to use the Business Activity Monitoring (BAM) feature in BizTalk for monitoring business processes, but its widespread use was admittably "light". Rodney had high regard for using Microsoft Operation Manager (MOM) for BizTalk application server monitoring.

There was an initiative to collapse data centers and their SOA implementation needed to support that.

StatedMicrosoft Technologies:

• BizTalk 2004 + BAM
• Microsoft Operations Manager
  
• Visual Studio Team System (all features: Work Item tracking, version control, etc).
• UDDI
  

In regards to BAM, all panelists plan on incorporating BAM support into their BizTalk implementations. When asked why they didn't implement as part of the first release cycle they all stated it was the time constraint that prevented it.


Avoiding 3 Common Pitfalls in Service Contract Design by Tim Ewald, Foliage Software Systems
Tim had a pretty good session and its unlikely that I am able to capture it all here. There was significant information presented, especially around version strategies, that might take some time to digest.

While approving of a cannonical data model, he discusses why canonical data formats for a business typically fail. They have three major problems that typically manifest as:

1. Too much required data
2. No realistic versioning
3. No system level extensibility

Too much required data
Typically a standards body is formed to defined the canonical data format. This standards body spends time ([n] weeks, months) interviewing business consumers identifying what data is required. This required data makes the schema, and those systems that consume it, brittle.

No realistic versioning
Schemas are typically versioned through the namespace. This will typically break any implementation that consumes it (effectively creating a brand new entity..it just happens to look a lot like the entity it is a revision of).
No system level extensibility
While the standards body is out interviewing different business teams and defining the model, by the time they get a standard, its likely that a business team will need a revision to it. However, because it takes this standards body time to review the change request, there needs to be a way for a system to extend the schema without breaking it. His recommendation is to include an Extension element, which is a ##any element. Effectively allowing a system to stuff anything it needs there. Of course, the receiving system will need to know what to do with it. The thought is that this will allow systems to be 'agile' until the standards body can get through the review process on whether or not to standardize the extension.

****

This begs the question of what to do with the maintenance issue that has just been created. The assumption is that the extension that a system adds might be so useful that it is adopted as part of the canonical data model. Now any new work will be able to take advantage of the new data model. However, what about the implementations that are already baked using the extension? Theoretically, they can continue to use the extension as is because it will 'always' be supported. I can imagine that you would eventually want to synch up with the standard though. This will cause rework on now legacy interfaces. Taking an example from our organization:

1. ERP produces an item update that needs to be published to consumers
2. BizTalk maps the ERP Item into an SCS Item
3. BizTalk then routes to consumers (e.g. Order Entry and WMS)
4. Mary Kay then onboards a new Manufacturing system that needs the SCS Item, but it also needs some additional data that wasn't part of the original SCS Item specification
5. The ERP system is modified to produce the item udpate with the additional data
6. BizTalk maps the ERP Item into the SCS Item, the addition is mapped into an Extension element
7. BizTalk then routes to consumers, original systems continue to use SCS Item as normal (remember, the ##any was still part of the schema and was an addition. If the system is validating, the instance is still valid and the new data is ignored by the original systems)
8. BizTalk will map from the SCS Item to the MFG Item, taking the new additions from the Extension element. The problem here is that the BizTalk mapper does not work with ##any elements. However, a custom XSLT can be used to map the remaining additional elements.

****

Any change must be communicated to a the clients.

Tim had a great table for when to version a schema, and or service. I'll try to repro here when more time exists. I'm betting its probably available online somewhere else.

Advanced Routing and Correlation with BizTalk Orchestrations by Lee Graber, Microsoft
Lee is the Lead Developer for the BizTalk Server product. His talks are usually chock full of BizTalk goodness.

The BizTalk runtime uses the WasPromoted property to determine whether or not to 're-promote' properties for routing.

Correlating subscriptions are deleted non-deterministically. E.g. They clean them up after they have been satisfied, but we don't know when.

A Listen shape with a Delay shape == 'Zombie Factory". A 'zombie' message is when there is a problem, either at the transport, or with the logic of what the process is trying to accomplish. In the case of the listen with the delay, a zombie will occur when the delay timespan expires, yet the message is received either at the exact same moment, or before the correlating subscription can be cleaned up. This results in a message 'completed but with discarded messages' message.

Lee also gave a great demonstration of a Resequencer Pattern using orchestration. The sample is availalbe for download, when I find it I'll post a link. It was incredibly simple and powerful. I'm also wondering if we can do some sort of batching with BizTalk using a similar pattern.

Large messages are fragmented into smaller chunks when stored at the database. This occurs at the Message Agent layer. Only the first chunk is loaded and sent to an service instance.

Its key to use a Distinguished Property when accessing message data in an orchestration so as to avoid reparsing of the message. Using the xpath construct within the orchestration will cause the reading of the message data to locate the xpath.
[Saw Marc Berry, a consultant who has given Deep Dive training on BizTalk!]
Understanding the Microsoft Application Platform Infrastructure Optimization Campaign by Pearson Cummings, Microsoft
Ah, man...this was almost a complete waste of time for a customer. I would love to understand my organizations position in the maturity model, but I got almost nothing from the presentation other than where to go for additional information: Microsoft Infrastructure Optimization Partner Kit

Developer Q&A Panel
Dominated by BizTalk questions..good stuff. This was a chance to bounce techie questions across members of the development teams of BizTalk Server, WCF, and WF.

QFE for versioning of MessageType is available if required. Not exactly sure what the context of the question was here.

There is a comprehensive WCF UDDI sample available as part of the WCF runtime.

Requests for a dynamic orchestration filter were common; this is likely a vNext feature.

A couple of people had implemented the JDE OneWorld XE BizTalk adapter and were expressing how painful this was. e.g.

• 2 months to implement with both Microsoft and iWay help
• JDE running on AS400
• Required .jar complilation leads to a very fragile implementation

They are working on the documentation for the BizTalk LOB adapters. Jay Lee, a CNC Consultant is putting together a whitepaper on the adapters that should be available soon.

The new adapter (R2?) for BizTalk PeopleSoft will be based on the 8.96 toolset and include using thinnet through the COM api, not the Java bridge.

Microsoft has signed a partnership with Oracle to assist in the development and support of these LOB adapters.