- Communication, Flow, Rules and Logic by Steve Swartz and Clemens Vasters (Microsoft)
- BizTalk Web Services: The Next Generation by Aaron Skonnard (Pluralsight) and Gruia Pitigo-Aron (Microsoft)
- Applying Maximum Sustainable Throughput to a Management Operations Strategy (Scott Colestock)
- BizTalk WCF Adapters In-Depth by Aaron Skonnard (Pluralsight) and Gruia Pitigo-Aron (Microsoft)
- Developing & Maintaining Business Rule Solutions by Richard Seroter (Microsoft)
- The Future of the Microsoft Application Server Platform by Steve Swartz and Clemens Vasters (Microsoft)
Saturday, October 07, 2006
BizTalk SOA and Business Process - Day (4)
Wow...its been an week. I have to bow out of the final day sessions in order to catch a plane back to Dallas. This is a shame because there are several sessions which look promising:
BizTalk SOA and Business Process - Day (3)
Breakfast
Literally a roundtable discussion of several topics with aproximately ~8 attendees around the breakfast table that comprise our 'panel'.
BizTalk and Virtualization
7 out of 8 use BizTalk in virtualized non-production environments. That means all development, staging and functional test environments were virtualized.
7 out of 8 use Microsoft's Virtual Server product over VMWare.
Can you guess who the lone hold out was that doesn't use BizTalk in a virtualized environment? I'll give you a hint, its the same person who uses VMWare. No one had anything against VMWare. I got the feeling that some panelists would have preferred it given a choice. However, all voted with confidence that virtualization was a great thing with the caveat being Sql Server. With a high end SAN to back it up, I wonder if the sentiment would have still been the same.
Virtualized Development Environments
Spoke about the need about having such a wide range of tools and technologies available on the desktop that are not necessarily compatible. Brian Prince (a consultant and speaker at TechEd 2006) spoke in detail how their entire development environment is virtualized, even on the desktop.
Brians team, being consultants, may be working as many as two or three projects at once. Each project could have an incompatible technology stack with the other. In such a scenario virtualization saved the day. For each project a developer is working on, the developer will receive VHD representing each tier. E.g. Development Workstation, BizTalk (or other application server) and SQL. One of the keys is evidently giving the VM's each a new SID and with BizTalk, installing, but not configuring the instance until ready. So its effectively a manual post build step after the scripts create the virtual environment.
Sunny, from EDS, explained how they script most of the servers for a virtual environment, from domain controllers, database, and through application servers.
Being able to quickly turn out new 'environments' seemed to be a huge productivity gain, giving faster proejct startup times and consistent developer workstation builds.
Continuous Integration, Repeatable Builds and Deployment
All the talk of virtualization and scripting the build of consistent environments quickly turned into a discussion of how to turn out builds quickly. Opinions seemed to be that library and ASP.NET (including web service) builds continuous integration worked fine. This not ony includes component builds, but deployments to an integrated environment. Brian had the most advanced process, which also included notifying the test team of the availability of the new build along with the change sets that were deployed.
With BizTalk projects, a daily, or regularly scheduled build was more appropriate. The general consensus was due to complexity of the deployment. However, all agreed that getting the BizTalk build and deployment down was critical to the forward motion of a BizTalk project.
It was worth noting that 7 of 8 panelists were now using Team Foundation Server to some degree and MSBuild to build their projects.
Sessions
Access and Identity Management by Steve Swartz & Clemens Vasters (Microsoft)
I don't know how much of it was about Identity Managment and the Access mean data access, not access as in authorization. So, while the session wasn't exactly what I expected I enjoyed it nonetheless. These guys work very well as a speaking team.
ACID transactions in data access are extremly costly and doesn't scale. Consider using compensation strategies instead.
Key take aways were to consider how the data in the application (any application) was to be accessed. "Correct data architecture is the ultimate performance optimization"; in those times when I've truely encountered problems with peformance, they were traced to data access.
High Availability, Fault Tolerance and Scalability with BizTalk Server 2006 by Jay Lee (eBI Solutions)
I believe this talk was given by a different person at TechEd 2006, at least it had the same 'Failover and Recover BizTalk in less than 5 minutes' demo. Its impressive and makes one want to double check that it works in your environment.
The configurations for BizTalk to achieve high availability were right out of BizTalk documentation you can find online here.
In the slide deck, it appears that Jay had the master secret server installed on the SQL boxes.
MSI is the prescribed method for deployment; reminder to import the msi from one server and then run the msi on the remaining servers in the server farm. This can be automated.
Again, saw BizTalk in a virtual environment: 1 Active Directory VM, 1 SQL VM, 2 BizTalk VM on a laptop. Just what kind of laptop was he running?!?
There was a recurring theme throughout the conference, reinforced here, that BizTalk Developer and BizTalk Operations were two distinct roles. Essentially the Developer role is responsible for development, building, packaging and configuring bindings. The Operational role is responsible for deployment, health, performance and scalability of the system.
Lunch with Covast: Newest Developments in B2B
Interesting that Covast is moving more into the partner space rather than the competitor against BizTalk Server. They also discussed an integration appliance for B2B scenarios using EDI formats.
Effective Techniques for Handling Large Messages in Service Oriented Solutions by Thomas Abraham (Digineer)
Briefly talked about MTOM and streaming methods before jumping into a custom solution he had developed that addressed a specific business case. Essentially he had to transport PDF files and these PDF files could be large. Sucking them through BizTalk, as base64 encoded data peformed fairly poorly. Thomas's custom solution was to split the data on the way in, stream the PDF data to a file share, and then correlate the message identifier back to the PDF on disk at the end of the process. Kudo's for being creative. One audience member brought up disaster recovery and its true, this should work fine as long as the backup's of the database and the fileshare were in synch. Otherwise, there might be some data loss, but it should be relatively easy to reconcile depending on the volume of data.
Here is a quick chart of message sizes and their categories.
Configuring, Building and Deploying BizTalk Applications in a Distributed Environment by Paul Gomez (ThinkBox Solutions)
Again, something of an identity crisis. ThinkBox Solutions shouldn't be confused with ThinkBox educational software (though they do have a Development Services group).
Paul had the hard luck of following a session that has already covered much of the material he was presenting. He discussed creating available BizTalk configurations that have already been covered pretty well.
The client solution that they had developed sounds fairly large (8 BizTalk Servers) and that definitely means that they've got issues around deployment.
Currently, ThinkBox uses NAnt (shelling out to BTSTask) to meet their BizTalk deployment needs. To help keep things streamlined, they deploy all bits to all boxes in the farm. This keeps the server builds conisistent, but allows for configuration of each node in the farm for its specific function, or role (Processor, Sender, Receiver).
Their security account management seemed overly complex with an account for each server role type (Processor, Sender, Receiver).
Prior to deployment to any integrated environment, they used the concept of a deployment staging server to configure bindings, etc.. This is the server that the binds will be exported from for use at deploy time.
Interestingly enough, they created their own direct submit adapter. This direct submit adapter simply submits a message to the message box. One thing that was odd, was that they had a pair of servers dedicated to running the host instances for this adapter. Something else to note is that in order to submit to the message box, each developer workstation had to be a member of the BizTalk workgroup.
Again, it was stressed that the Developer role is to develop and package the build (including bindings) and the Operational role to deploy, monitor, and tune. ThinkBox, like many organizations with a large deployment, has a dedicated deployment guru.
Where the discussion was weak was around the physical organization of the project structure for ThinkBox solutions, how they handled versioning, etc..
BPM Q&A Panel
Little to no discussion around Business Process Management; it was essentially a continuation of developer/architect Q&A on products and platforms. This could, and should, have been moderated better.
One problem might have been that the majority of the 'business people' seemed to have disappeared. I don't know if they bugged out early, or were just attending different sessions. I would have sincerely liked to have found a couple to engage in discussions of whats important to them in their business.
Literally a roundtable discussion of several topics with aproximately ~8 attendees around the breakfast table that comprise our 'panel'.
BizTalk and Virtualization
7 out of 8 use BizTalk in virtualized non-production environments. That means all development, staging and functional test environments were virtualized.
7 out of 8 use Microsoft's Virtual Server product over VMWare.
Can you guess who the lone hold out was that doesn't use BizTalk in a virtualized environment? I'll give you a hint, its the same person who uses VMWare. No one had anything against VMWare. I got the feeling that some panelists would have preferred it given a choice. However, all voted with confidence that virtualization was a great thing with the caveat being Sql Server. With a high end SAN to back it up, I wonder if the sentiment would have still been the same.
Virtualized Development Environments
Spoke about the need about having such a wide range of tools and technologies available on the desktop that are not necessarily compatible. Brian Prince (a consultant and speaker at TechEd 2006) spoke in detail how their entire development environment is virtualized, even on the desktop.
Brians team, being consultants, may be working as many as two or three projects at once. Each project could have an incompatible technology stack with the other. In such a scenario virtualization saved the day. For each project a developer is working on, the developer will receive VHD representing each tier. E.g. Development Workstation, BizTalk (or other application server) and SQL. One of the keys is evidently giving the VM's each a new SID and with BizTalk, installing, but not configuring the instance until ready. So its effectively a manual post build step after the scripts create the virtual environment.
Sunny, from EDS, explained how they script most of the servers for a virtual environment, from domain controllers, database, and through application servers.
Being able to quickly turn out new 'environments' seemed to be a huge productivity gain, giving faster proejct startup times and consistent developer workstation builds.
Continuous Integration, Repeatable Builds and Deployment
All the talk of virtualization and scripting the build of consistent environments quickly turned into a discussion of how to turn out builds quickly. Opinions seemed to be that library and ASP.NET (including web service) builds continuous integration worked fine. This not ony includes component builds, but deployments to an integrated environment. Brian had the most advanced process, which also included notifying the test team of the availability of the new build along with the change sets that were deployed.
With BizTalk projects, a daily, or regularly scheduled build was more appropriate. The general consensus was due to complexity of the deployment. However, all agreed that getting the BizTalk build and deployment down was critical to the forward motion of a BizTalk project.
It was worth noting that 7 of 8 panelists were now using Team Foundation Server to some degree and MSBuild to build their projects.
Sessions
Access and Identity Management by Steve Swartz & Clemens Vasters (Microsoft)
I don't know how much of it was about Identity Managment and the Access mean data access, not access as in authorization. So, while the session wasn't exactly what I expected I enjoyed it nonetheless. These guys work very well as a speaking team.
ACID transactions in data access are extremly costly and doesn't scale. Consider using compensation strategies instead.
Key take aways were to consider how the data in the application (any application) was to be accessed. "Correct data architecture is the ultimate performance optimization"; in those times when I've truely encountered problems with peformance, they were traced to data access.
High Availability, Fault Tolerance and Scalability with BizTalk Server 2006 by Jay Lee (eBI Solutions)
I believe this talk was given by a different person at TechEd 2006, at least it had the same 'Failover and Recover BizTalk in less than 5 minutes' demo. Its impressive and makes one want to double check that it works in your environment.
The configurations for BizTalk to achieve high availability were right out of BizTalk documentation you can find online here.
In the slide deck, it appears that Jay had the master secret server installed on the SQL boxes.
MSI is the prescribed method for deployment; reminder to import the msi from one server and then run the msi on the remaining servers in the server farm. This can be automated.
Again, saw BizTalk in a virtual environment: 1 Active Directory VM, 1 SQL VM, 2 BizTalk VM on a laptop. Just what kind of laptop was he running?!?
There was a recurring theme throughout the conference, reinforced here, that BizTalk Developer and BizTalk Operations were two distinct roles. Essentially the Developer role is responsible for development, building, packaging and configuring bindings. The Operational role is responsible for deployment, health, performance and scalability of the system.
Lunch with Covast: Newest Developments in B2B
Interesting that Covast is moving more into the partner space rather than the competitor against BizTalk Server. They also discussed an integration appliance for B2B scenarios using EDI formats.
Effective Techniques for Handling Large Messages in Service Oriented Solutions by Thomas Abraham (Digineer)
Briefly talked about MTOM and streaming methods before jumping into a custom solution he had developed that addressed a specific business case. Essentially he had to transport PDF files and these PDF files could be large. Sucking them through BizTalk, as base64 encoded data peformed fairly poorly. Thomas's custom solution was to split the data on the way in, stream the PDF data to a file share, and then correlate the message identifier back to the PDF on disk at the end of the process. Kudo's for being creative. One audience member brought up disaster recovery and its true, this should work fine as long as the backup's of the database and the fileshare were in synch. Otherwise, there might be some data loss, but it should be relatively easy to reconcile depending on the volume of data.
Here is a quick chart of message sizes and their categories.
| Message Size | Size Category |
|---|---|
| < 10KB | Ideal |
| 10KB - < 100KB | Small |
| 100KB - < 1MB | Medium |
| 1MB - < 5MB | Large |
| > 5MB | Very Large |
Configuring, Building and Deploying BizTalk Applications in a Distributed Environment by Paul Gomez (ThinkBox Solutions)
Again, something of an identity crisis. ThinkBox Solutions shouldn't be confused with ThinkBox educational software (though they do have a Development Services group).
Paul had the hard luck of following a session that has already covered much of the material he was presenting. He discussed creating available BizTalk configurations that have already been covered pretty well.
The client solution that they had developed sounds fairly large (8 BizTalk Servers) and that definitely means that they've got issues around deployment.
Currently, ThinkBox uses NAnt (shelling out to BTSTask) to meet their BizTalk deployment needs. To help keep things streamlined, they deploy all bits to all boxes in the farm. This keeps the server builds conisistent, but allows for configuration of each node in the farm for its specific function, or role (Processor, Sender, Receiver).
Their security account management seemed overly complex with an account for each server role type (Processor, Sender, Receiver).
Prior to deployment to any integrated environment, they used the concept of a deployment staging server to configure bindings, etc.. This is the server that the binds will be exported from for use at deploy time.
Interestingly enough, they created their own direct submit adapter. This direct submit adapter simply submits a message to the message box. One thing that was odd, was that they had a pair of servers dedicated to running the host instances for this adapter. Something else to note is that in order to submit to the message box, each developer workstation had to be a member of the BizTalk workgroup.
Again, it was stressed that the Developer role is to develop and package the build (including bindings) and the Operational role to deploy, monitor, and tune. ThinkBox, like many organizations with a large deployment, has a dedicated deployment guru.
Where the discussion was weak was around the physical organization of the project structure for ThinkBox solutions, how they handled versioning, etc..
BPM Q&A Panel
Little to no discussion around Business Process Management; it was essentially a continuation of developer/architect Q&A on products and platforms. This could, and should, have been moderated better.
One problem might have been that the majority of the 'business people' seemed to have disappeared. I don't know if they bugged out early, or were just attending different sessions. I would have sincerely liked to have found a couple to engage in discussions of whats important to them in their business.
BizTalk SOA and Business Process - Day (2)
Sessions
Customer Panel: SOA Success Stories Panel: Ryan Garner, JetBlue Airways, Justin Myrick, Clear Channel Communications & Rodney Turpin, Hewlett Packard Ryan Garner, JetBlue Airways
JetBlue's SOA evolved out of a business driver to standardize partner integration and take advantage of additional inventory channels.
Part of their success was attributed to creating a standards body for service creation. Stressed the importance of tackling areas such as:
They use a purchased product called Service Manager from SOA Software (yes, agreeably confusing) for their web service monitoring and Service Level Agreement (SLA) compliance.
They also were they only member of the panel to discuss setting up a forum for partner discussion and feedback using Community Server. While they seemed to have more partner integration that others of the panel, there was no discussion of partner management (ran out of time). If I can catch Ryan at the conference later I will try to get that question in front of him.
They followed a contract-first, or schema first, approach to their service creation and contribute that as a success factor.
Justin Myrick, Clear Channel Communications
Their SOA also evolved out of a business requirement. Evidently there was a proliferation of ERP systems; essentially one per market....and Clear Channel is in a lot of markets.
Stated Microsoft Technologies:
Clear Channel used a WSE filter to log message request/response data to a database that was used as a debugging and operational aid.
Rodney Turpin, Hewlett Packard
While the other two panelists had much more recent SOA implementations (less than a year), HP has had an ongoing SOA initiative that has been evolving for several years. HP has something on the order of ~11,000 diseperate applications that needed to be integrated, or their functionality composed into services easily consumed by new systems.
HP is the only panelist to use the Business Activity Monitoring (BAM) feature in BizTalk for monitoring business processes, but its widespread use was admittably "light". Rodney had high regard for using Microsoft Operation Manager (MOM) for BizTalk application server monitoring.
There was an initiative to collapse data centers and their SOA implementation needed to support that.
StatedMicrosoft Technologies:
Avoiding 3 Common Pitfalls in Service Contract Design by Tim Ewald, Foliage Software Systems
Tim had a pretty good session and its unlikely that I am able to capture it all here. There was significant information presented, especially around version strategies, that might take some time to digest.
While approving of a cannonical data model, he discusses why canonical data formats for a business typically fail. They have three major problems that typically manifest as:
Typically a standards body is formed to defined the canonical data format. This standards body spends time ([n] weeks, months) interviewing business consumers identifying what data is required. This required data makes the schema, and those systems that consume it, brittle.
No realistic versioning
Schemas are typically versioned through the namespace. This will typically break any implementation that consumes it (effectively creating a brand new entity..it just happens to look a lot like the entity it is a revision of).
No system level extensibility
While the standards body is out interviewing different business teams and defining the model, by the time they get a standard, its likely that a business team will need a revision to it. However, because it takes this standards body time to review the change request, there needs to be a way for a system to extend the schema without breaking it. His recommendation is to include an Extension element, which is a ##any element. Effectively allowing a system to stuff anything it needs there. Of course, the receiving system will need to know what to do with it. The thought is that this will allow systems to be 'agile' until the standards body can get through the review process on whether or not to standardize the extension.
Tim had a great table for when to version a schema, and or service. I'll try to repro here when more time exists. I'm betting its probably available online somewhere else.
Advanced Routing and Correlation with BizTalk Orchestrations by Lee Graber, Microsoft
Lee is the Lead Developer for the BizTalk Server product. His talks are usually chock full of BizTalk goodness.
The BizTalk runtime uses the WasPromoted property to determine whether or not to 're-promote' properties for routing.
Correlating subscriptions are deleted non-deterministically. E.g. They clean them up after they have been satisfied, but we don't know when.
A Listen shape with a Delay shape == 'Zombie Factory". A 'zombie' message is when there is a problem, either at the transport, or with the logic of what the process is trying to accomplish. In the case of the listen with the delay, a zombie will occur when the delay timespan expires, yet the message is received either at the exact same moment, or before the correlating subscription can be cleaned up. This results in a message 'completed but with discarded messages' message.
Lee also gave a great demonstration of a Resequencer Pattern using orchestration. The sample is availalbe for download, when I find it I'll post a link. It was incredibly simple and powerful. I'm also wondering if we can do some sort of batching with BizTalk using a similar pattern.
Large messages are fragmented into smaller chunks when stored at the database. This occurs at the Message Agent layer. Only the first chunk is loaded and sent to an service instance.
Its key to use a Distinguished Property when accessing message data in an orchestration so as to avoid reparsing of the message. Using the xpath construct within the orchestration will cause the reading of the message data to locate the xpath.
[Saw Marc Berry, a consultant who has given Deep Dive training on BizTalk!]
Understanding the Microsoft Application Platform Infrastructure Optimization Campaign by Pearson Cummings, Microsoft
Ah, man...this was almost a complete waste of time for a customer. I would love to understand my organizations position in the maturity model, but I got almost nothing from the presentation other than where to go for additional information: Microsoft Infrastructure Optimization Partner Kit
Developer Q&A Panel
Dominated by BizTalk questions..good stuff. This was a chance to bounce techie questions across members of the development teams of BizTalk Server, WCF, and WF.
QFE for versioning of MessageType is available if required. Not exactly sure what the context of the question was here.
There is a comprehensive WCF UDDI sample available as part of the WCF runtime.
Requests for a dynamic orchestration filter were common; this is likely a vNext feature.
A couple of people had implemented the JDE OneWorld XE BizTalk adapter and were expressing how painful this was. e.g.
The new adapter (R2?) for BizTalk PeopleSoft will be based on the 8.96 toolset and include using thinnet through the COM api, not the Java bridge.
Microsoft has signed a partnership with Oracle to assist in the development and support of these LOB adapters.
Customer Panel: SOA Success Stories Panel: Ryan Garner, JetBlue Airways, Justin Myrick, Clear Channel Communications & Rodney Turpin, Hewlett Packard Ryan Garner, JetBlue Airways
JetBlue's SOA evolved out of a business driver to standardize partner integration and take advantage of additional inventory channels.
Part of their success was attributed to creating a standards body for service creation. Stressed the importance of tackling areas such as:
- Security
- Health Checking
- Monitoring
- Logging
They use a purchased product called Service Manager from SOA Software (yes, agreeably confusing) for their web service monitoring and Service Level Agreement (SLA) compliance.
They also were they only member of the panel to discuss setting up a forum for partner discussion and feedback using Community Server. While they seemed to have more partner integration that others of the panel, there was no discussion of partner management (ran out of time). If I can catch Ryan at the conference later I will try to get that question in front of him.
They followed a contract-first, or schema first, approach to their service creation and contribute that as a success factor.
Justin Myrick, Clear Channel Communications
Their SOA also evolved out of a business requirement. Evidently there was a proliferation of ERP systems; essentially one per market....and Clear Channel is in a lot of markets.
Stated Microsoft Technologies:
- BizTalk 2006
- Microsoft .NET 2.0
- ASP.NET
- Sharepoint
- Reporting Services
- SQL Server 2005
- K2
- Peoplesoft
- UDDI
Clear Channel used a WSE filter to log message request/response data to a database that was used as a debugging and operational aid.
Rodney Turpin, Hewlett Packard
While the other two panelists had much more recent SOA implementations (less than a year), HP has had an ongoing SOA initiative that has been evolving for several years. HP has something on the order of ~11,000 diseperate applications that needed to be integrated, or their functionality composed into services easily consumed by new systems.
HP is the only panelist to use the Business Activity Monitoring (BAM) feature in BizTalk for monitoring business processes, but its widespread use was admittably "light". Rodney had high regard for using Microsoft Operation Manager (MOM) for BizTalk application server monitoring.
There was an initiative to collapse data centers and their SOA implementation needed to support that.
StatedMicrosoft Technologies:
- BizTalk 2004 + BAM
- Microsoft Operations Manager
- Visual Studio Team System (all features: Work Item tracking, version control, etc).
- UDDI
Avoiding 3 Common Pitfalls in Service Contract Design by Tim Ewald, Foliage Software Systems
Tim had a pretty good session and its unlikely that I am able to capture it all here. There was significant information presented, especially around version strategies, that might take some time to digest.
While approving of a cannonical data model, he discusses why canonical data formats for a business typically fail. They have three major problems that typically manifest as:
- Too much required data
- No realistic versioning
- No system level extensibility
Typically a standards body is formed to defined the canonical data format. This standards body spends time ([n] weeks, months) interviewing business consumers identifying what data is required. This required data makes the schema, and those systems that consume it, brittle.
No realistic versioning
Schemas are typically versioned through the namespace. This will typically break any implementation that consumes it (effectively creating a brand new entity..it just happens to look a lot like the entity it is a revision of).
No system level extensibility
While the standards body is out interviewing different business teams and defining the model, by the time they get a standard, its likely that a business team will need a revision to it. However, because it takes this standards body time to review the change request, there needs to be a way for a system to extend the schema without breaking it. His recommendation is to include an Extension element, which is a ##any element. Effectively allowing a system to stuff anything it needs there. Of course, the receiving system will need to know what to do with it. The thought is that this will allow systems to be 'agile' until the standards body can get through the review process on whether or not to standardize the extension.
****
This begs the question of what to do with the maintenance issue that has just been created. The assumption is that the extension that a system adds might be so useful that it is adopted as part of the canonical data model. Now any new work will be able to take advantage of the new data model. However, what about the implementations that are already baked using the extension? Theoretically, they can continue to use the extension as is because it will 'always' be supported. I can imagine that you would eventually want to synch up with the standard though. This will cause rework on now legacy interfaces. Taking an example from our organization: - ERP produces an item update that needs to be published to consumers
- BizTalk maps the ERP Item into an SCS Item
- BizTalk then routes to consumers (e.g. Order Entry and WMS)
- Mary Kay then onboards a new Manufacturing system that needs the SCS Item, but it also needs some additional data that wasn't part of the original SCS Item specification
- The ERP system is modified to produce the item udpate with the additional data
- BizTalk maps the ERP Item into the SCS Item, the addition is mapped into an Extension element
- BizTalk then routes to consumers, original systems continue to use SCS Item as normal (remember, the ##any was still part of the schema and was an addition. If the system is validating, the instance is still valid and the new data is ignored by the original systems)
- BizTalk will map from the SCS Item to the MFG Item, taking the new additions from the Extension element. The problem here is that the BizTalk mapper does not work with ##any elements. However, a custom XSLT can be used to map the remaining additional elements.
****
Any change must be communicated to a the clients.Tim had a great table for when to version a schema, and or service. I'll try to repro here when more time exists. I'm betting its probably available online somewhere else.
Advanced Routing and Correlation with BizTalk Orchestrations by Lee Graber, Microsoft
Lee is the Lead Developer for the BizTalk Server product. His talks are usually chock full of BizTalk goodness.
The BizTalk runtime uses the WasPromoted property to determine whether or not to 're-promote' properties for routing.
Correlating subscriptions are deleted non-deterministically. E.g. They clean them up after they have been satisfied, but we don't know when.
A Listen shape with a Delay shape == 'Zombie Factory". A 'zombie' message is when there is a problem, either at the transport, or with the logic of what the process is trying to accomplish. In the case of the listen with the delay, a zombie will occur when the delay timespan expires, yet the message is received either at the exact same moment, or before the correlating subscription can be cleaned up. This results in a message 'completed but with discarded messages' message.
Lee also gave a great demonstration of a Resequencer Pattern using orchestration. The sample is availalbe for download, when I find it I'll post a link. It was incredibly simple and powerful. I'm also wondering if we can do some sort of batching with BizTalk using a similar pattern.
Large messages are fragmented into smaller chunks when stored at the database. This occurs at the Message Agent layer. Only the first chunk is loaded and sent to an service instance.
Its key to use a Distinguished Property when accessing message data in an orchestration so as to avoid reparsing of the message. Using the xpath construct within the orchestration will cause the reading of the message data to locate the xpath.
[Saw Marc Berry, a consultant who has given Deep Dive training on BizTalk!]
Understanding the Microsoft Application Platform Infrastructure Optimization Campaign by Pearson Cummings, Microsoft
Ah, man...this was almost a complete waste of time for a customer. I would love to understand my organizations position in the maturity model, but I got almost nothing from the presentation other than where to go for additional information: Microsoft Infrastructure Optimization Partner Kit
Developer Q&A Panel
Dominated by BizTalk questions..good stuff. This was a chance to bounce techie questions across members of the development teams of BizTalk Server, WCF, and WF.
QFE for versioning of MessageType is available if required. Not exactly sure what the context of the question was here.
There is a comprehensive WCF UDDI sample available as part of the WCF runtime.
Requests for a dynamic orchestration filter were common; this is likely a vNext feature.
A couple of people had implemented the JDE OneWorld XE BizTalk adapter and were expressing how painful this was. e.g.
- 2 months to implement with both Microsoft and iWay help
- JDE running on AS400
- Required .jar complilation leads to a very fragile implementation
The new adapter (R2?) for BizTalk PeopleSoft will be based on the 8.96 toolset and include using thinnet through the COM api, not the Java bridge.
Microsoft has signed a partnership with Oracle to assist in the development and support of these LOB adapters.
BizTalk SOA and Business Process - Day (1)
Quick statistic:
This is year 5 of the conference. Year 1 had ~50 attendees; Year 5 had ~800. There were roughly ~120 invited to the Day 0 events.
Keynotes
SOA, BPM and Microsoft: A Pragmatic View by David Chappell (Principal, Chappell Associates)
Turns out, that there are three David Chappells and two out of the three have appeared on tv. Given that, it might be easy to understand how there might be some initial confusion; which David Chappell are we talking about? Is it David Chappell, David Chappell, or David Chappell?
Dynamic speaker, has great stories and a good sense of humor. Poked fun at both Gartner and Forrester definitions of an Enterprise Service Bus (ESB).
Discussed upcoming Windows Communication Foundation (WCF) as this generations tcp/ip standard for the Microsoft Platform. Also indicated that the Service Component Architecture is the Java worlds answer to WCF. There was some discussion that Sun technologies were fragmenting and that the platform might be left in chaos. This statement is based on three technologies competing for shares on the Java platform: Open SOA, Service Component Architecture, and Open Source.
Noted that there is no queing standard with WCF for application to application communication. I had incorrectly assumed that WS-Reliable Messaging (WS-RM) fullfilled this requirement. Thats not the case. The message is not durable (ala MSMQ), however, it is guaranteed much like a TCP/IP connection. Its indicated that this is a target area for future development for WCF.
Real World SOA by John deVadoss (Director of Architecture Strategy, Microsoft Corporation)
Somewhat dryer material. The biggest message seemed to be damage control on the SOA negative spin in the media lately. Kept referring to the "hype cycle" typical of industry buzzwords and not to "throw out the good with the bad".
Sessions
The Architecture of SOA by John Evdeman (Microsoft)
Again dry technology material. Biggest message is that SOA is not technology, or platform, specific.
Pointed out that Readings in Service Orientation is a free, online book which contains some best practices when composing SOA.
Choosing the Right Technology for Exposing and Composing Services by Kris Horrocks (Microsoft)
I've caught talks delivered by Kris at TechEd 2006 and he does a good job at keeping the information at 'just the right level'. The biggest topics are around deciding when to use SSIS, BizTalk or Windows Workflow Foundation (WF).
There might be an opportunity to replace the proprietary SCSWS Sql Queue with a Sql Server Service Broker implementation.
Selling BizTalk-Based Engagements by David Chappell (Principal, Chappell Associates)
I went into this session thinking that it would be about selling BizTalk as a solution to integration opportunities within an organization, and for the most part it was. However, it was definitely parnter focused in helping you move from the Sales Call through The Close. Much of it was still applicable to stake holders within an organization.
Partner Expo
The partner expo was definitely no comparison to something you might see at a PDC or TechEd. There was little to no swag to be had. The only thing worthwhile was a copy of the book BizTalk 2006 Receipes. However, they weren't giving them away, they were just for display. When I asked how I could get one of them, they joked that several of the authors were here and that if I could get them to tell the staff to give me one, they would. Well, holding down an author and forcing him to give me a free copy of his book wasn't high on my list of priorities (already have a copy of the book on the way). But, grabbing a bite to eat and getting back to the hotel for a 6AM conference call with Dallas was. The snacks were just that...snacks.
Since the snacks didn't fill me I went ahead and ordered room service; vegetable lasagna. It was worst dish I have had in a while. There were raw carrots in the middle of it. Not little carrots mind you, but the big fat kind. blech.
This is year 5 of the conference. Year 1 had ~50 attendees; Year 5 had ~800. There were roughly ~120 invited to the Day 0 events.
Keynotes
SOA, BPM and Microsoft: A Pragmatic View by David Chappell (Principal, Chappell Associates)
Turns out, that there are three David Chappells and two out of the three have appeared on tv. Given that, it might be easy to understand how there might be some initial confusion; which David Chappell are we talking about? Is it David Chappell, David Chappell, or David Chappell?
Dynamic speaker, has great stories and a good sense of humor. Poked fun at both Gartner and Forrester definitions of an Enterprise Service Bus (ESB).
Discussed upcoming Windows Communication Foundation (WCF) as this generations tcp/ip standard for the Microsoft Platform. Also indicated that the Service Component Architecture is the Java worlds answer to WCF. There was some discussion that Sun technologies were fragmenting and that the platform might be left in chaos. This statement is based on three technologies competing for shares on the Java platform: Open SOA, Service Component Architecture, and Open Source.
Noted that there is no queing standard with WCF for application to application communication. I had incorrectly assumed that WS-Reliable Messaging (WS-RM) fullfilled this requirement. Thats not the case. The message is not durable (ala MSMQ), however, it is guaranteed much like a TCP/IP connection. Its indicated that this is a target area for future development for WCF.
Real World SOA by John deVadoss (Director of Architecture Strategy, Microsoft Corporation)
Somewhat dryer material. The biggest message seemed to be damage control on the SOA negative spin in the media lately. Kept referring to the "hype cycle" typical of industry buzzwords and not to "throw out the good with the bad".
Sessions
The Architecture of SOA by John Evdeman (Microsoft)
Again dry technology material. Biggest message is that SOA is not technology, or platform, specific.
Pointed out that Readings in Service Orientation is a free, online book which contains some best practices when composing SOA.
Choosing the Right Technology for Exposing and Composing Services by Kris Horrocks (Microsoft)
I've caught talks delivered by Kris at TechEd 2006 and he does a good job at keeping the information at 'just the right level'. The biggest topics are around deciding when to use SSIS, BizTalk or Windows Workflow Foundation (WF).
There might be an opportunity to replace the proprietary SCSWS Sql Queue with a Sql Server Service Broker implementation.
Selling BizTalk-Based Engagements by David Chappell (Principal, Chappell Associates)
I went into this session thinking that it would be about selling BizTalk as a solution to integration opportunities within an organization, and for the most part it was. However, it was definitely parnter focused in helping you move from the Sales Call through The Close. Much of it was still applicable to stake holders within an organization.
Partner Expo
The partner expo was definitely no comparison to something you might see at a PDC or TechEd. There was little to no swag to be had. The only thing worthwhile was a copy of the book BizTalk 2006 Receipes. However, they weren't giving them away, they were just for display. When I asked how I could get one of them, they joked that several of the authors were here and that if I could get them to tell the staff to give me one, they would. Well, holding down an author and forcing him to give me a free copy of his book wasn't high on my list of priorities (already have a copy of the book on the way). But, grabbing a bite to eat and getting back to the hotel for a 6AM conference call with Dallas was. The snacks were just that...snacks.
Since the snacks didn't fill me I went ahead and ordered room service; vegetable lasagna. It was worst dish I have had in a while. There were raw carrots in the middle of it. Not little carrots mind you, but the big fat kind. blech.
Did I say that?
BizTalk SOA and Business Process - Day (0)
First off, let me say how beautiful Bellevue is. Its an extremely beautiful town, with great shopping, dining and bars. I'll definitely have to drag Sally if I ever make it back to town!
There were many personalities here that I've met in the past Doug Girard, Kris Horrocks and Scott Colestock (just to drop a few names). Both Doug and Kris are part of the Connected Systems Division, which now includes the BizTalk Server product and were part of a group, along with Eddie Churchill that visited our facilities some time ago. Scott developed the deployment framework we use to deploy our BizTalk 2004 solution.
For the keynotes, we heard from Oliver Sharp (General Manager, BizTalk Server), Janelle Hill (VP, Business IT of Research Division - Gartner) and Steve Guggenheimer (General Manager, Application Platform Marketing). We also heard from Ed Barrie (Sr. Treasury Product Manager) discussing a case study of using BizTalk Server 2006 and its BPM features for the treasury implementation at Microsoft.
The keynotes were fairly repetetive of material presented as part of
TechEd 2006, but doesn't hurt to hear it again. A fairly significant
message from the keynotes were definitely around the need for Business
Process Management (BPM) and a suggestion for the following around BPM:
Either
A) Wait for the domain offerings to mature
B) Identify existing skillsets within the organization and find complementary technology solution to match. Make a 'tactical' (2 year) decision to 'roll your own' in as much of a vanilla way as possible. Limiting scope of custom code.
A warning around using a vendor (Oracle/SAP) ability to provide an explicit solution to BPM, largely due to a vendors inability to provide a blanket solution for every industry/market.
An Integration Competency Center (ICC) plays an important role in an organizations BPM. e.g. a compentency center is always a good idea when there is a requirement to share resources/information across an organization. With eBiz, for example, an ICC would have established the standards for integration so that there would have been less guesswork when it came time for them to actually integrate. This adds value to the organization in re-using experience, and knowledge, when defining integration patterns and operational strategies across the enterprise. The ICC would also provide the standards to partners.
There are a couple of areas to consider for future:
Had two great break out discussions with peers attending the Day-0 event. These discussions covered Deployment, Configuration, Management and Monitoring of BizTalk solutions. It allowed us to provide some significant input to the BizTalk server team.
The biggest pain points across the groups were:
Thats the brain dump for now...the regular conference starts tomorrow.
There were many personalities here that I've met in the past Doug Girard, Kris Horrocks and Scott Colestock (just to drop a few names). Both Doug and Kris are part of the Connected Systems Division, which now includes the BizTalk Server product and were part of a group, along with Eddie Churchill that visited our facilities some time ago. Scott developed the deployment framework we use to deploy our BizTalk 2004 solution.
For the keynotes, we heard from Oliver Sharp (General Manager, BizTalk Server), Janelle Hill (VP, Business IT of Research Division - Gartner) and Steve Guggenheimer (General Manager, Application Platform Marketing). We also heard from Ed Barrie (Sr. Treasury Product Manager) discussing a case study of using BizTalk Server 2006 and its BPM features for the treasury implementation at Microsoft.
The keynotes were fairly repetetive of material presented as part of
TechEd 2006, but doesn't hurt to hear it again. A fairly significant
message from the keynotes were definitely around the need for Business
Process Management (BPM) and a suggestion for the following around BPM:
Either
A) Wait for the domain offerings to mature
B) Identify existing skillsets within the organization and find complementary technology solution to match. Make a 'tactical' (2 year) decision to 'roll your own' in as much of a vanilla way as possible. Limiting scope of custom code.
A warning around using a vendor (Oracle/SAP) ability to provide an explicit solution to BPM, largely due to a vendors inability to provide a blanket solution for every industry/market.
An Integration Competency Center (ICC) plays an important role in an organizations BPM. e.g. a compentency center is always a good idea when there is a requirement to share resources/information across an organization. With eBiz, for example, an ICC would have established the standards for integration so that there would have been less guesswork when it came time for them to actually integrate. This adds value to the organization in re-using experience, and knowledge, when defining integration patterns and operational strategies across the enterprise. The ICC would also provide the standards to partners.
There are a couple of areas to consider for future:
- BizTalk Accelerator for SWIFT
- SWIFT is a standards body for banking transactions and includes:
- Banking solutions for payments and cash management
- Capital market solutiosn for trading, treasury, custody and corporate actions
- Corporate solutions for payments (including supply chain transactions), receipts, cash management, treasury
- We know that we've had a couple of requirements for integration with banking systems that might be fulfilled by adopting a standard for integration with financial institutions.
- Industry standard for representing supply chain data; e.g. Rosetta.NET
Had two great break out discussions with peers attending the Day-0 event. These discussions covered Deployment, Configuration, Management and Monitoring of BizTalk solutions. It allowed us to provide some significant input to the BizTalk server team.
The biggest pain points across the groups were:
- Configuration management across multiple environments. e.g. Bindings. BizTalk 2006 should mitigate some of the pain we feel today; it could be easier.
- Versioning; e.g. dependency management for schemas, components, orchestrations.
- Monitoring business process impact on performance; e.g. finding corralaries between executing business processes and system performance.
- Sourcing event data through a common api to a central store with a central portal that would have views of the data based on role. Much can be accomplished today through BAM; however, BAM introduces many different complexities.
Thats the brain dump for now...the regular conference starts tomorrow.
BizTalk SOA and Business Process - Day (-1)
Wow, Sunday already. Seemed only last night I was 'camping' in the backyard with the kids...oh, wait. I was.
Today is the first day back in Bellevue, WA. This time around its for a conference hosted by the BizTalk Server team on Service Oriented Architecture (SOA) and Business Process Management.
Staying at the Westin Bellevue, a new Starwood facility in the heart of Bellevue, Washington. I've stayed at Westin before and have always been impressed with it. This hotel is no exception, but its going to take some time. The views are great, but the ameninties are lacking just yet; I don't want to have to walk 3 blocks to [insert your favorite convenience store here] just to pickup water and snacks for breakfast.
At first, I thought I had screwed up and arrived a full day early. After all, I had been told about participating in Day 0 events, but hadn't received anything regarding it. So I came in on Sunday to be ready for Monday (Day 0), the conference actually starts on Tuesday (Day 1). All I get handed at the front desk is a single sheet of paper with a bus schedule and an itenerary that starts with cocktails on Monday night. WTF?? So, I get to 'hang' in Bellevue for a full day? How am I supposed to explain that?
Turns out, I had missed my 'Day 0' itenerary..it starts @ 7:20AM and doesn't let out until after 5PM, with the cocktails and conference registration at 5:30PM.
Here's hoping that it doesn't turn into a first class boondoggle! Although any conference that has free food and alcohol 2 out of the 3 nights your going to be there is a sure bet.
Sorry, no photos for this trip; Of course, I'm already halfway regretting it, but I didn't want to lug an extra device and was trying for this trip to be as lean as possible.
Today is the first day back in Bellevue, WA. This time around its for a conference hosted by the BizTalk Server team on Service Oriented Architecture (SOA) and Business Process Management.
Staying at the Westin Bellevue, a new Starwood facility in the heart of Bellevue, Washington. I've stayed at Westin before and have always been impressed with it. This hotel is no exception, but its going to take some time. The views are great, but the ameninties are lacking just yet; I don't want to have to walk 3 blocks to [insert your favorite convenience store here] just to pickup water and snacks for breakfast.
At first, I thought I had screwed up and arrived a full day early. After all, I had been told about participating in Day 0 events, but hadn't received anything regarding it. So I came in on Sunday to be ready for Monday (Day 0), the conference actually starts on Tuesday (Day 1). All I get handed at the front desk is a single sheet of paper with a bus schedule and an itenerary that starts with cocktails on Monday night. WTF?? So, I get to 'hang' in Bellevue for a full day? How am I supposed to explain that?
Turns out, I had missed my 'Day 0' itenerary..it starts @ 7:20AM and doesn't let out until after 5PM, with the cocktails and conference registration at 5:30PM.
Here's hoping that it doesn't turn into a first class boondoggle! Although any conference that has free food and alcohol 2 out of the 3 nights your going to be there is a sure bet.
Sorry, no photos for this trip; Of course, I'm already halfway regretting it, but I didn't want to lug an extra device and was trying for this trip to be as lean as possible.
Subscribe to:
Posts (Atom)