The concept of atomic transactions has played a cornerstone role in creating today’s enterprise application environments by providing guaranteed consistent outcome in complex multiparty business operations and a useful separation of concerns in applications. While numerous multiparty business applications involve various patterns based on atomic transactions in order to solve non-trivial business problems, it was not until recently the word “business transactions” accumulated any concrete meaning. Rapid developments in Internet infrastructure and protocols has yielded a new type of application interoperation that makes concepts which could only previously be considered in an abstract form an implementation reality. The effects of such changes have been felt most strongly in business environments, fueling the mindset for a transition from traditional atomic transactions to extended transaction models better suited for Internet interoperation.

Most business-to-business applications require transactional support in order to guarantee consistent outcome and correct execution. These applications often involve long running computations, loosely coupled systems and components that do not share data, location, or administration and it is thus difficult to incorporate traditional Atomic, Consistent, Isolated, Durbale (ACID) transactions within such architectures. For example, an airline reservation system may reserve a seat on a flight for an individual for a specific period of time, but if the individual does not confirm the seat within that period it will be unreserved.

In 2001, a consortium of companies including Hewlett-Packard, Oracle and BEA began work on the OASIS Business Transaction Protocol (BTP)which was aimed at business-to-business transactions in loosely coupled domains such as Web Services [BTP]. The specification developed two new models for “transactions”, requiring business-level decisions to be incorporated within the transaction infrastructure. By April 2002 it had reached the point of a committee specification. Hewlett-Packard quickly followed this with the first commercial product based on this specification.

However, notable by their absence from BTP were Web Services heavy weights IBM and Microsoft, who in August 2002 released their own specifications: Web Services Coordination (WS-C)[WS-C] and Web Services Transactions (WS-T)[WS-T]. Although the WS-T specification also defines two “transaction” models, they are both very much grounded in existing transactional infrastructures. In particular, business applications would use WS-T in the same way applications use traditional transaction systems; there is no attempt to force this business logic into the transactional infrastructure.

Then in July 2003, Arjuna Technologies, Fujitsu, IONA Technologies, Oracle and Sun Microsystems released the Web Services Composite Application Framework (WS-CAF)[WS-CAF], which includes a separate specification, Web Services Transaction Management (WS-TXM)[WS-TXM]. WS-TXM defines three "transaction" models, and is explicitly aimed at leveraging existing transaction infrastructures and investments and providing interoperability between them. As with WS-T, there is no attempt to force business logic into the transactional infrastructure.

Unfortunately for developers who require transaction support in their Web Services, the release of three apparently competing specifications has effectively frozen the market. Is this a case of the “not invented here syndrome” (sometimes known as “my protocol is better than yours”) or is there some fundamental reason why three different protocols have arisen? Have requirements really changed so much in the last two years?

Atomic transactions are a well-known technique for guaranteeing consistency in the presence of failures [OTS]. The ACID properties of atomic transactions (Atomicity, Consistency, Isolation, Durability) ensure that even in complex business applications consistency of state is preserved, despite concurrent accesses and failures. This is an extremely useful fault-tolerance technique, especially when multiple, possibly remote, resources are involved.

The structuring mechanisms available within traditional atomic transaction systems are sequential and concurrent composition of transactions. These mechanisms are sufficient if an application function can be represented as a single atomic transaction. Transactions are most suitably viewed as “short-lived” entities executing in a closely-coupled environment, performing stable state changes to the system; they are less well suited for structuring “long-lived” application functions (e.g., running for hours, days, …). Long-lived atomic transactions (as typically occur in business-to-business interactions) may reduce the concurrency in the system to an unacceptable level by holding on to resources (e.g., locks) for a long time; further, if such an atomic transaction rolls back, much valuable work already performed could be undone.

It has long been realized that ACID transactions by themselves are not adequate for structuring long-lived applications. To ensure ACID-ity between multiple participants, a multi-phase (typically two) consensus mechanism is required, as illustrated in Figure 1: during the first (preparation) phase, an individual participant must make durable any state changes that occurred during the scope of the atomic transaction, such that these changes can either be rolled back (undone) or committed later once consensus to the transaction outcome has been determined amongst all participants, i.e., any original state must not be lost at this point as the atomic transaction could still roll back. Assuming no failures occurred during the first phase (in which case all participants will be forced to undo their changes), in the second (commitment) phase participants may “overwrite” the original state with the state made durable during the first phase.

Figure 1. Two-phase commit protocol

In order to guarantee consensus, two-phase commit is necessarily a blocking protocol: after returning the phase 1 response, each participant which returned a commit response must remain blocked until it has received the coordinator’s phase 2 message telling it what to do. Until they receive this message, any resources used by the participant are unavailable for use by other atomic transactions, since to do so may result in non-ACID behavior. If the coordinator fails before delivery of the second phase message these resources remain blocked until it recovers. In addition, if a participant fails after phase 1, but before the coordinator can deliver its final commit decision, the atomic transaction cannot be completed until the participant recovers: all participants must see both phases of the commit protocol in order to guarantee ACID semantics. There is no implied time limit between a coordinator sending the first phase message of the commit protocol and it sending the second, commit phase message; there could be seconds or hours between them.

Obviously having a blocking protocol can be a problem in general and not just in Web services. Imagine being unable to access your bank-account details because a failure of a transaction has resulted in participants being blocked. That's the reason that very early on in the development of transaction systems heuristic outcomes were introduced: participants that have been prepared are allowed to make autonomous decisions about whether to commit or rollback. For example, a participant that hasn't received the coordinator's final decision after a few hours may decide to rollback. This obviously breaks the blocking nature of two-phase commit, but introduces other problems. What if the participant's decision goes against the coordinator's? Ultimately heuristics can lead to non-atomic behavior that typically has to be resolved off-line by a system administrator. As a result, they should be used with extreme care.

Therefore, structuring certain activities from long-running atomic transactions can reduce the amount of concurrency within an application or (in the event of failures) require work to be performed again. For example, there are certain classes of application where it is known that resources acquired within an atomic transaction can be released “early”, rather than having to wait until the atomic transaction terminates; in the event of the atomic transaction rolling back, however, certain compensation activities may be necessary to restore the system to a consistent state. Such compensation activities (which may perform forward or backward recovery) will typically be application specific, may not be necessary at all, or may be more efficiently dealt with by the application.

So, if current transaction systems are not sufficient for the loosely-coupled Web services world, what systems (or specifications) are? The answer back in 2001 was that there was nothing really suitable. Because transactions are fundamental to any enterprise application, the development of a transaction specification for Web services was integral to the evolution of Web services.

In order to answer this question it is first necessary to give some (brief) background on what each specification provides and what has driven its development.

Although at first glance it may seem like there is commonality between the specifications (all of them support a two-phase completion protocol, for example), there are significant differences from both a protocol and implementation perspective.

4.3. Pros and Cons of WS-TXM

WS-TXM is over 100 pages, compared to WS-T. It is obvious, however, that more time and attention has been paid by the authors to implementation and interoperability details. For example, heuristics are supported in WS-TXM as is distributed recovery.

Ignoring the WS-Context and WS-CF specifications, the transaction protocols defined by WS-TXM sit strongly with existing infrastructure investments. Atomic Transaction can be used as a bridge between proprietary transaction service implementations such as MSDTC [Microsoft DTC] and implementations based on standards such as [OTS], something which until recently has been extremely difficult to accomplish. As with WS-T, this model sits clearly in a single domain, that of ACID transactions - it is unsuitable for long duration interactions because the two-phase protocol is blocking.

The two extended transaction models provided by WS-TXM cover different aspects of business-to-business interactions. The LRA model allows the creation of (nested) scopes, where each scope performs work in whatever manner is appropriate to the business logic. Compensators for that work are registered with the scope so that if it were to cancel, the work can be undone. There are some similarities with the Business Activity model in WS-T; however, the LRA model allows arbitrary nesting of scopes and hence compensations.

In the BP model all parties involved in a business process reside within business domains, which may themselves use business processes to perform work. Business process transactions are responsible for managing interactions between these domains. A business process (business-to-business interaction) is split into business tasks and each task executes within a specific business domain. A business domain may itself be subdivided into other business domains (business processes) in a recursive manner.

Each domain may represent a different transaction model if such a federation of models is more appropriate to the activity. Each business task (which may be modeled as a scope) may provide implementation specific counter-effects in the event that the enclosing scope must cancel. In addition, periodically the controlling application may request that all of the business domains checkpoint their state such that they can either be consistently rolled back to that checkpoint by the application, or restarted from the checkpoint in the event of a failure.

An individual task may require multiple services to work. Each task is assumed to be a compensatable unit of work. However, as with the LRA model described earlier, how compensation is provided is an implementation choice for the task.

For example, consider the purchasing of a home entertainment system example shown in Figure 2. The on-line shop interacts with its specific suppliers, each of which resides in its own business domain. The work necessary to obtain each component is modeled as a separate task, or Web service. In this example, the HiFi task is actually composed of two sub-tasks.

Figure 2. Business processes and tasks

In this example, the user may interact synchronously with the shop to build up the entertainment system. Alternatively, the user may submit an order (possibly with a list of alternate requirements) to the shop which will eventually call back when it has been filled; likewise, the shop then submits orders to each supplier, requiring them to call back when each component is available (or is known to be unavailable).

Two years ago the world of Web services and transactions looked like a new frontier, requiring new techniques to address the problems that it presented. BTP was seen as the solution to those problems. Unfortunately, with the benefit of hindsight it did not address what users really want: the ability to use existing enterprise infrastructures and applications and for “Web services transactions” to operate as the glue between different corporate domains. And it had better be simple to use and understand as well!

The BTP model is similar to WS-T/WS-TXM in several respects, but crucially it does not address the issues of transaction interoperability: most enterprise transaction systems do not expose their coordinators through the two-phase protocol. In addition, BTP has many subtle (and some not-so-subtle) impacts on implementations, both at the transaction level but more importantly at the user/service level.

With no exceptions all of the major Web services players who originally participated in the OASIS BTP effort have moved on to either WS-T or WS-CAF. So, although at the time of writing BTP is the only specification at a recognized standards body (though not an adopted specification), it is unlikely to play a major role in the future.

The real issue is between WS-T and WS-TXM. There are technical differences between the various models that both specifications define, but there is nothing that would prevent them from being merged into a single specification. Interestingly the WS-CAF specifications have been designed to allow WS-C or WS-T to plug into them and replace/augment the WS-CF or WS-TXM specifications. As we indicated earlier the main reason for two different specifications appears to be political. However, there is another important difference: the WS-C/T specifications are not available royalty free or in a standards process, whereas the WS-CAF specifications are royalty free and currently progressing through a standards body.

While much of this may be hidden to end users (except for possible added cost), there is a large community of businesses and users that actually use these specifications and standards directly to build their products and services. Particularly with Web services, which is envisioned as enabling a large and rapidly growing marketplace/ecosystem of services, it is critical to prevent the foundational standards to be threatened by IP that is not publicized.

Even end-user customers and businesses who don’t build the services themselves are concerned with this when they consider the long-term impact of having a key foundation for their business controlled by a couple of big players. Open and royalty-free standards provide predictability, interoperability and reduced costs. How many of today’s successful e-businesses would be content to discover that a company who once had engineers contribute, for example, to the TCP/IP or HTTP specification efforts, now claims intellectual property rights over their contributions, and demands royalties for use of those standards?

Hopefully the authors of these two specifications will be able to co-operate to consolidate them into a single specification and ultimately standard. However, there are strong polarization forces at work here:

So where does this leave the humble end-user? At the moment there are really only two transactions specifications to watch out for: WS-T and WS-TXM. As we've seen, OASIS BTP was a valiant effort, but suffered from naivety and lack of direction. Whether WS-T or WS-TXM (via WS-CAF) become the predominante standard is unknown at this time. Unless Microsoft and IBM are willing to work through an open-standards process and compete in the marketplace on implementation rather than specification, it is unlikely that there will ever be a single specification for Web services transactions.

Looking back over the brief history of Web services transactions. does this mean that the answer to Web services transactions is what we have had for the past 20+ years but using XML and SOAP? Yes! In the real world, it is unreasonable (and naive) to assume that people can (or will want to) throw away their corporate investments in infrastructure, training etc. The investment in transaction processing systems over the past few decades has cost $billions and any scheme to leverage that investment rather than replace it is the way forward.

Much has been made of the fact that ACID transactions aren’t suitable for loosely coupled environments like the Web. However, very little attention has been paid to the fact that these loosely coupled environments tend to have large strongly coupled corporate infrastructures behind them! When BTP started, the question should not have been “what can replace ACID transactions?”, but rather “how can we leverage what already exists?”