The past four years have seen XML move to the preeminent position as the accepted language for defining new business-document formats. Nearly every industry group has initiated new "EDI-like" standards-development activities based on XML: RosettaNet, AIAG, UCC/EAN, CIDX, etc.

However, a recent e-mail sent to the XML/EDI Group e-list raised a simple, yet provocative question:

From: Paul
Sent: Wednesday, August 28, 2002 8:11 AM
To: XMLEDI Group
Subject: Safe XML Transportation
I'm looking to send XML business data across the Internet and just
wondered what the safest and most secure way of doing this is?
Paul

The successful migration to XML-based e-business will have to address this issue. We review the alternative solutions available today and present an understanding of the evolution of messaging technologies themselves to guide the development of the roadmap and key requirements along with the factors involved in making choices.

In the early days of EDI (circa 1985), the choices of protocols to transport business data were limited to the communications protocols supported by the major computer vendors. These evolved from asynchronous (character-based) to BSC (synchronous byte-stream) to SDLC (synchronous bit-stream). The "messaging protocol" of that time period was Batch RJE (Remote Job Entry), which was used to upload files into a computer (using punched cards or 80-character records) and to download them (in 80- or 132-character records). Most EDI data was exchanged using a "wrapping" into these record formats.

The commercialization of the Internet (circa 1994) introduced a new set of data-transfer protocols based on the TCP/IP protocol stack. The most popular of these were FTP (File Transfer Protocol) and SMTP (Simple Mail Transport Protocol). This led to early work sending EDI messages via the Internet, developed through the EDI standards organizations themselves (EDIINT). The next major change came when the technology of the World Wide Web (HTTP/HTML) was adopted by commercial enterprises to do business over the Internet.

Several stakeholders have driven the evolution of new messaging protocols:

Let's look at how each of these players was involved.

The Internet SOCiety (ISOC) is a professional membership society with more than 150 organizational and 11,000 individual members in over 182 countries. It provides leadership in addressing issues that confront the future of the Internet, and is the organizational home for the groups responsible for Internet infrastructure standards.

The process currently used by the Internet community for the standardization of protocols and procedures can be found in a document entitled "The Internet Standards Process - Revision 3" (http://www.ietf.org/rfc/rfc2026.txt). The Internet Standards Process is an activity of the ISOC organized and managed on behalf of the Internet community by the Internet Engineering Steering Group (IESG), Internet Engineering Task Force (IETF), and the Internet Architecture Board (IAB).

One of the early commercial uses of XML technology came from the vendors who developed software for public B2B e-marketplaces and private exchanges. The industry was legitimized by the formation of CommerceNet in 1994, a consortium of eCommerce vendors with a mission "to transform the Internet into the world's largest and most efficient marketplace dramatically changing how the world transacts business" (http://www.commercenet.com).

Within two years, the first standards for real-time, secure, Internet-based procurement had been developed: the OBI (Open Buying on the Internet, http://www.openbuy.org, using an adaptation of existing EDI-based mechanisms) and the Information and Content Exchange (ICE) protocol work (see http://www.icestandard.org and http://www.w3.org/TR/NOTE-ice), using pure XML-based workflow mechanisms. From these groups, vendors spun off to develop their proprietary solutions based on XML message formats (and in fact the design and architecture from ICE was copied widely, and manifestations from it can be seen widely today). Such companies as CommerceOne (xCBL) and Ariba (cXML) were the early entrants, and each developed its own messaging transports to exchange these XML business documents between buyers and sellers over the Internet.

Dissatisfied with the existing TCP/IP-based protocols for moving messages (FTP, SMTP), several of the major industry action groups chose to design their own message transport protocol to exchange EDI/XML documents reliably over the Internet.

GISB (Gas Industry Standards Board) was founded in 1993 in response to the natural gas industry's regulatory and technological developments that were transforming the marketplace. In 1995-96, the GISB EDM (Electronic Delivery Mechanism) standard was developed to reduce costs while retaining reliability and security of transactions over the Internet. In 1996, the Department of Energy mandated that every Interstate Natural Gas Pipeline Company use EDM with PGP to sign/encrypt Internet e-commerce transactions. EDM was the first HTTP-based Internet EDI standard.

The EDM specification was recently upgraded to v1.5 to incorporate some of the features of AS2, but continued to promote the use of PGP encryption procedures instead of S/MIME. The AIAG (Automotive Industry Action Group), looking for a HTTP "flavor" to meet its needs, developed a protocol (based on EDM) in 1997 that was called E-5. The AIAG chose conversational HTTP sessions and HTML forms to accomplish what it referred to as "Electronic Commerce Message Routing over TCP/IP Networks."

The AIAG later updated its specification to E-5 2000, which replaces the HTML parameters with an XML document and incorporates security enhancements such as digital signatures.

The earlier work on OBI led to the creation of the RosettaNet consortium, focused on the electronics industry supply chain. RosettaNet distinguished itself by focusing on the process, rather than on the transactions, and developed a set of definitions for exchanges within the electronics industry. It defined a complete set of Internet-based e-business standards, including

Founded in 1998 by 40 organizations, RosettaNet grew into a global consortium of companies throughout the hi-tech industry (electronic components, semiconductor manufacturing, and information technology). Two important events emerged from RosettaNet:

The retail industry, through the EAN.UCC standards organizations, initially was focused on the EDIINT standards (extended discussion is provided below). This industry, too, however, adopted the ebXML approach with an announcement from its industry association, the Global Commerce Initiative (GCI - http://www.globalcommerceinitiative.org), and are now extending their earlier work, such as CPFR that was created by Wal-Mart (see http://www.cprf.org) and promoted through the VICS industry group (http://www.vics.org). With the recent merger of RosettaNet, the EAN.UCC is also focused on continuing the developing e-business methodologies (such as CPFR), based on ebXML principles with efforts such as their Global Registry.

In Europe, Odette International is an organization formed by the automotive industry that sets the standards for e-business communications, engineering data exchange, and logistics management, and links the 4,000 plus businesses in the European motor industry and their global trading partners. The OFTP (ODETTE File Transfer Protocol) was defined to extend the traditional file transfer capabilities of FTP (over TCP/IP) to support binding of EDIFACT-specific messages and transport over a wider range of communications, including X.25 networks and ISDN facilities.

What exactly is a "standards organization"? Clearly, groups and organizations such as ISO, ANSI, and UN/CEFACT would fall into this classification (although the latter is the one of the few groups mandated by national or international laws).

Typically, standards organizations are exemplified by broad open memberships and are distinguished by their controlling bodies being unaligned. Consortiums, on the other hand, have boards consisting of commercial entities that control and steer the focus to meet their own business requirements.

However, they all develop specifications that they promote as standards and, as such, attempt to focus on the definition that ISO uses for its own work:

Standards are documented agreements containing technical specifications or other precise criteria to be used consistently as rules, guidelines, or definitions of characteristics, to ensure that materials, products, processes and services are fit for their purpose.

Of course, some use standards to promote and obtain a commercial position within a marketplace. Others use standards to drive quality and protect consumer interests.

The ISO standards organization has served as a repository and steward for many standards that originally were developed in the commercial world.

In addition to the industry trade associations and groups, many consortia of computer industry companies and large IT users have been formed over the past 10 years with a mission to create "documented agreements" among their members (e.g., the W3C, OASIS, OMG, WS-I, and so forth) with the goal of promoting their commercial solutions jointly within the marketplace.

The first step in choosing a protocol for XML-based e-business is to document the available choices today, where they are purposed, and their unique aspects. Excluded from this, however, are the industry-specific protocols mentioned earlier (such as E-5, EDM, RNIF), because these are inherently restricted to only the industries utilizing them, rather than broader solutions.

This list therefore includes the following (acronyms are referenced elsewhere in this document):

We now continue by examining these and analyzing their focus and mission.

By far, the dominant historical protocol used to transfer large B2B documents is FTP. Because of the robustness of the TCP/IP protocol for data transport, FTP is considered a highly reliable method to move large XML documents. However, in its pure standardized form (RFC 959), FTP lacks some critical features required for secure, reliable messaging:

However, recognizing these weaknesses, software vendors developed value-added features in their commercial products, both FTP clients and servers.

In 1999, the IETF approved an informational RFC that documented the FTP security considerations (http://www.ietf.org/rfc/rfc2577.txt). Several IETF draft documents have been submitted to address potential solutions. The two key ones are:

Despite its ease-of-implementation and multi-vendor support, the use of FTP was not considered viable as a transport for inter-enterprise EDI exchanges over the Internet for two main reasons:

In February 1996, The EDIINT (EDI over the Internet) Working Group (WG) of the IETF was formed to understand the requirements and develop specifications to address these needs.

In August 2001, the EDIINT WG released an important document titled "Requirements for Inter-operable Internet EDI" (http://www.ietf.org/internet-drafts/draft-ietf-ediint-req-09.txt). This document's focus was on EDI MIME content transported using SMTP (Simple Mail Transport Protocol), the Internet's mail or messaging system. The first draft specification was released in May 1998 (http://www.ietf.org/proceedings/98aug/I-D/draft-ietf-ediint-as1-08.txt) and became known as AS1 (Applicability Statement 1). This document has been superseded by RFC 3335 ("MIME-based Secure Peer-to-Peer Business Data Interchange over the Internet (http://www.ietf.org/rfc/rfc3335.txt?number=3335).

As concerns increased over security issues when business data is transported over the Internet, the feasibility of using FTP or SMTP to transport business data through the corporate firewall became questionable. However, the commercial use of the WWW forced security experts to safely allow HTTP sessions into a Web server (in the DMZ), with connectivity to business applications behind the firewall.

The AS2 specification (Applicability Statement 2), which is still an IETF Draft (http://www.ietf.org/internet-drafts/draft-ietf-ediint-as2-11.txt), modifies the AS1 specification by incorporating the packaging/binding of the data (including security) to the HTTP transport protocol and specifically includes additional (non-EDI) data types.

In addition to the support of S/MIME or PGP/MIME for data security, AS2 also includes support for HTTPS, where "S" implies the use of SSL v3.0 or TLS v1.0 to secure the HTTP Session between the AS2 client and server. On the server side, an HTTP server can provide the SSL/TLS feature.

These protocols represent messaging solutions developed by three key players in the computer industry (Microsoft, IBM and Sun, respectively). Each vendor provided open APIs so that a wide range of vendors and users could participate in their initiatives to standardize these frameworks.

The BizTalk framework (implemented in the MS BizTalk Server product) was the first commercial protocol to use XML schema for enveloping the documents. The BizTalk messaging protocol was an early version release of the Simple Object Access Protocol (SOAP).

MQSeries provides business-quality messaging features for those applications designed around message queuing and message-driven processing. The MQSeries Framework defines several APIs so that users and independent software vendors are provided with the opportunity to extend (or even replace) IBM's implementation. MQSeries has been rated the #1 EAI and middleware product for several years (based on number of licenses sold) and it is supported on almost every computing platform and operating system.

Not to be outdone by the IBM and MS architects, Sun Microsystems defined the Java Message Service (JMS) as a vendor-agnostic API for enterprise messaging. However, unlike the other vendors, Sun defined only the messaging services and APIs, and left the implementation to other vendors (e.g., http://www.sonicsoftware.com and http://www.spiritsoft.com). For a complete discussion of JMS, I suggest the Java Message Service, published by O'Reilly & Associates (December 2000) and written by Richard Monson-Haefel and David Chappell.

SOAP is most often considered in the broader context of the Web services framework, and the ongoing standards activities around SOAP fall in the domain of the XML Protocol Working Group in the W3C (http://www.w3.org/TR/SOAP).

The model for Web services is more akin to the RPC (Remote Procedure Call) specifications for point-to-point application interactions over SOAP messaging than to the formal messaging layers discussed so far. Indeed, SOAP itself stems from the earlier work on XML-RPC (http://www.xmlrpc.com/spec).

Web services has its own description syntax in XML. The Web services Description Language (WSDL - http://www.w3.org/TR/wsdl) defines the parameters needed to interact and communicate with a particular Web service component.

Formally, the WSDL is an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information. The operations and messages are described abstractly, and then bound to a concrete network protocol and message format to define an endpoint. Related concrete endpoints are combined into abstract endpoints (services). The bindings provided in the specifications today describe how to use WSDL in conjunction with SOAP 1.1, HTTP GET/POST, and MIME.

The overall Web services implementation model divides the entire process into three steps or domains: describe a service, bind it with a concrete implementation, and publish it over a registry. A full description of these aspects is out of scope here, but the process is mentioned for completeness only.

For the purposes of protocols, Web services depend on the underlying SOAP protocol that they are directly coupled to for delivery, verification, and security. Therefore, Web services have the same characteristics and reliability profiles as SOAP itself (see the previous discussion on SOAP as a protocol). Web services today are useful as a lightweight and adaptive real-time interaction solution, where access control to content and delivery verification are not critical. For these reasons, Web-service implementations today have predominantly been internal between application components.

The W3C is working hard on enabling the specifications around Web services to extend the useful and business models that can be reliably addressed. Many vendor members of W3C have declared support for and are implementing the existing base capabilities today. Clearly, Web services are a significant technology for application interactions using Internet-based messaging protocols.

The SOAP 1.1 specification was developed by IBM, Microsoft and UserLand Software, Inc., and was release in May 2000. This specification defined a complete message enveloping structure based on an XML Schema sent within an HTTP request. The specification was then submitted to the W3C, which has now taken over further development of the specifications.

SOAP 1.2 was released earlier this year and is often called SWA (Soap with Attachments), because a major enhancement to Version 1.1 was the creation of SOAP bindings that transmit attachments along with a SOAP envelope, and provide for reference of those attachments from the envelope. SOAP attachments are described using the notion of a compound document structure, consisting of a primary SOAP message part and zero or more related documents parts known as attachments.

While the previous specifications discussed have all been adopted widely in North America, the following information describes two protocols that were successful internationally, but were never adopted seriously in the US.

X.435 is an X.400-based standard designed to support electronic data interchange (EDI) messaging. It is an ITU-T recommendation (http://www.itu.int/ITU-T) and widely used in countries where strong X.400 networks were implemented (for you old-timers, the ITU-T was formerly named CCITT).

The MHS message structure is similar to the MIME message structure. It has both a header and a body. The body can be broken up into multiple parts, with each part being encoded differently. For example, one part of the body may be text, the next part a picture, and a third part encrypted information.

EDI needs more stringent security than typical e-mail because of its business nature. In support of these security requirements, X.435 defines, in addition to normal EDI messages, a set of EDI "notifications." Positive notification implies that the recipient has received the document and accepts the responsibility for it; negative notification means that the recipient refused to accept the document due to a specified reason; and forwarding notification means that the document had been forwarded to another recipient. Together, these notifications form the basis for a system that can provide security controls comparable to those in the paper-based system that EDI replaces.

The OFTP (ODETTE File Transfer Protocol) was defined by the European automotive industry to extend the traditional file-transfer capabilities of FTP (over TCP/IP). OFTP supports acknowledgments, binding of EDIFACT-specific messages, and a wider range of communications, including X.25 networks and ISDN facilities.

With thanks to Matthew Mackenzie of XML Global Technologies Inc., for providing text here - drawn from his contributions on the ebMS IIC implementation and specification work.

The ebXML specifications have been developed over the past three years by technical teams with membership drawn from the UN/CEFACT and OASIS standards groups. The ebMS work is continuing under the auspices of OASIS (http://www.oasis-open.org/committees/ebxml-msg).

Traditional business information exchanges have conformed to a variety of standards-based syntaxes. These exchanges were largely based on electronic data interchange (EDI) standards born out of mainframe and batch processing. Some of the standards defined bindings to specific communications protocols. These EDI techniques worked well; however, they were difficult and expensive to implement. Therefore, use of these systems was normally limited to large enterprises possessing mature information technology capabilities.

The proliferation of XML-based business interchanges served as the catalyst for defining a new global paradigm that ensured all business activities, regardless of size, could engage in electronic business activities. The prime objective of ebMS is to facilitate the exchange of electronic business messages within an XML framework. Business messages, identified as the "payloads" of the ebXML messages, are not necessarily expressed in XML. XML-based messages, as well as traditional EDI formats, are transported by the ebMS. Actually, the ebMS payload can take any digital form-XML, ASC X12, HL7, AIAG E5, database tables, binary image files, etc.

Prominent in the ebXML ebMS specification is the notion of a separation between a message service specification and the communication protocol over which the service may operate. Although the ebMS is designed to be protocol-neutral, the specification notes how the message service may be bound to HTTP or SMTP transport protocols. Sending a SOAP message directly over HTTP or sending a SOAP message as the body of an e-mail message demonstrates this.

It is also helpful to briefly discuss the category of messaging and indeed the actual messaging applications for which the ebXML Message Service was designed. Typically, messaging systems such as ebMS are contained under the umbrella of Enterprise Messaging. Categories within Enterprise Messaging may broadly be defined as: Enterprise Application Integration (eAI), Business-to-Business (B2B), Web services, and Electronic Data Interchange (EDI).

Early indications suggest that the adoption of ebMS will be primarily as a messaging solution for eAI and B2B. The ebMS delivery may also enhance or replace some EDI messaging solutions because of the superior business process control functions in ebXML that ebMS supports.

In contrast, ebMS may not be suitable for Message Oriented Middleware (MOM) solutions. MOM is dominated by IBM's MQ Series with a small but growing segment of the market moving to Java Message Service-based solutions (JMS). It works on the concept of queues as a means to reliably and securely delivers messages between applications.

Indeed, the overhead of SOAP plus attachments enveloping required by ebMS will probably make ebMS too "heavy" for most intra-enterprise application integration. However, the open nature of the ebMS standard and its support for a wide range of business payloads makes it ideal for inter-enterprise business messaging.

It is important to note the distinction between ebMS and an ebXML Message Service Handler (MSH). An ebXML MSH is a software implementation of the ebMS. To execute a message delivery or messaging conversation, it is required that an ebXML MSH exists at each end of the wire, thus enabling the secure and reliable delivery of messages. SOAP (Simple Object Access Protocol) has proven itself as a useful messaging protocol among software developers, particularly between heterogeneous applications within an enterprise. SOAP's success, based in large part on its foundation in XML, has led system architects and developers to explore ways of extending SOAP beyond the enterprise. This has led to SOAP-based Web services that are appealing because of their use of open standards (XML and HTTP), and their intrinsic simplicity of design.

However, these same system architects and developers have found that SOAP as a standalone messaging protocol may not, in fact, be directly useful in the world of business-to-business commerce. For SOAP, in which standard security mechanisms do not exist, there are no agreements over which a message-based "conversation" can take place, and there is no reliability guarantee in message delivery. To be of practical use, a messaging standard should incorporate at least these features. The ebXML Message Service (ebMS) specification tackles these issues head on and is designed to allow users to extend the reach of their applications to participate in secure, reliable, and standards-based exchanges with external partners.

The ebXML Message Service's usefulness is getting the validation its authors and champions had hoped for. By design, ebMS is Internet-friendly, has the reliability and security features that enterprise users require, is suited for XML and non-XML business payloads, and-most importantly-is getting endorsements from major industry associations such as GCI, RosettaNet, ATA, AIAG, and many more. It is clearly the emerging standard for e-business messaging exchanges.

Now that we know some of the choices that exist, how do we decide which protocol fits a specific business need? Most people avoid choosing protocols that are primarily supported by only one (or two) vendors, or supported by only one industry, unless nearly all of their transaction activity is in that industry. (Examples of industry specific transports are RosettaNet and the FIX protocol.) Conversely, products like IBM's MQSeries and Sun's JMS are not considered "vendor-proprietary" because they have open APIs, are supported on a wide variety of computing platforms, and are broadly adopted. So, overall, people gravitate toward open protocols that will enable them to communicate with the maximum numbers of their business partners through the one interface. The goal is to avoid having to install and support multiple communications systems.

In an effort to define what quality metrics make for excellence in message exchanges, the Business Quality Messaging Group (BQM SIG) has defined a functional specification of the minimum set of attributes and behaviors that messaging products must support in order to be considered as "BQM-compliant." The BQM SIG was formed in 1997 as a special interest group of the Electronic Messaging Association (http://www.ema.org).

Table 1 lists the protocols along the top, and on the left-hand side is a list of protocol features. Inside the boxes are my ratings on how each protocol implements these features.

Note 1: Includes Authentication, Authorization, Data Integrity, Confidentiality, and Non-Repudiation

Note 2: Also known as "Reliable Messaging"; includes Guaranteed Delivery, No Duplicate Delivery, Unique Message IDs, Message Types, Message Priorities, Acknowledgement Modes.

Note 3: A rating of "H" implies an "easy" implementation, the ratings here reflect vendor implementation, not necessarily end-users working with products with embedded messaging solutions.

Note 4: Recently, there have been several new specifications released for review that improve the security and reliability associated with using SOAP messaging for XML document. There are not reflected in this evaluation.

If the Internet is not involved (i.e., only leased lines or frame relay connections are used), then FTP is the simplest most direct point-to-point protocol to deliver both small and large messages. However, it has no Business Quality Messaging (BQM) features, so you will have to build your own enhancements to your FTP implementations if you want to ensure that messages do not get lost (e.g., overwritten in the same directory) or delivered multiple times after a restart/recovery situation. Base FTP is suitable for internal applications. However, for external exchanges Secure FTP (or VPN security) is to be preferred and is widely available from software vendors, firewall vendors, and service providers.

If your solution will utilize only J2EE-compliant servers, then several vendors have delivered robust implementations of JMS Server products that are compatible across many platforms. Generally, JMS implementations are less costly and easier to implement then MQSeries. MQSeries provides arguably the best implementation of the BQM functions today. However, both of these technologies provide mostly raw delivery services and have no features designed for Internet transport or B2B/EDI integration.

For communications over the Internet, additional security mechanisms are required. If real-time EDI or rapid responses are not required, AS1 (i.e., SMTP and S/MIME with EDI extensions) is the simplest to implement using existing Internet mail capabilities. SOAP (v1.2) and AS2 both provide conversational, real-time messaging to exchange documents, but AS2 is preferred for B2B messaging because it has included all of the BQM and Security Services required for reliable exchange of business critical documents.

Implementations of ebXML are just becoming available from major vendors (see (http://65.201.162.121/dailywire/S20021001_10.html) and support from several industry consortia clearly indicate that ebXML represents a new, robust framework for XML-based B2B messaging framework.

The ebMS transport, integrated within an ebXML framework, provides significantly more capabilities, including business process management, communications partner profiles, enveloping, routing, security, and control features.

Finally, it is important to note that this paper did not attempt to evaluate these protocols in terms of the "cost of implementation". This may be a critical factor in making a decision and should be considered on a case-by-case basis for the specific business application.