XML 2003 logo

XML MESSAGING - ACROSS THE CHASM

Abstract

The abstract was not available at the time the proceedings were created. Please check an updated version of the paper abstracts at the conference proceedings web site.


Table of Contents

1. INTRODUCTION
2. HISTORICAL PERSPECTIVE
3. WHERE DO THESE XML MESSAGING FRAMEWORKS COME FROM?
3.1. MARKETPLACES/EXCHANGES
3.2. INDUSTRY CONSORTIA
3.3. OTHER STANDARDS ORGANIZATIONS
4. INDUSTRY STATUS
5. UCCNET - DOES YOUR DATA NEED SYNCHRONIZATION?
6. OTHER INDUSTRIES
7. WEB SERVICES (WS) AND ebXML - HOW DO THEY FIT IN ?
8. AN INTERNATIONAL PERSPECTIVE
8.1. ASIA-PACIFIC
8.2. EUROPE
9. THE FUTURE OF XML MESSAGING
Biography

1. INTRODUCTION

In the new millennium, XML has moved to the preeminent position as the accepted language for defining new business-document formats. The acceptance of XML Messaging as the de facto standard for B2B standards, and the associated commitment from Industry Associations to drive these standards into their end-to-end supply/demand chains, has been nothing less than phenomenal.

XML messaging standards have become a fever. What the Hi-Tech industry developed (RosettaNet standards), the Chemical Industry adopted quickly and customized to their needs (Chem eStandards). The automotive industry has chosen the OAGIS Version 8.0 as the core XML technology for the exchange of new business messages and documents.

Last year, my paper at XML 2002 focused on the growing popularity of Internet-based Transport Protocols whose enveloping and security schemes were based on emerging XML technologies. The most successful program involved the adoption of EDI-INT AS2 in the Retail market. However, since this was not a "new" B2B program (just a migration from one protocol to another), EDI-formatted documents were still the primary content exchanged over these AS2 sessions.

The release of RosettaNet V2.0 standards and several new initiatives in the Retail and Automotive industries will drive more XML messaging across many other market segments.

In this paper, I will cover the current status of two successful XML messaging programs: RosettaNet and EAN.UCC Global Data Synchronization Network. I will also summarize some of the activities in other industries. My focus will be on the impact of XML in North America but, to a lesser degree, I will also discuss some activities in Europe and the Asia-Pacific/Japan regions.

To borrow a concept made famous by Geoffrey A. Moore, XML Messaging technologies have moved "beyond the chasm"!!

2. HISTORICAL PERSPECTIVE

In the early days (let's say pre-EDI), the choice of business document formats was totally left up to the application developers or, in some cases, proprietary formats developed for each industry. The most highly used formats for data were those which best fit on a magnetic tape, and the "messaging protocol" of that time period was Batch RJE (Remote Job Entry), which was used to exchange tape files between computers (using 80-character or 132-character records).

With the emergence of EDI standards, most EDI-formatted data was still exchanged using a "wrapping" into the 80-byte or 132-byte records. The commercialization of the Internet (circa 1994) introduced a new set of data-transfer protocols which eliminated the concept of data formatted as fixed length records and replaced it with the concept of a "TCP/IP data stream" which supported the transfer of variable length business information (e.g., in files, messages or documents). The next major change came when the technologies of the World Wide Web (HTTP/HTML/XML) were adopted by commercial enterprises to do business over the Internet.

3. WHERE DO THESE XML MESSAGING FRAMEWORKS COME FROM?

Several stakeholders have driven the evolution of new XML messaging standards:

  • Commercial implementers of marketplaces and exchanges

  • Industry vendor consortia such as the W3C, OASIS and OAG

  • Other standards organizations such as ISO, ANSI and UN/CEFACT

Let's look at how each of these players is involved.

3.1. MARKETPLACES/EXCHANGES

One of the early commercial uses of XML technology came from the vendors who developed software for public B2B e-marketplaces and private exchanges. The industry was legitimized by the formation of CommerceNet in 1994, a consortium of eCommerce vendors with a mission "to transform the Internet into the world's largest and most efficient marketplace, dramatically changing how the world transacts business" (http://www.commercenet.com).

Within two years, the first XML "standards" had been developed to automate the real-time, secure, Internet-based procurement process, referred to as the OBI (Open Buying on the Internet, http://www.openbuy.org and the Information and Content Exchange (ICE) protocol (http://www.icestandard.org and http://www.w3.org/TR/NOTE-ice). From these groups, vendors spun off to develop their proprietary solutions based on XML message formats (and in fact the design and architecture from ICE was copied widely, and manifestations from it can be seen widely today). Such companies as CommerceOne (xCBL message formats) and Ariba (cXML message formats) were the early entrants, and each developed its own messaging transports to exchange these XML business documents between buyers and sellers over the Internet.

3.2. INDUSTRY CONSORTIA

Dissatisfied with the slow adoption of XML by traditional standards organizations, several of the major industry action groups chose to design their own frameworks to exchange EDI/XML documents reliably over the Internet. The earlier work on OBI led to the creation of the RosettaNet consortium, focused on the electronics industry supply chain. RosettaNet distinguished itself by focusing on the process, rather than on the transactions, and developed a set of definitions for exchanges within the electronics industry. It defined a complete set of Internet-based e-business standards, including:

  • XML-based dialogs that define business processes between trading partners ("PIPs") (

  • An implementation framework covering trading partner agreements and information exchange ("RNIF") (

  • Business and technical directories to reduce confusion resulting from each company's uniquely defined terminology.

  • Its own messaging-control mechanisms and envelopes.

Founded in 1998 by 40 organizations, RosettaNet grew into a global consortium of companies throughout the hi-tech industry (electronic components, semiconductor manufacturing, and information technology). On August 5, 2002, RosettaNet became a subsidiary of the UCC, while continuing to operate as a separate entity serving its members.

The retail industry, through the EAN.UCC standards organizations, initially was focused on the EDIINT standards. With the recent merger of the UCC and RosettaNet, the EAN.UCC is continuing the development of e-business methodologies (such as CPFR and GDS), based on XML messaging frameworks.

3.3. OTHER STANDARDS ORGANIZATIONS

What exactly is a "standards organization"? Clearly, groups and organizations such as ISO, ANSI, and UN/CEFACT would fall into this classification (although the latter is one of the few groups mandated by national or international laws).

In addition to the industry trade associations and global standards organizations, many consortia of industrial companies and large IT users have been formed over the past 10 years with a mission to create "documented agreements" among their members (e.g., the W3C, OASIS, OMG, WS-I, and so forth) with the goal of promoting their commercial solutions jointly within the marketplace.

Fundamentally, they all develop specifications that they promote as standards and, as such, attempt to focus on the definition that ISO uses for its own work:

Standards are documented agreements containing technical specifications or other precise criteria to be used consistently as rules, guidelines, or definitions of characteristics, to ensure that materials, products, processes and services are fit for their purpose.

4. INDUSTRY STATUS

ROSETTANET - FROM IMPLEMENTATION TO INTEROPERABILITY

RosettaNet was truly a pioneer in its development of a complete B2B Framework, which is built upon a comprehensive set of business process specifications, XML-based business documents, and dictionaries defining the properties for processes and products common to the industry. Here are some of the advances made since the turn of the century:

July 2001 - A product launch from Microsoft, the BizTalk Server Accelerator for RosettaNet, represents a major step in commoditizing this important technology. The BizTalk framework (specifications released in 1999) was the first commercial protocol to use XML schema for enveloping the documents. The BizTalk messaging protocol was an early version release of the Simple Object Access Protocol (SOAP).

February 2002 - RosettaNet.org released the specifications for Version 2.0 of the RosettaNet Implementation Framework (RNIF). The core of RNIF2.0 is a new RosettaNet Business Message format that integrates and packages all of the necessary components for reliable message transport. The new RNIF v2.0 packaging contains an instance of Preamble, Delivery and Service Header components; each itself defined by XML Schemas. Other changes were made to the message structure to bring it into compliance with existing standards (e.g., MIME and S/MIME), allowing this message to become "transfer protocol independent".

August 2002 - RosettaNet and the Uniform Code Council (UCC) announce merger and strategic partnership to help advance the adoption of XML-based B2B integration standards horizontally across their industry members.

December 2002 - Intel, a leader in the implementation of RosettaNet in its supply chain, announces that it has migrated over 10 percent of it demand/supply chain transaction to RosettaNet standards. This represented over $3B in customer orders and $2B in supplier purchases during 2002.

January 2003 - Global eXchange Services (GXS), a leading supplier of B2B products and services and a long-time player in the EDI Services market, announced its RosettaNet Basics, a package design to enable mid-tier suppliers an affordable set of software and services required to implement RosettaNet with their key high-tech customers and integrate these XML transactions with their back-office systems.

February 2003 - Ten software vendors (including GXS) complete the first trials of the RosettaNet Software Interoperability Testing. Each of the ten participating companies performed a suite of 36 test scenarios to validate their conformance to standards and their ability to successfully orchestrate RNIF v2.0 XML-based dialogs based on messages and choreography defined by the RosettaNet Partner Interface Processes (PIPs).

June 2003 - RosettaNet established a formal collaborative alliance with OASIS to further extend the focus on interoperability that started with the UCC merger. A primary focus of the alliance is to develop universally accepted standards, such as those developed by OASIS (e.g., ebXML and UBL).

August 2003 - RosettaNet announces the launch of a new foundational program: RosettaNet Multiple Messaging Services in an effort to investigate the business requirements and convergence opportunities for RosettaNet to extend its support of messaging standards.

5. UCCNET - DOES YOUR DATA NEED SYNCHRONIZATION?

UCCnet was formed in 1999 in response to UCC member companies who were seeking a cost-effective solution to problems in the validation and synchronization of product information between manufacturers and retailers. In the CPG industry a research study showed that more than 30% of product data, and their attributes, were incorrect within retailer's systems. These errors and inconsistencies caused more than half of the invoices generated by vendors to be rejected by their customers' business applications resulting in returns, charge-backs, additional shipping costs and significant delays in the delivery of new items to retail shelves.

In response, UCCnet designed a one-stop service offering to provide:

  • The GLOBALregistry ™ where all product data and attributes would adhere to global standards and would be "validated" to ensure that the data complies with EAN.UCC Standards, and

  • SYNCservices - a data synchronization service where vendors "publish" new products (or revised product attributes) and retailers "subscribe" to these products and "authorize"(a.k.a., "confirm") their inclusion in their Item Master catalog, ensuring that the data is valid and synchronized between the two Partners and remains that way

The long-range vision of UCCnet is the continued ability to synchronize product data. There is an envisioned evolution to a federation of interoperable, certified Data Pools (i.e., in-house catalogs or 3rd party catalog service providers) and a GLOBALregistry that collectively provide for the synchronization of Master Item Data between trading partners on a global basis. The architecture for this vision is referred to as the Global Data Synchronization Network (GDSN).

Most importantly (for this paper, anyway) is the fact that there are UCCnet Messages currently defined by XML DTDs (for backward compatibility reasons) and XML (XSD) Schemas. This facilitates the use of low cost XML tools and secure Internet transport.

So, has UCCnet "crossed the chasm" ? The best way I could answer that is to give you an extract of a letter distributed by Wal*Mart Stores Inc. earlier this year:

"Dear Valued Wal-Mart Supplier:

Wal-Mart Stores Inc. has been using UCCnet's standards based Global Item Registry and synchronization services for the last year. We are currently in production with over 50 of our suppliers.

At Wal-Mart, we expect to receive all item data electronically by January 2004. Wal-Mart is ready to begin item data synchronization with your company as soon as possible.

We appreciate our trading relationship with your company, and look forward to a more productive and efficient relationship in the future."

Similar letters have been distributed by many other major retailers in the several different industry segments (e.g., grocery and hardlines). Lets review some of the major accomplishments:

April 2000 - UCCnet, went live in July with a half-dozen food industry giants, including Procter & Gamble Co., Supervalu Inc., Ralston-Purina Co. and PepsiCo. The Internet-based exchange was designed to allow the sharing of synchronized, real-time updates on prices and shipment information by swapping data according to XML structures set by the Uniform Code Council Inc. (UCC). The UCCnet technical standards are intended for open use by all supply-chain exchanges so they can be linked.

July 2001 - As part of its Electronic Commerce Strategy, The UCC announces the formation of an XML Steering Committee whose goal is to set the overall direction of XML standards for B2B and B2C programs.

August 2001 - The first set of XML Schemas for eCommerce, jointly developed by UCC and EAN International, is released as global voluntary standards for the exchange of XML documents.

6. OTHER INDUSTRIES

Every major industry segment has one or more development and/or implementation programs involving XML messaging. However, I chose RosettaNet and UCCnet primarily due to my personal experience with commercial implementations of these technologies. Here is a list of some of the major activities and events in other industries:

CHEMICAL

July 2000 - CIDX (Chemical Industry Data Exchange) announces a new initiative, referred to as Chem eStandards™, to develop a broad set of nonproprietary, XML standards to facilitate business-to-business data exchange across the chemical industry worldwide. CIDX chose to leverage the work already done by RosettaNet.

June 12, 2002 - CIDX, PIDX (Petroleum Industry Data Exchange) and RAPID (representing the agricultural industry) announce a collaborative initiative to develop common, cross-industry XML-based standards for trade within and between their respective industries.

AUTOMOTIVE

APRIL 2002 - The OAG (Open Application Group) announces the release of OAGIS version 8.0, WHICH defines a new structure, the BOD (business object document), as a horizontal message architecture. In parallel, the AIAG endorses the Open Applications Group (OAG) and recommends the use of BODS as a starting point for trading partners wanting to exchange XML documents.

AUGUST 2002 - STAR (Standards for Technology in Automotive Retail), the IT standards body for the North American retail automotive industry, and OAG approve seven XML standards (BODs) for automotive dealer-to-manufacturer transactions (e.g., Parts Order/Return/Pick_List, Repair Order, Vehicle Service History and Warranty Reconciliation). By July 2003, the list of published BOD specifications has grown to 62(http://www.starstandard.org).

MARCH 2003 - The AIAG launches the Inventory Visibility and Interoperability (IV&I) Project. The IV&I project team will tackle the problems related to inventory visibility tools and the lack of interoperability. The XML methodology used for data content development (of business process message exchanges) will be based on the OAGIS BOD guidelines.

HEALTHCARE

September 1996 - HL7 (Health Level Seven, Inc.), a not-for-profit, ANSI-accredited standards organization leading healthcare standards development, announces the formation of the XML Special Interest Group to recommend the use of XML as an alternative syntax for HL7 messages.

September 2000 - HL7 membership ratifies Version 1 of the Clinical Document Architecture, which defines an XML framework for the exchange of clinical documents. CDA is approved as an ANSI Standard in November 2000.

December 2001 - Initial release of HL7 Version 3 messages slated to use only XML encoding.

7. WEB SERVICES (WS) AND ebXML - HOW DO THEY FIT IN ?

Both WS and ebXML are frameworks for conducting eBusiness over the Internet and B2B integration. However, the use of XML is growing within many functional layers of these frameworks (business process specifications and message choreography , transport protocol enveloping, and interfaces to registries and repositories). For example, both ebXML and WS use SWA (SOAP with Attachments) as the core of their message transport architecture.

WS has its own description syntax in XML. The Web Services Description Language (WSDL - http://www.w3.org/TR/wsdl) defines the parameters needed to interact and communicate with a particular Web service component. Formally, the WSDL is an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information.

8. AN INTERNATIONAL PERSPECTIVE

I would not want the reader to be left with the impression that XML technology has not yet taken a foothold in countries outside the USA. So, here are a few examples of major industry and government initiatives in other parts of the globe.

8.1. ASIA-PACIFIC

For the electronic components and semiconductor marketplaces, the large number of supply chain partners throughout this region has made RosettaNet the most popular B2B XML framework. Starting with Japan in early 2000, the RosettaNet organization has announced expansion of its operations in this region with new affiliate organizations in Korea, Malaysia, and Singapore. In 2003, Thailand and the Philippines were added as affiliates and China's Ministry of Science and Technology (MOST) formed a strategic alliance with RosettaNet.org that will promote the adoption and implementation of RosettaNet standards within China's high technology industry.

Established in January 2002 at the University of Hong Kong, the Center for E-Commerce Infrastructure Development (CECID) conducts e-commerce research and development with the vision of helping businesses to increase their competitiveness in the international arena. This year, the CECID announced two "pilot" applications developed under their Project Phoenix are ready to move into production.

The first is a B2B e-procurement solution that replaces paper documents and manual reconciliation with electronic XML documents (Deliver Notices and Invoice Statements) and automates the current manual reconciliation of these two documents. The Pilot participants were Saggio Limited (the largest office supplies procurement companies in Hong Kong) and the MTR Corporation (which operates the underground metro-line in Hong Kong).

The second successful pilot is a B2G (business-to-government) implementation to facilitate the electronic submission of Dangerous Goods (DG) Manifests to the Marine Department (which operates the Port of Hong Kong). The first company to use this new automated capability was Orient Overseas Container Line (OOCL).

8.2. EUROPE

A major XML standards initiative in Europe is being driven by the financial services industry. XML standards are replacing message formats and data dictionaries developed many years ago by ISO and SWIFT (the industry-owned cooperative supplying secure, standardized messaging services and interface software to 7,500 financial institutions in 199 countries). These new standards, referred to as the ISO 15022 XML standards" are considered a "fundamental building block" to achieving higher levels of end-to-end Straight Through Processing (STP) for the financial transaction lifecycle.

9. THE FUTURE OF XML MESSAGING

When it comes to predicting the future, I generally quote Alan Kay's well-known insight:

"The best way to predict the future is to invent it"

so I prefer to make two predictions based on the trends that I see today in my business and standards development activities.

  1. BPO - The Value is in the Total Solution

    Technology, including XML toolkits and capabilities, will benefit from the open source and free software initiatives. Successful vendors (of products and services) will focus on solutions to real business problems. One example is Business Process Outsourcing (BPO). Many companies will not want to replace their legacy EDI systems with new Web-based business process integration and management solutions that use XML Frameworks like RosettaNet and UCCnet's GDS. This will offer the vendors an opportunity to design and operate these capabilities as part of BPO services.

  2. CONVERGENCE - The road toward global, horizontal standards

    Throughout the world and throughout almost every major industry, efforts have begun to converge towards universal standards. In the B2B market outside the US, ebXML has already taken the lead as the standard framework for convergence. In the US, many industries developed their own frameworks initially, but they have initiated programs and work groups to look at convergence strategies for both the Message Transport and Message Content (payload).

    Both of these ideas are simply the "lessons learned" from the failure for EDI standards to (1) become global and (2) be integrated into applications.

  3. BCM/CAM - XML Messaging Made Easy

    Although XML standards have been widely accepted, there has still been significant time spent by companies, industry consortia and even standards organizations developing unique XML Vocabularies for business transactions for a "localized implementation". OASIS has formed two Technical Committees working on technologies that portend the coming of an open, global, interoperable set of specifications that defines a set of agile techniques to engineer the "XML message and dialog implementation" from the business processes and business collaboration agreements.

    BCM (Business-Centric Methodology) focuses on a business mechanism which allows business users to design and develop business process scenarios and how components of the scenarios collaborate to produce an e-business solution. From these "business process scenarios, CAM (Content Assembly Mechanism) defines a mechanism to bring business process context and localized implementation business rules to "assemble" valid, consistent XML Schemas to be incorporated in an XML Messaging Framework.

Biography

Mark has more than 30 years of technology development and consulting experience focusing on data networking, B2B messaging, EDI and e-Commerce solutions.

Mark has worked for Global eXchange Services (GXS, formerly GEIS) over 20 years and has held management positions in engineering, product management, proposal development, technical consulting and, most recently in solutions architecture. He has also participated in several industry groups including AIAG, CIDX, and AISI.

Mark has been assigned some of the most challenging project management responsibilities throughout his career supporting clients such as Apple, Phillip Morris and Ford Motor Company.

Mark has been heavily involved with implementation of secure Internet-based messaging protocols (e.g., AS2, S/FTP and RNIF) and serves on the steering committee of the ebXML Joint Marketing Team (JMT) and the eBusiness Work Group at the UN/CEFACT Forum (TMG-EBWG).