Testing tools for conformance and interoperability are emerging as a key technology for enabling and maintaining interoperability between e-business partners. Verifying adherence to a common standard (conformance) is just the first step. In addition, it is critical that the selected options, bindings, and deployment settings of the implementations, are compatible across partners. Also, as a stack of several standards - as in ebXML - is usually involved, interoperability increasingly relies on the bindings across these standards, their version compatibility, and explicit contracts.

This paper reviews the challenges and solutions of interoperability testing for the current set of ebXML standards, presents the status of the ebXML Test Framework and test suites developed by the ebXML Implementation, Interoperability and Conformance Technical Committee (IIC) under OASIS, and also compares with similar efforts on the set of standards more broadly understood as defining Web services.

Such testing initiatives are taking place under the Organization for the Advancement of Structured Information Standards (OASIS), and also under Web Services Interoperability organization (WS-I). Although the scope of IIC work spans all ebXML specifications, we focus here on messaging, as the IIC has recently completed test suites for this specification. The paper also describes the underlying IIC testing architecture — the ebXML Test Framework - currently under development at NIST. Another organization, WS-I, has also chartered a testing working group, whose goal is to define test assertions and develop testing tools for verifying interoperability between various instances and platforms of web Services. We also comment on this initiative and its testing approach, and compare with the ebXML IIC work.

The authors have been involved in both organizations and activities for more than a year, working at design and implementation level.

Among the general challenges in testing e-business systems that are addressed, along with proposed solutions, are:

· As interoperability testing is an "expensive" activity (in terms of logistics, time and disruption), it is beneficial to focus first on the conformance of each end-point to the reference specifications, and to the proper combination of these (also called "profile" in WS-I).

· As the systems of e-Business partners will evolve and upgrade independently (versions, proprietary capabilities), it is critical to be able to execute interoperability tests on-demand, point-to-point, on a frequent basis. We call "interoperability maintenance" this form of regression testing. This calls for automation, ease of use: executable XML-scripted test suites control the choreography of exchanges over distributed test components.

· Interoperability starts with the proper deployment and configuration of a B2B platform. The IIC has defined a deployment template, from which a community of business users can define and share guidelines on how to use and deploy a standard (here, ebXML MS).

· Future test centers will provide basic logistic and testing services (coordination, notification, reporting, monitoring, reference implementation hosting, test suites downloading), and acting as hub between test subscribers.