Proceedings

Abstract:

Among the popular emerging integration needs in the market today is information aggregation, normalization, and presentation from multiple back-end data sources to front-end applications. Termed Enterprise Information Integration by some vendors in the market, this type of solution relies on a centralized common object model to provide a data access interface to client applications. Applications can used this common interface to request data from one or more data sources in a single query, with the intricate details of resolving the query left to the integration tool. This session will explain the architecture of an enterprise information integration solution in general, highlight some of the vendors and their approaches in this market space, and explain the use of such as solution through a real-world example with a large financial services organization.

This sort of solution has four primary components: the data source systems that own the original data; a central normalization layer; a transformation and formatting capability; and the presentation or front-end application. Formats of data sources are mapped to the central normalization layer’s common object model during the design phase, with the resulting mappings often stored and managed in a metadata repository. Depending on requirements – such as data volume, query frequency, and the type of data involved – the central normalization layer can be implemented as a metadata repository alone (with the data remaining in its origin system and format), or as a data warehouse (with the source data copied and normalized into the central repository).

A recent integration initiative at a large financial services company in New York highlights the leading role XML-based technologies can play in this type of solution. The project requirements included a blend of traditional ETL, metadata management, mapping and transformation, and transactional application access using web services-based interfaces. This company put together an end-to-end solution using a set of best of breed technologies for each of its core requirements, and examined in detail the relative benefits of adopting a metadata-repository-only approach versus a more conventional intermediate-data-warehouse architecture.

The mapping and transformation component of this type of solution has most commonly been delivered using a commercial mapping and transformation engine, either as a standalone component or as part of an enterprise application integration (EAI) product. Many of the leading products on the market offer script-based or programmatic mapping and transformation logic, which customers can use to build the necessary functionality into their solution. With the rise of XML as a preferred intermediate data storage format in the integration market, in more recent releases many tools have begun to support or, in a few cases, rely on XSLT to provide mapping and transformation logic.