Figure 1: High level view of the PSB architecture
Keywords: Application Architecture, Data Interchange, Conversion, E-Government, Integration, Interoperability, Metadata, Middleware, XML
Biography
Sean McGrath is CTO of Propylon. He is an internationally acknowledged authority on XML and related standards. He served as an invited expert to the W3C's Expert Group that defined XML in 1998. More recently he has been instramental in advising the REACH agency acting on behalf of the Irish government define and implement its interoperability and service delivery architectures, using Service Oriented Architectures and XML. He is the author of three books on markup languages published by Prentice Hall.
Biography
Fergal leads Propylon's operations in the USA and Canada from the company's New York office. Fergal has a deep understanding of the implications of Service Oriented Architectures and Service Oriented Business Applications for enterprise architecture and technology strategy in large distributed organizations. His is the co-author with Sean McGrath of several white papers addressing interoperability and enterprise architecture in government.Fergal has over 15 years experience as a technology professional and senior executive in Europe and North America at companies including DigitalThink and Charles Schwab in San Francisco and Gallup in London. Before Propylon, he served as CEO of Antefacto in Dublin. Fergal holds BSc. in IT from Dublin City University and an MBA from University College Dublin.
The Public Services Broker (PSB), currently under construction by the Irish Government agency Reach (http://www.reach.ie), is an integrated set of electronic processes, systems and procedures based on a service-oriented architecture (SOA) approach to e- government infrastructure. In this paper, the SOA architecture of the PSB is outlined and the business, as well as technological benefits it provides, are presented.
1. Biography
2. Introduction
3. Overview of the PSB architecture
3.1 Business level explanation of the architecture
3.2 Technology level explanation of the architecture
3.3 Business benefits of the architecture
3.3.1 Business people understand the PSB architecture
3.3.2 Linear application connectivity costs
3.3.3 Shared services
3.3.4 Business level monitoring
3.3.5 Decentralization and federation via hubs
3.3.6 No mandatory replacement of line-of-business systems
3.3.7 Blending manual and automated business functions
3.3.8 Insulation from front end volatility
3.3.9
Clear road map to progress from e-forms to process-to-process integration
3.3.10 Interrogative and declarative integration patterns
3.3.11 Zero technology creep across agency boundaries
3.3.12 Highly flexible at the business process level
3.3.13 Recombinant Growth
3.4 Technical benefits of the architecture
3.4.1 Transport adapters - messaging and footprint
3.4.2 Flexibility
3.4.3 Synchronous behavior provided on top of asynchronous substrate
3.4.4 Temporal de-coupling
3.4.5 Document centric, not API centric
3.4.6 Autonomic aspects
3.4.7 Horizontal Scalability
3.4.8 Web Services Evolution
Bibliography
Sean McGrath is CTO of Propylon, an XML solutions company headquartered in Dublin. He has been working closely with Reach for two years developing the PSB architecture and its associated interoperability framework. He is an internationally acknowledged authority on XML and related interoperability standards. He served as an invited expert to the W3C's Expert Group that defined XML in 1998. He is the author of three books on markup languages published by Prentice Hall and writes the weekly 'e-business in the enterprise' newsletter for ITWorld. He runs a technology-oriented weblog at http://seanmcgrath.blogspot.com.
Fergal Murray is VP - North America for Propylon, an XML solutions company. He is the co-author, with Sean McGrath of several white papers on Service Oriented Architecture and e-Government. He has deep experience in the application of XML processing technologies to complex document management and workflow challenges, particularly in legislative and regulatory environments. He holds a BSc. in Information Technology and an MBA.
Reach is an agency established by the Irish Government to develop a strategy for the integration of public services and to develop and implement a framework for e-government in Ireland. The Public Services Broker (PSB), currently under construction by Reach, is an integrated set of electronic processes, systems and procedures based on a service-oriented architecture (SOA) approach to e-government infrastructure.
In this paper, the SOA architecture of the PSB is outlined and the benefits - both business and technical - are presented. The key role of business document messaging in the PSB will be detailed along with the interoperability framework upon which it is built.
The interoperability framework of the PSB will be presented and the natural separation of human facing and machine facing technology it provides is discussed. This separation provides significant flexibility, facilitating seamless blending of manual and automated business processes, It also provides a clear road map beyond the electronic forms paradigm into advanced process-to-process integrations via the PSB.
The PSB architecture is based on a decentralized, asynchronous XML messaging hub. The benefits of this approach for local, devolved, national and inter-national eGovernment are presented.
A high level view of the PSB architecture is shown in Figure 1.
The key abstraction in the PSB architecture is the concept of a "service".
The PSB architecture splits the concept of a "customer facing service" into three distinct component pieces:
In the PSB architecture, a web portal is simply a service like any other service. The primary purpose of a portal is to interact with customers in order to assemble business level customer service request messages. These messages are then sent to a fulfillment service (typically hosted in a back end government agency) via the integration part of the PSB.
Before turning to the business benefits of this architecture, a more technical overview of the PSB architecture is provided in section 2.2. Non-IT readers can safely skip over the next section if desired.
The key abstraction in the PSB architecture is the concept of a "service". A "service" is any mechanism - human or mechanical - that achieves some business function through the sending and receiving of XML messages via reliable "exactly once" messaging.
Services can perform their business function solely in terms of XML message transformations or (more usually) with the aid of side-effects. e.g. writing data to a database, initiating a customer contact etc.
Services communicate with other services solely by exchanging XML messages via an asynchronous reliable messaging hub. There is no shared state information between services, which are temporally as well as physically decoupled. For each discrete message type, there is a discrete XML schema (in a variety of notations including W3C XML Schema and Relax NG). These schemas are registered in a central catalog of schemas and message types known as the SDEC - Services and Data Exchange Catalog.
Examples of XML message types include:
Messages are sent/received via persisted message queues that are managed on the PSB on behalf of services. Each service has a pair of persisted message queues associated with it - an inward bound message queue and an outward bound message queue as shown in Figure 1.
All messages are encapsulated in the same business level envelope (itself having an XML schema). The envelope contains routing, monitoring, auditing and security information to allow each message to be inspected and understood both at a technology and business level.
All message exchange is asynchronous, that is to say, there is no architecture-level guarantee that a message will be responded to within any given time frame. Conceptually, all services are stateless, long running, event-driven business processes.
We now turn to the business level benefits of this architecture and return to some more technological benefits later.
When business specialists speak in terms of application X "talking" to application Y, their conceptual model of "talking" revolves around data flowing between discrete data processing units. IT specialists on the other hand, are likely to interpret the word "talking" in terms of "function call" or "method invocation", concepts which do not feature in the business specialist's lexicon.
The mismatch between these two conceptual models of data processing has resulted in business specialists having trouble staying on "the same page" as technologists when discussing eGovernment integration strategies. In particular, the technical language associated with Object Oriented integration approaches (e.g. the CORBA, DCOM, J2EE worlds) has proven to be problematic as a vehicle for shared understanding between business specialists and IT specialists.
Reach's experience indicates that the conceptual model of the PSB is easily understood by both business and technical specialists. It creates a powerful, shared language that business specialists can use to express their requirements to technology specialists. Equally importantly, the conceptual model of the PSB maps directly onto an implementation model in terms of services and message exchange that technologists understand.
Services (and thus the applications which sit behind the service boundary) do not interact directly to each other on the PSB. They always connect via a messaging hub. This arrangement means that with a single connection to the hub, a service can exchange messages with any other service reachable via the hub.
This contrasts with the exponential growth in costs associated with establishing "bi-lateral" connections between services in non-hub architectures. As the number of services grows, the number of bi-lateral exchange interfaces that need to be created grows exponentially (Travis).
Services in the PSB architecture are not tied to any existing organizational boundary and are not tied to any particular physical location. Service ownership can move around from government agency to government agency, new services can be created as shared services, all within the same simple services abstraction.
The services abstraction purposely does not mirror "bricks and mortar" organizational structures in order to encourage the sharing of business processes and business data across organizational boundaries which traditionally have been barriers to successful enterprise-wide integration of business processes.
A useful conceptualization of the PSB architecture is in terms of a logistics network where raw materials (messages) are shipped around between factories (services).
As with any logistics network, there are well defined points at which it is possible to attach monitoring "probes" to find out what is going on at a business level.
A number of critical business metrics "drop out" of the logistics approach taken in the PSB. Obvious examples of useful business metrics include sizes of services queues, nature of messages being exchanged between services A and B, current location of message M, average message processing time through service C and so on.
In systems designed to more IT oriented models such as Object Oriented Design, business level monitoring and reporting can be an expensive and time consuming process. This is because it requires reverse engineering the business specialist's logistics centric view of the system from its object centric implementation.
The PSB architecture naturally encompasses the idea of services organized into a hub which connect to other hubs as shown in Figure 2.
This is a natural extension of the logistics infrastructure conceptualization of the PSB and the common phenomenon of transport "hubs" - particularly in aviation - resonates strongly. Purposely, there is nothing in the architecture that prescribes where hub boundaries should be placed. As with services, there is no implied mapping of hubs onto existing organizational structures.
In Figure 2, a number of distinct portals are shown. This may or may not be desirable depending on the organizational details of a particular government or agency. The architecture supports the creation of a single portal - a single customer point of contact - connected to a single messaging hub. An equally valid configuration would be one with a single portal but multiple messaging hubs corresponding to sectoral boundaries such as health, social welfare, defense and so on. These sectoral hubs could have their own portals if desired. In countries with devolved administrations, the configuration chosen would most likely reflect the devolved structure i.e. a federal hub with multiple subsidiary hubs, each of which might have autonomous sectoral hubs covering health, social welfare etc. Each of these might in turn can have their own portals if desired.
Service boundaries crisply delineate what an existing line of business application must do in order to (a) act as a PSB service unto itself and (b) avail of other services on the PSB.
A wide range of integration possibilities exist ranging from manual through to full automation. In the PSB architecture, these integration possibilities are referred to collectively as "transport adapters" and form a critical part of the interoperability framework of the PSB.
Any changes required to increase the level of automation across the service boundary can be phased in over time with a starting point of "no changes" being viable in most cases. The minimum amount of integration required to act as a service on the PSB is the ability for a user to use a Web Browser to send and receive messages. This phased approach gives the business specialists a clear road map and the ability to do cost-benefit analysis of back end service integration expenditure.
Each service on the PSB processes messages sent to it, one by one, in the order they have been queued and to a tempo that suits the service. i.e. there is nothing imposed by the architecture that requires messages to be dealt with within any specific time frame.
As a consequence of this, business functions that are manual in nature can be blended seamlessly with business functions that are fully automated. From a business process engineering perspective, analysts can decompose business processes into their component services without regard to which are automated and which are manual.
This gives business owners a clear mechanism for establishing an automation road map and allows them to grow the degree of automation in their business processes on an incremental basis.
History has shown that the most volatile piece of any e-business system is the piece closest to the user. To take a simple example, whilst the basics of double entry bookkeeping have not changed since the time of Charles Dickens, many accounting systems have been re-engineered from scratch multiple times over the last thirty years because of dramatic changes in user-facing technology. Green screens gave way to client/server, which gave way to graphical user interfaces, which gave way to web browsers, which are now giving way to web service clients and mobile user interfaces.
By cleanly isolating the human facing component from the service fulfillment component of a business process, the PSB architecture insulates the core business processes of eGovernment from the inevitable volatilities of front end technology.
In simple terms, the front end is responsible for creating fully formed service request messages and then routing them to the back end fulfillment service. The fulfillment service acts purely on the message it is sent - not on any aspect of the user interface.
If (when) the front end technology changes, the back end does not need to change as long as the structure, content and orchestration of the business level messages do not change. This approach also makes it straightforward to deploy multiple front ends, all of which generate the same message notations to send to back end fulfillment services. For example, a front end might be created solely for voice browsers or solely for two-way SMS use. As long as the message notations created are correct, the back end business process does not know or care what "channel" the message came from.
In the PSB architecture, fulfillment services do not know and do not care what type of front end technology is used to create the XML messages upon which they act. A consequence of this is that messages need not have originated from human-facing technology.
For example. Consider a scenario in which a Child Benefit payment is activated by means of receipt of a paper form, filled in by the customer. An obvious first step in an eGovernment system would be to provide an electronic equivalent of the form (an e-form) for the customer.
In the PSB context, the Services and Data Exchange Catalog would contain an electronic message format corresponding to this "application for child benefit". Services that are human facing such as the portal would provide an interactive mechanism for "filling in" this message format and sending it to the fulfillment service. However, there would be nothing to stop other business processes from creating and sending the same message format to the fulfillment service without requiring human intervention.
This example can be extended to illustrate another aspect of process-to-process integration on the PSB. Once a notification of birth message is available, it may make sense to automatically trigger a child benefit payment. In this scenario, the application form for Child Benefit is completely removed from the equation and a direct process-to-process integration achieved. This is a direct consequence of the clean separation of front end and back end concerns on the PSB.
As another example, consider a shared phone based "help-desk" for handling customer interactions. Such a bureau (publicly or privately run) could be working with a set of custom front end applications which generate Child Benefit application messages and submit them via the PSB for fulfillment. Again, the fulfillment service would not know or care whether the request came direct from a customer or from a custom application used by a help desk agent.
As a final example, consider the commercially developed applications used in vertical industries such as doctors, accountants, solicitors and so on. By making the PSB message schemas from the SDEC available to developers of such applications, these vertical industries can provide their own human-facing part and allow agencies to concentrate on the back end service fulfillment part.
There are two ways in which services can talk to each other on the PSB from a business perspective. Firstly, a service can send a message containing data to another service. Secondly, a service can send a message containing a question to another service. In the PSB, the former is referred to as declarative messaging and the latter, interrogative messaging.
The PSB architecture does not distinguish declarative and interrogative messaging. Each service can decide itself what form suits it best. From a business perspective, there are advantages to interrogative messaging that make it worthwhile analyzing declarative message exchanges to see if interrogative messaging might be more suitable.
Consider a scenario in which agency A sends paper based information about people to Agency B on a regular basis. It may be that agency B needs all the data A provides on paper. However, it may also be the case that B really only needs to answer a single question about the data owned by A.
To make this concrete, imagine that A holds information about people and B needs to know if particular people are entitled to a free telephone allowance. In the first stages of automation, it may make sense to simply automate the message exchange between A and B. However, by analysing the use case, it appears that B really just needs to be able to ask A a question - "Is customer X entitled to a free telephone" and get an appropriate answer.
The business advantages of the interrogative approach include reduced data duplication, and the ability to factor out useful pieces of business logic. For example "test for eligibility for free telephone allowance" can become a standalone service that can be used by many other services who would otherwise end up duplicating the business logic required.
The only technological requirement placed on services by the PSB is the ability to send/receive XML messages. Agencies can use whatever technology they wish to talk to the PSB via the transport adapters provided. Moreover, if agency A decides to use technology X, there is no requirement for any other agency to have technology X in order to interact with its services. This is one of the key interoperability principles of the PSB architecture.
Moreover, because message exchange is fundamentally asynchronous, there is no need for agencies to have equally powerful technology in order to host services that exchange messages. A large agency for example, might use a mainframe computer to send messages to another agency using simple PCs. The persistent message queues provided by the PSB, combined with the asynchronous message exchange, allow fast machines to talk to slow ones without any difficulties.
In eGovernment, the message storage "buffer-zone" provided by the PSB architecture is particularly important for services that have "flash-flood" characteristics such as tax returns, annual license applications and so on.
The combination of transport adapters and asynchronous, persisted message queuing together minimize the amount of technology "creep" that occurs when an agency interacts with the PSB.
Network architectures which encourage edge based innovation are more flexible than those that embed complex functionality into a central core (Lessig). The PSB architecture takes the so-called "end-to-end principle" (Saltzer) referred to by Lessig very much to heart. The PSB is "hollow" by design. There is nothing in the center of the architecture other than simple routing and comprehensive security and logging. All other functionality - business process orchestration engines, publish/subscribe capability, message prioritization etc. - is housed as services on the PSB.
This approach combines flexibility with high availability and high solidity. There is so little going on in the center of the architecture that little can go wrong. Given robust mechanisms for starting/stopping/replacing services, wholesale modifications to the functionality of the PSB can be achieved without any down-time of the PSB itself.
By treating process orchestration as itself an example of a service, the PSB architecture encourages the edge based re-combination of existing services into new business processes over time. The economist Hal Varian (Varian) coined the term "recombinant growth" to refer to the reassembling of existing technologies into novel products. The concept is powerful from a business perspective as it amounts to the ability to create new business processes quickly and easily out of existing service components.
The potential value of such a recombinant capability is best illustrated by Reeds Law (Reed) which states that the value of a network of size n is of the order of 2 to the power of n, if the n component pieces are free to form sub-groups with the other n-1 members.
This suggests that with a well chosen set of re-usable services on a PSB-like architecture, the value to be extracted from them over time grows dramatically as the number of recombinant services available grows.
In the Irish context, we have already seen an early example of recombinant growth thanks to the PSB architecture. The availability of birth event messages made it possible to combine the Birth Registration process with the Child Benefit Payment process in a new way that benefited the customer and the service provider alike. Namely, it became possible to automatically pay Child Benefit on notification of a birth, thus removing the need for 30,000 annual applications for child benefit payments.
This sort of "spontaneous integration" (Udell) capability - driven by business people, and executed quickly and at low cost by technologists - is a key goal of the PSB architecture.
The PSB utilizes the concept of a "transport adapter layer" in each messaging hub to insulate services from the technology used by other services. Consider two services A and B communicating on the PSB. Service A may decide to use Microsoft Biztalk to communicate with the hub. As far as service A is concerned, the PSB appears as just another Biztalk box. Service B may elect to use IBM MQSeries to talk to the PSB. As far as service B is concerned, the PSB appears to as just another MQSeries box.
The PSB provides a variety of transport adapters that give services the illusion that all other services use the same messaging technology they do. Moreover, the entry level messaging technology is simply a web browser interface to PSB hosted message queues.
Consequently, the PSB "footprint" in an agency is very small. If the agency has no existing messaging technology they can simply use a browser to manually send/receive messages. If they elect to use automated messaging they can choose from a variety of open source and proprietary technologies, each of which has a PSB-supplied transport adapter.
This provides a high level of technical interoperability. On top of that layer, agreement amongst the business owners over the structure and semantics of data is also required. For the most part, this is a business as opposed to a technology problem.
Systems spend most of their lives in what is known as "maintenance". Maintenance is often a euphemism for responding to changing needs. Change is inevitable, indeed change is the norm - not the exception - in eGovernment systems. Consequently, it is imperative that an eGovernment architecture take the concept of flexibility very seriously indeed.
Systems often tend to mirror the organizational structures that created them in the first place (Conway). It is important that an eGovernment architecture not dictate any particular organizational structure, rather it should give business planners the flexibility to re-arrange and evolve their organizational structures without impediment.
The federated hub architecture of the PSB provides that flexibility through a combination of:
Most software developers prefer to conceptualize distributed systems in terms of remote function calls. i.e. for any given function X, they would like to be able to invoke X, passing it any information it needs and getting an answer back synchronously. i.e. as long as the invocation of X is in progress, the local process stops.
History has shown how using this simple of distributed computing leads to fundamental problems in terms of robustness and reliability (Waldo).
On the PSB, the synchronous abstraction is made available to developers without compromising on robustness or reliability by means of a transport adapter. This adapter gives the developer a synchronous invocation view of any PSB exposed service. Behind the scenes, the synchronous adapter uses a pair of messages - a request message and a response message - to invoke the destination service and retrieve a response.
Although this mechanism makes it possible to treat any service as synchronous, it is not a good idea to do so for reasons explained in the following section.
There is an increasing awareness that loose coupling is a vital aspect of any enterprise application integration strategy. Loose coupling comes in a variety of forms, the most important of which is temporal decoupling. In a distributed system, latency is an ever-present and unavoidable phenomenon. Latency comes from a variety of sources: the network may be busy or unavailable, the business process you wish to talk to may not be online and so on (Deutsch).
Given the pervasive nature of latency, it is necessary to model business processes with the concept of latency built in, rather than to try and design it away. The key to temporal decoupling is to minimize the time dependency between services. This is best achieved using a stateless, event- driven process modeling approach in which requests and replies are separate events.
In the PSB, services expose message exchange contracts - not APIs. All services share a single, homogenous API for sending and receiving messages. All other semantics, all other "interfacing" is achieved within the message payload itself. Messages are autonomous and conversations are stateless i.e. messages contain all the information required for a service to process them. There are no "sessions" inherent in the architecture. This design is strongly influenced by web architecture principles first articulated as REST - REpresentational State Transfer (Fielding).
The PSB houses the vast majority of functionality behind simple service boundaries. The core of the design is purposely extremely simple business level message routing, auditing and security. A very valuable consequence is that the core of the design is very stable. Services can be started, stopped, upgraded and so on without bring the system as a whole down. This is obviously important in eGovernment architectures where carrier grade availability is a requirement.
The stateless nature of the PSB makes load balancing straight forward. Messages can be routed independently via as many parallel routing processes as required. Multiple instances of the same fulfillment services can be deployed invoked via a load balancing proxy service. This makes the architecture cheap to scale - both in terms of hardware costs and software complexity.
The technology stack around the edges of the PSB is necessarily very volatile, especially with Web Services stacks which invariably create interoperability problems. The transport adapter architecture provides an effective defense mechanism against this volatility and the end-to-end lock-in effects that unfortunately can accompany even the most standards compliant of applications.
XHTML rendition made possible by SchemaSoft's Document Interpreter™ technology.
