Biography
As chief technology officer at Actional, Dan Foody leverages his extensive experience in enterprise systems software toward designing robust and manageable service-oriented architectures. He is an active participant in the Web services standards community, including WS-I and OASIS, where he directs Actional's contributions on the OASIS Web Services Distributed Management Committee (WSDM). Dan's experience with distributed systems technologies including middleware, integration and Web services, gives him a broad knowledge of the complexities and requirements for managing real-world enterprise software deployments.
While there is real and measurable ROI for Web services when used in the context of a project, leveraging Web services technology to build a service-oriented architecture will dramatically improve overall ROI. However, several critical pitfalls must be avoided in order to effectively realize the benefits of SOA.
1. Why Web Services and SOA?
2. Adoption Report Card
3. Adoption Challenges
4. Conclusion
One of the ongoing challenges for business today is finding ways to do more with less. Companies are under relentless pressure to deliver products and services to market faster, better and cheaper than ever before. The expectation of information technology (IT) organizations is that the company's IT investments will drive the business forward, not only in terms of gaining efficiencies and increasing responsiveness, but also in creating new top line opportunities.
Ironically, most corporate IT organizations allocate 75-85 percent of the annual budget just to "keep the lights on." At the same time, research shows that the average utilization of a typical server is 15-25 percent. While these figures may not reflect your particular organization, they highlight the fact that there is a great deal of inefficiency in IT today, which consumes large portions of a budget that could be put to better use for more strategic initiatives.
What is driving this inefficiency? There is no single cause, but much of it is due to mergers, acquisitions, or functional silos within the organization - all of which contribute to a bloated IT infrastructure, rife with overlapping and duplicate applications and systems. Often, in order to get things up and running as quickly as possible, IT has opted to live with the redundancy - integrating systems where necessary to keep them consistent. While this approach may result in quick time-to-delivery, it severely impacts recurring costs.
While some IT organizations may be content with this status quo, "aligning IT with business" has become the number one IT concern of executive management. However, with as little as 15 percent of the budget available to service the changing needs of the business, IT just doesn't have room to maneuver. Clearly a fundamental change is required.
It's this realization that is leading many organizations toward a service-oriented architecture (SOA), an approach to architecting your IT infrastructure that eliminates redundancy and accelerates project delivery via consolidation and reuse.
For example, a financial services firm might have 19 different applications, each of which needs to perform bond-yield calculations. In a traditional IT environment, each application would have its own bond-yield calculation engine - leaving the organization to bear the cost of 19 separate bond-yield calculation engines, one for each "silo". In contrast, in an SOA environment, the bond-yield calculation engine would be a shared "service". While you would still have 19 different applications, each would share the same bond-yield calculation service. This change alone lowers the total cost of ownership (TCO) by a minimum of 5x through the elimination of unnecessary hardware, software, and capacity. At the same time, as yield-prediction algorithms improve, you can take advantage of these improvements through a single change to a single "service" - not 19 changes - enabling IT to respond to the business need with either 19 times fewer resources, or 19 times faster. This is the essence of an SOA: it gives you the ability to leverage economies of scale that are impossible in a world of application silos.
Of course, no organization will implement an SOA overnight. Moving to an SOA is an evolutionary process. It often combines elements of proactive forethought (" For the new applications we are building which services should we factor out and turn into first-class reusable services? ") together with reactive, but necessary, afterthought (" We already have redundancy, how do we consolidate services to reduce it? "). Each step along the path toward an SOA should rightfully be subjected to its own return-on-investment and total-cost-of-ownership analysis to make sure that objectives are being met.
Most IT professionals recognize that the idea of a service-oriented approach to applications is not new. Yet, while the ideas behind SOA have been around for many years, the persistent challenge has been how to implement an SOA cost-effectively. To do this, using the above example as an illustration, each of the 19 different applications (regardless their particular development platform) needs to communicate with the bond-yield calculation service. In short, each application needs speak a common language in order to be understood by the service it wishes to use. In the past, such a "common language" for inter-application communication did not exist. Instead, each large platform vendor had a proprietary and therefore incompatible approach. Some companies built their own as well. But once you introduce multiple ways of communicating, or have to support a fully custom-crafted infrastructure, cost and complexity skyrocket and you lose much of the benefit of SOA.
Happily, for the first time in the history of the IT industry, there is virtually universal agreement on a "common language" to allow applications to communicate with each other: Web services (in the form of XML, SOAP, WSDL, and other related standards). Because of this near universal agreement, Web services have begun to break the log-jam that made SOA impractical in the past and, as such, have become the key foundation technology for building an SOA.
So, given the cost and time-to-market advantages of an SOA, is it making inroads in mainstream IT? Or is it just another new idea to discuss around the water cooler? Are developers just dabbling or are there real companies out there reaping results from their SOA investment.
From an industry and standards perspective, SOA and Web services technologies are actually quite advanced. Web services technologies have broad industry support - it's hard to find a vendor today that doesn't have at least some level of support for Web services in their products.
From a standards perspective, the base standards underlying web services - XML, SOAP, and WSDL - are relatively stable and mature, having been standardized and in use since 1998, 2000, and 2001 respectively. Industry analysts such as Gartner Group indicate that these standards are ready for prime time. According to Gartner, these standards have reached the "plateau of productivity," a term they apply to technologies whose real-world benefits are demonstrated and accepted.
Beyond the base standards, there are many supporting standards in various states of progress as well. For example, the key Web service security standard, WS-Security, has recently been adopted. Many vendors are already shipping implementations of early drafts of the standard or of the final adopted version. While not as far along as security, standards efforts in the area of reliable messaging are progressing both via technical committees such as the OASIS Web Services Reliable Messaging technical committee. It is expected that adopted standards will emerge this year, and many vendors are already working on interoperability based on draft versions of the standards.
Even more importantly than the standards themselves, the industry has achieved an important first with the formation of the WS-I - the Web Services Interoperability Organization - an organization driven by vendors to making sure that the "common language" of Web services (and all the supporting infrastructure around it) is interoperable among implementations. Historically, while vendors have usually talked up the benefits of standards-based interoperability, the reality was often quite different. This, however, is the first time that vendors have put their money where there mouth is to make sure interoperability is real, not just a checklist marketing item. The WS-I has already published a "base profile" which describes how to build and use the base web services standards to ensure interoperability. In addition, a draft version of the "security profile" has also been published. The industry is clearly taking interoperability seriously this time around.
While there is great progress in industry and standards adoption, it's also important to look at Web services and SOA from a corporate perspective. How is corporate adoption progressing?
A recent IDC study (data gathered in Q3 2003) has shown that, of organizations with more than 1000 employees, 96 percent are actively pursuing Web services technologies. Of these organizations, 50 percent already have at least one Web service project in production, while the remaining 50 percent are either evaluating or running pilots with the technologies. Of the organizations with web services in production, 81 percent have more than one web service project in production. Anecdotally, we've found many organizations where managers are unaware that they already have Web services in production - so with this in mind, and the fact this data was gathered a year ago - it's clear these numbers underestimate the current state of Web service adoption.
As companies like these begin to deploy Web services projects and SOA technologies, they must be able to measure the benefits derived and benchmark their progress. As indicated in the IDC study, many early Web services projects are integration initiatives. Many initiatives end up generating over 150% ROI in the first year for point integration projects versus traditional approaches. Using this metric as a general estimate, it's evident that every project deployed with Web services versus traditional approaches provides tangible value to your enterprise. So, a quick measure of the relative benefit you are deriving is the number of Web service projects you have in-production. XML firewalls and Web services management products further improve ROI as they keep IT from getting bogged down in infrastructure issues such as security, routing, versioning, provisioning, and transformation of Web services.
Beyond the tangible ROI benefit of using Web services and related products, you also need to consider TCO (total cost of ownership), which factors in the costs of any investment over its lifetime. While the use of Web services instead of traditional integration approaches does provide TCO benefits, they are nominal. For example, if 10 Web services are each used as a point solution in 10 different projects but the Web services are never re used, you will gain a benefit on each project - but you will be very far from achieving the full return possible with an SOA.
To dramatically improve the TCO value gained via these technologies, organizations need to begin harnessing the economies of scale of consolidation and reuse - in other words, you need to move from ad hoc use of collections of Web services to a more formalized SOA.
Case in point: If you have just one Web service but, either through consolidation or reuse, it is used by five different applications you have achieved an important SOA economy of scale and have significantly reduced TCO. How is that TCO benefit measured? Every time you can reuse an existing service (whether proactively via planned reuse or reactively via decommissioning of redundant services), this equates to fewer machines to buy and operate, less software to license and operate, fewer individual service lifecycles to manage, etc. - all significant quantifiable benefits. A quick measure of the relative benefit you derive from SOA is the number of times you have reused services in different projects (not the number of services, but the number of different uses of services).
In sum, the industry report card is mixed. Standards are progressing. Organizations are embracing these standards and deploying Web services for projects and initiatives. However, most organizations have yet to fully capture the value available to them through a strategic use and reuse of services through a service-oriented architecture. The next section will highlight some of the challenges to effective SOA and achieving maximum benefit.
The fundamental goal of Web services and SOA is to improve ROI and TCO. However, just because you use Web services - whether you have 10 or 10,000 - doesn't guarantee good results. There are a number of barriers -- many of which are organizational and not strictly technical -- that your organization needs to proactively address in order to be able to achieve measurable benefit.
Challenge #1: Standards Won't Take You All the Way
One key issue is that standards don't guarantee interoperability. By their very nature, standards are designed to support many different uses across many different types of organizations. For example, the security standards allow many different types of credentials (how a user is identified - by username, by digital certificate, etc.). If the sender and receiver don't both understand the same types of credentials then they can't talk together - the common language breaks down. Narrowing down, from all the available options, the set of acceptable ways to use standards is something your organization needs to address via policies and procedures. Of course, many of these policies and procedures will naturally need to change over time as regulatory or other compliance requirements change over time, so when thinking about TCO you need to consider the inevitable cost of change.
Challenge #2: Who Pays for the Service?
With traditional line of business applications, figuring out who should pay for it is quite easy - whether you are talking about the initial implementation cost or ongoing operational and maintenance costs. But, who pays for a service shared by many applications? Ideally, each line of business should pay proportional to their use - the lines of business that use it the most should pay the most. Essentially this is a transfer pricing model. Beyond the financial considerations of how to effectively setup transfer pricing policies and procedures, you also need to consider how you even determine the usage incurred by each line of business!
Challenge #3: How Do You Ensure Service Levels to All "Customers"
To the "customers" of the IT infrastructure - those who use an application but have no idea what's driving it - the application is defined by what they see of it. To end users the order management application is still the order management application whether it is built as a monolith or whether it leverages shared services. In either case it must meet the same expectations of performance, availability, and functionality. Conversely, if the same "customer" uses two different applications, they may have different expectations of each - even if both applications leverage the same set of shared services. While a shared service serves many applications, in many ways it needs to be able to be treated almost as if there is a dedicated instance for each application so that the "customer experience" of using the tangible applications can be assured. For example, shared services need to be able to be replaced or upgraded without impacting the applications they serve. Shared services may need to provide a different level of security for each application or use - especially important when some applications are internal and some are external. In addition, shared services may need to ration available capacity if they are overloaded - allocating capacity proportional to the business value of each application and use.
Challenge #4: Avoiding Unintended Consequences of Change
One of the great advantages of Web services and SOA is they enable IT to quickly respond to the needs of the business. But, because services are shared in an SOA, changes can lead to unintended consequences. For example, if an application using a service has an unexpected load increase, this consumes capacity of the service - most likely reducing the service level for all of the other applications. In another example, a seemingly simple change to security policy (for example, changing encryption key sizes) may have a similar impact on all the other applications. This, of course, can further have a ripple effect in cases where services depend on other services. The use of shared services means that understanding the nature of interdependencies plays a key role in understanding the effects of any changes.
Challenge #5: No One Person Knows All the Moving Parts
As one team begin leveraging services built by other teams, they no longer have visibility into all of the moving parts that make up their overall application. This has a number of implications. For example, how does an application team ensure that their overall application is secure, when it is built using services from other teams? If their application will be exposed to partners and customers via the internet, how do they ensure that the services they use are not vulnerable to malicious attacks aimed at stealing or corrupting information? The services they leverage may not have been built with the same set of requirements in mind. Similarly, when something goes wrong, how can you figure out if the problem is due to the application itself, or due to services leveraged by the application?
The adoption of service-oriented architecture and Web service technologies is well on its way. Gartner has stated the core standards have reached the "plateau of productivity" and IDC has determined that over 96 percent of large organizations are actively pursuing these technologies. Organizations pursuing the benefits of these technologies can achieve both measurable and significant return on investment as well as dramatically lower total cost of ownership.
However, ad-hoc implementations of Web services will only realize a small fraction of the potential benefits of ROI and TCO (and, in some cases, may actually increase costs). In addition, ad-hoc implementations of Web services can expose you to new risks such as new forms of intrusions and attacks, degraded service levels experienced by customers, and longer time-to-resolution of production problems. In order to effectively harness the benefits or ROI and TCO, the outlined challenges need to be addressed head on before beginning the migration to a service-oriented architecture.
XHTML rendition made possible by SchemaSoft's Document Interpreter™ technology.