Figure 1: Components in an OCC Model
Keywords: Application architecture, Browser, CORBA, Data representation, Database, Deployment, Design, DOM, E-business, Enterprise applications, Handheld device, HTTP binding, IDL, Integration, Internet, Interoperability, Java, JavaScript, Metadata, Middleware, Mobile, Portal, Schema, SOAP, SQL, Transaction, Validation, W3C XML Schema, Web Services, WSDL, WWW, XHTML, XML, XML-RPC, OCC, Occasionally Connected
Biography
Rich Rollman is a founder and Vice President of Engineering and Product Development at AgileDelta, Inc. , an independent software vendor that supplies XML and Web Services infrastructure software for use on mobile computing platforms.
Prior to joining AgileDelta, Mr. Rollman was founder and CEO of DataConcert, a startup that built E-Business solutions using XML and Web Services to link enterprises with their small and mid-size trading partners. At DataConcert, he was responsible for the design, development, marketing, and sales of DataConcert's E-Business solutions.
Mr. Rollman has been working on XML technologies since their inception. He was the Group Program Manager for Microsoft's core XML technologies (MS XML) and SQL Server's XML functionality (SQLXML). He later led product management at Crossgain Corporation where he built XML-centric programming models for web services.
Mr. Rollman writes the monthly XML Explorer column for SQL Server Magazine. He has 20 years experience building Internet and client-server software on a variety of computing platforms. He holds a Bachelor's degree in Computer Science and a Master's degree in Electrical Engineering.
Biography
John Schneider is a founder and Chief Technology Officer at AgileDelta, Inc., an independent software vendor that supplies XML and Web Services infrastructure software for use on mobile computing platforms.
Prior to founding AgileDelta, Mr. Schneider led Engineering and Program Management for Crossgain Corporation, a high profile Microsoft spin-off founded to radically simplify building and deploying web services and applications (sold to BEA). He was also Principal Systems Engineer at MITRE where he kick-started and led a variety of international XML initiatives for the United States and NATO.
Mr. Schneider founded and is leading ECMA's ECMAScript for XML (E4X) group, which is developing the first mainstream XML programming language and has active participation from Microsoft, BEA, IBM, MacroMedia, RIM and others. He helped develop the XML Query Language and the XML Schema Language. He was a U.S. delegate to NATO on technology initiatives related to XML, messaging, data interoperability and integrated business processes.
Mr. Schneider has twenty years experience developing technology with a focus on mobile technologies, data encoding, information management, data interoperability and messaging. He has a Masters degree and Bachelors degree in Computer Science.
We live in an occasionally connected world where the availability, speed, and latency of the network vary greatly by location. Traditional browser-based mobile applications provide remote access to data but become unresponsive or fail to work when the network or server access is unavailable. Occasionally connected computing describes a new model in which applications are made resilient to latent and intermittent network connections using a local data cache and queued user actions.
The success of the occasionally connected computing model will hinge on the ability to access a large variety of data sources and run on the most used mobile devices. Adopting web services as the foundation for data access leverages the web services ecosystem and enables occasionally connected applications to exchange data across platforms and computing environments. Targeting mass market mobile devices (digital cell phones) increases the value of the occasionally connected applications by allowing more users to access more data from more locations.
This paper motivates the need for a lightweight, standards-based web services implementation that runs on mass market mobile devices. It describes the advantages of using web services and the challenges which must be overcome to use web services on mass market devices with limited computing power and network bandwidth. The paper concludes by describing a new approach to web services which drastically reduces the code required to exchange data with remote services, enabling the creation of more compelling applications with sophisticated user interfaces and application logic.
1. An Occasionally Connected World
2. Occasionally Connected Applications
3. Components of the Computing Model
4. Mass Market Mobile Devices
5. Challenges on Mass Market Mobile Devices
6. The Role of Web Services
7. Web Services: A New Approach
8. Conclusions
Mobile computing is being driven by the increasing availability of wireless technologies. 2.5G and 3G cellular networks, widespread deployment of WiFi technologies, and technologies like Bluetooth currently promise to connect you to your data anytime and anywhere. In an ideal world, a mobile computing device could use any or all of these means to stay continuously connected. But that ideal world doesn't yet exist and some question whether it ever will. Today's world requires that mobile applications be resilient to intermittent, slow, high latency network connections.
We live in an occasionally connected world. Our offices and homes are generally connected by inexpensive, high speed networks with high availability. But once we begin to move around the speed of the network decreases dramatically, latency increases, and availability becomes spotty. WiFi networks enable mobility and provide a relatively high speed, low latency connection. But their limited range leaves coverage restricted to a sparse set of hot spots. Cellular networks extend the range of WiFi but with lower bandwidth, higher latency and unpredictable availability. If you are a cell phone user, you've almost certainly experienced dropped calls and dead zones where your phone simply won't work. Connectivity is not universally available. The result: if you are mobile you will only occasionally have a connection to the network and will often be subjected to high latency, slow networks.
Internet and client-server applications and the programming frameworks used to build them assume that a connection to the network always exists. Without a connection they provide little value. For example, browser based applications are almost useless without a network connection because there is no model for offline browsing and manipulation of dynamic content. And legacy client-server frameworks (DCOM, CORBA) and present generations (RMI/IIOP) all have strong assumptions on network connectivity that prevent them from working offline. Even Web Services, the most recent incarnation of client-server computing, lacks a built-in model for building applications where network connectivity is not guaranteed. But Web Services do provide a set of technologies which can be used as the basis for building occasionally connected applications.
Occasionally connected applications are the new generation of compelling, interactive, and responsive applications for mobile computing platforms. Gone are the days of sluggish browser-based applications in which all critical application data, business logic and user interface logic resided on the server. Today's mobile applications execute locally on the mobile device and provide a consistent, responsive user experience regardless of whether they are connected or not. They work in conjunction with the application server and network but don't depend on them for user interface, business logic or application data storage.
Criteria can be established for differentiating occasionally connected applications from client-server or browser-based applications. The criteria are:
Applications which meet these criteria are occasionally connected applications. Let's look at a couple of examples that will illustrate how occasionally connected applications work.
The first example of an occasionally connected application is one that is common to everyone: an e-mail client. We all take the operation of our e-mail client for granted. But it truly is an occasionally connected application. At the office, the e-mail client periodically retrieves e-mail from the e-mail server. A copy of the e-mail is stored locally in the client cache. Then, when the user is away from a network connection, e-mail remains available and can be read from the cache. Similarly, the user can also respond to messages or compose new ones. When the user presses send and no network connection is available, the 'send' action (outgoing e-mail) is queued on the client. When a network connection is reestablished, the e-mail will be sent to the mail server for delivery. Correspondingly, the mail server receives e-mail on the client's behalf and queues it so it can be retrieved when the client reconnects to the network. Because e-mail clients are occasionally connected, users can continue using them when the network or server is unavailable. In addition, the responsiveness of the application does not depend on the responsiveness of the network or the server.
The benefits of occasionally connected applications extend beyond e-mail to a wide variety of applications. Another example of an occasionally connected application is a personal air travel manager. With a network connection present the application downloads your travel itinerary including flight numbers, times, and destinations and caches them locally. As you itinerary is updated, the application receives notifications and updates your local cache. On your day of departure, the application retrieves and caches real-time departure time, departure gate assignments, arrival gates and baggage claim locations. These functions occur in the background without requiring the user to make a gesture in the user interface. When disconnected, the application displays the cached data and an indicator that shows when the data was last updated. The application can then notify you when boarding times approach. Like the e-mail application, the personal air travel manager application meets the criteria established above for occasionally connected applications.
There are many more examples of applications which provide a natural fit with the occasionally connected model. These include applications for managing sales and inventory, order entry, surveys, auctions, stock portfolios, etc. With this understanding of the characteristics and behaviors of occasionally connected applications, let's now look at the components used to build a model for occasionally connected applications around web services.
Occasionally connected applications are differentiated from client-server applications using the four criteria specified above. Each of these criteria specifies a unique function that occasionally connected applications perform differently from other applications. These functions can be encapsulated in a set of components that, when combined, form a computing model for building occasionally connected applications.
The block diagram in Figure 1 shows a one possible computing model for building occasionally connected applications. The application is partitioned into user interface components, a client-side data cache, and communication components for exchanging data with outside services. These different components are all managed by application logic that orchestrates the components to perform the application's tasks.
The user interface components are responsible for providing a compelling, interactive user interface for the application. The components draw to the screen, accept and process user input, and invoke the application's logic in response to a user action. Application logic may access and manipulate cached data or send and receive messages via web services.
The communications components and queue management work together to exchange data with outside services. The communications components send and receive messages between the application and web services on the network. Queue management is responsible for ensuring consistent behavior with and without a network connection. When working offline, requests are queued until a network connection is available. Queue management is also responsible for passing the return messages or new requests to the application logic for processing.
The cache component provides a client-side data cache which is managed by the application logic. Some applications will use mapping and correlation functions to store data locally on the client. Mapping functions are responsible for marshalling between the local storage format and the message formats used by web services. Correlation enables asynchronous communication and ensures that the application logic can track the status of queued requests when no network connection is available.
This computing model provides the functionality required to implement occasionally connected applications which provide access to data any time and anywhere. What remains is to ensure that an implementation of the model, and an application built with it, will run on the maximum number of devices.
The value of data is directly proportional to the number of users who can access that data. It follows that to maximize the number of users who can access the data, it needs to be available on the mobile devices carried by the greatest numbers of users. These mobile devices include portable personal computers, PDAs, smart phones, and mass market mobile devices (the digital cell phones that most consumers and business people carry everywhere they go).
As the name implies, mass market mobile devices are the largest category of device in use today. This is illustrated in Figure 2 which shows the actual and projected sales of personal computers, mobile handsets, Java enabled mobile handsets and PDA-Size devices for the years 2000 through 2006. In 2003, only 12.1 million PDA-sized devices were sold, including all PDA-phones and smart phones. In contrast, over 150 million Java enabled devices were sold, outselling all personal computers combined (with 135 million sales).
Mass market mobile devices represent the largest category of mobile devices. Deployment of these devices is expected to outstrip all others for the foreseeable future. To maximize the availability of your data, it must be available to users running mass market mobile devices.
Mass market mobile devices have specific characteristics which differentiate them from smart phones, PDAs, and personal computers. Mass market mobile devices are typically smaller, lighter, and simpler to use. They have longer battery life, but have less processing power, memory and screen real estate. This means applications for mass market mobile devices must be highly optimized and more efficient than their desktop and PDA counterparts. This presents some challenges for building occasionally connected applications.
Application size is the first and largest challenge. The maximum size of an application on most mass market mobile devices is limited to 64 KBytes or smaller. A compelling user interface, application logic, and communication components, including the web services stack, all need to fit inside this small amount of space.
Another challenge is the speed of the network. Mobile networks, especially GSM and CDMA networks, tend to be very slow relative to LAN speeds. Bulky protocols and data formats can result in sluggish application response times and a poor user experience. The W3C has solicited proposals for an Efficient XML encoding that would drastically reduce the size of XML messages. A more efficient XML encoding would enable higher throughput on existing networks.
A related challenge is network latency. The time required to establish a network connection on a mobile device can result in long wait times especially when applications use synchronous messages. Asynchronous messaging can be employed so applications don't block waiting for a request for data to complete.
In the desktop computing world, the application size and network speed challenges discussed above would be easily solved over time by Moore's law. Periodic increases in memory, processor speed and network bandwidth would enable applications to be used on ever more capable devices. But Moore's law does not apply to battery powered devices. Increases in main memory, processor speed, and network speed often translate into reduced battery life or larger, heavier, more expensive devices which lack mass market appeal. Over time the natural evolution of the mobile computing platform will provide more powerful and capable devices. But the pace of the evolution will be much slower than on the desktop. In addition, the ever growing list of compelling mass market features, such as video cameras, MP3 players, Bluetooth and WiFi access will continue to compete for small improvements in available battery life and device manufacturing budgets, This makes it clear that simply waiting for the next technology cycle won't suffice for the mobile platform.
To make more data available to more people, in more locations, and on more devices, occasionally connected applications will need to communicate with many different sources of data. Each data source could potentially reside on a different computing platform and execute on a different computing environment, e.g. J2EE or Microsoft .NET. If custom integration was required with each of these different environments, the value of mobile applications would be greatly reduced due to the complexity and time required to build the applications. Thankfully, the XML and web service technologies were built specifically to solve this problem.
XML, SOAP and WSDL provide the framework for cross-platform data exchange. XML provides a standard data format that can be transmitted to virtually any device regardless of the network protocols and transports in use. SOAP provides a standard message format for exchanging XML data with remote services. And WSDL provides a mechanism that describes the messages sent and received from a service. Used together, these technologies provide a good solution for data exchange. But the benefits don't stop there.
Web services are experiencing wide adoption. This creates a competitive environment or ecosystem in which high quality, low cost tools are available for exposing data through web services. As the tools make building web services easier, ever increasing types and amounts of data are being made available. This increases the value of the data by making it accessible to any device capable of using web services.
XML and web services are a logical choice for the communications mechanism used in occasionally connected applications on mass market mobile devices. But the size and complexity of these technologies make it difficult to build compelling applications. Recall that most mass market mobile devices have very low limits on the size of an application, usually 64 Kbytes or smaller. A typical XML and web services implementation on a desktop system requires over 20 times that amount of memory and that's before the user interface and application logic are written. To use web services in mobile applications, the size problem must be solved.
Past attempts to build smaller, watered down mobile web service implementations have been problematic. They typically remove features required by a large percentage of web services, making a large amount of data inaccessible without custom programming. They also push complex programming problems up to the developer, making mobile application development more difficult, time consuming and error prone. And although smaller, these implementations still require a disproportionate amount of the mobile device's resources leaving little to no room for application features.
For example, Figure 3 below illustrates the amount of memory required for a typical application using a popular mobile web services toolkit called kSOAP. Note that kSOAP's XML and web services implementation leaves just enough room for a very small application.
Web services are the ideal communications mechanism for occasionally connected applications. Widespread adoption results in more data being available. The use of standards ensures that the development of occasionally connected applications can dovetail with other corporate efforts to expose data through web services. But in order for web services to be viable on mass market mobile devices, the size problem must be addressed.
Reducing the web services stack to fit on mass market mobile devices is not a novel idea. A few mobile web service toolkits, such as kSOAP (mentioned previously) reduce the size of the stack by removing features required for accessing many web services and exposing more complexity to the programmer. And although these efforts result in a smaller web services stack, they still require a disproportionate amount of the mobile devices resources leaving little room for application, business and user interface logic which make for a compelling application. In order to build compelling applications and benefit from the web services ecosystem a new solution is required.
AgileDelta has developed a new approach for implementing web services on mobile devices called the Mobile Information Client (or MIC). MIC uses compiler technology and compile-time optimization to reduce the web services stack to a manageable size. In many cases, the XML parser, web services stack and a binding layer for simpler programming require only 8 Kbytes as illustrated in Figure 4. This leaves ample room for a full-featured occasionally connected application with a compelling user interface, business logic, and offline data.
MIC contains a number of key innovations which provide some unique benefits to mobile applications. An intelligent compiler framework is used to produce highly optimized code for constructing and exchanging messages with remote services. Although highly optimized, the code preserves the extensibility features of XML and web services. This code utilizes a very small and efficient XML parser, object model, and SOAP stack which saves valuable memory, reduces the number of instructions required to execute a given operation, and conserves battery power.
A small binding layer is also provided to make programming easier and hide the complexity of XML and web services. The developer is no longer required to build XML messages, understand the gory details of a WSDL service description, and construct SOAP messages. Instead of writing tedious, difficult to maintain and error prone code, the developer is free to focus on the user interface and business logic portions of the application. The binding layer is composed of a service object whose methods correspond to the operations specified in the WSDL description of a service and a set of data objects used as accessors for the XML data. The service object is used to invoke the web service. The data objects used to read and write the data exchanged with the web service and are passed as parameters to the methods of the service object..
The binding layer also reduces development time and cost. It simplifies and accelerates application development by automating complex, time consuming development tasks. Developers are provided with an extremely simple programming model that eliminates the requirement to master complex protocols, programming models and software libraries.
MIC also reduces deployment costs. By shrinking the size of the XML and web services, MIC enables PDA-sized mobile applications to be deployed to mass market devices. This drastically reduces or eliminates recurring hardware costs and allows enterprises to consolidate mobile applications on a single platform.
MIC is completely standards based and does not require proprietary programming languages, platforms or protocols. The intelligent compiler framework generates native byte codes that run on hundreds of millions of devices straight from the factory. It implements standard web service protocols that communicate with off-the-shelf servers from leading providers, such as Apache, BEA, IBM, Microsoft and JBoss.
MIC also supports standard APIs for communicating between applications and web services, such as the J2ME Web Services Specification (JSR 172). The compiler framework generates a JSR 172 compliant binding layer, allowing developers to use a standards-based API for invoking web services. This provides investment protection in the form of portability and further increases the efficiency of the implementation on devices with JSR 172 support.
To summarize, by drastically reducing the size of the XML and web services stack, MIC enables a whole new class of occasionally connected applications to be built. The resulting applications have compelling user interfaces, more features, run on more devices, exchange data with more back-end servers and run faster and require far less storage, bandwidth and battery life.
Traditional mobile applications allow access to data from remote locations but become unresponsive or fail to work when the network or server are slow or unavailable. Since a constant network connection is not possible, or probable, a new occasionally connected computing model is needed that caches data locally and continues to operate when a network connection is not available.
Occasionally connected applications need to insulate users from the effects of slower, intermittently available mobile networks. Users should see little appreciable difference in the application's behavior whether it is connected to the network or not. Applications need to cache data locally and use asynchronous messages to exchange data with services on the network.
To be successful occasionally connected applications must communicate with as many data sources as possible, regardless of the platform or computing environment in which they are hosted. XML and web services provide a standards-based communications infrastructure with broad industry support. The availability of development tools and runtime support for XML and web services across a variety of different platforms and computing environments virtually guarantees that most data will (eventually) be accessible through web services. By adopting web services to exchange data, occasionally connected applications are able to exchange data with a large and growing number of data sources.
The value of occasionally connected applications is exponentially proportional to the number of devices they work on. The ability to run on more devices means more users can benefit from the application. To minimize the cost of custom hardware and maximize the number of potential users, occasionally connected applications need to run on mass market mobile devices.
Building compelling occasionally connected applications on mass market mobile devices turns out to be challenging. Due to limited memory, computing power, storage, and strict limits on application size, mass market mobile devices cannot run typical web services implementations. The challenge is to reduce the size of the XML and web services stack without increasing the complexity of the development task in order to provide additional room for expanded user interface and application logic.
AgileDelta's Mobile Information Client meets this challenge head on. MIC provides an intelligent compiler framework, a lightweight XML parser and object model, and a lightweight SOAP stack which shrinks the web services implementation by orders of magnitude. By reducing the size of the communication stack, MIC gives developers the space they need to build more sophisticated user interfaces and application logic. MIC enables the creation of occasionally connected applications for mass market mobile devices.
XHTML rendition made possible by SchemaSoft's Document Interpreter™ technology.
