Figure 1: DeltaXML Sync process flow
Keywords: XML synchronization, XForms, forms, Cocoon, XUL
Biography
Thomas has been developing commercial software since 1986 and is still amazed that he gets paid to do what he’d choose to be doing anyway. He particularly enjoys working in the turbulent waters between geeks and suits, and with DeltaXML he’s finally found a revolutionary, home-grown technology and is helping keep it lightweight and agile. He drinks too much tea.
Biography
Nigel Whitaker is a Software Architect at DeltaXML and leads theresearch and development of the company's Java products. Hestarted his career working with discrete event simulation and EDA,moved into compiler engineering and data format work, and is nowheavily involved with Java, XML and XSLT technologies. Nigel hasPh.D. in Computer Science from the University of Manchester.
Biography
Robin La Fontaine is CEO of DeltaXML, a company providing change management for XML including a method for representing changes to XML documents and data in XML. Robin has contributed to various ISO standards and has been project manager of several European research projects including the XML/EDI European Pilot Project. His background is in CAD data exchange and Lisp programming. Robin has a degree in Engineering Science from the University of Oxford and a Masters degree in Computer Science.
Conflict is a recurring feature of human existence. Whenever a society or a system evolves over time, with people introducing their own agendas according to their own priorities, there will inevitably be different and inconsistent ideas and plans - these are the basis of conflict. Only with stasis and death does conflict end, so we should see conflict as a sign of vitality and growth, not as something to be avoided.
From a data processing perspective, though, conflicts lead to problems. Conflict resolution is required wherever we have multiple concurrent changes to a single information set. In practical terms this applies, for example, to concurrent editing environments, to replicated database instances which are being updated independently, to address-book changes on a PDA that must be merged into a master database that has itself been changed.
Resolving these conflicts very often requires human intervention. This paper looks at the use of XML forms of various types to reduce the drudgery involved and to take advantage some of the greatest strengths of XML, using pipelining and easily-understood representations to allow a decision-maker to work with minimal drag.
We take a look at current implementations, and ask how much of this vision can currently be made a reality.
1. Introduction
2. XML Pipelines
3. XML Pipelines With Lumps - Conflict Resolution
4. Sync Forms To The Rescue
5. A Note On Terminology
6. Practical Challenges
7. Mozilla XUL
8. XForms
8.1 Overview
8.2 XForms Code
8.3 Server-side
8.3.1 Chiba
8.3.2 OXF
8.3.3 IBM XML Forms Package
8.4 Client-side
8.4.1 X-Smiles
8.4.2 Novell/Silverstream
8.4.3 x-port formsPlayer
8.4.4 DENG
8.4.5 Mozilla
8.4.6 The Future of XForms
8.5 Design Lessons
9. Cocoon
9.1 Past
9.2 Present
9.3 Future
10. Reality Check
11. Meta-Reality Check
12. Conclusions
Appendix 1. Checklist
Appendix 2. Further Reading
Acknowledgements
Bibliography
As part of our work with XML change control software, we wanted a way to simplify manual intervention in the XML pipelines we use to configure and extend our core technologies, without losing the benefits of an “XML everywhere” approach. Specifically, we wanted to be able to deliver demos and custom implementations of our DeltaXML Sync XML synchronization product as transparently as possible, preferably without requiring software installation by the client. DeltaXML Sync is a generic XML synchronization package for merging concurrent changes to XML documents. We knew that HTML forms could be made to deliver the functionality we were after, but that this could be both painful to develop and difficult to maintain. Our earlier experiences when using HTML forms for the online DeltaXML Compare Live service, which provides basic XML change identification, led us to look for a simpler approach, preferably one with a greater focus on XML.
It seemed that a useful solution might be found in one of the XML forms technologies, which led us to evaluate the options available and then assess which to use for our sync conflict resolution.
One of the emerging trends in XML adoption is the preference for “pipeline” processing models over one-size-fits-all monolithic approaches. Such “XIXO” (XML In, XML Out) pipelining follows engineering best practices developed over the past couple of centuries, which have allowed ever more sophisticated constructions to be fabricated from pre-built components. Relatively little current software development involves starting with a blank document and coding an entire application from scratch – by analogy, we are moving from the “artisan” economy of the late 18th century to the industrial economy of a century later, with automated and repeatable processes where once was craftsmanship.
The case for XML pipelines, and the current state of their implementation, are beyond the scope of this paper - for more, see [JAXPPipelines]. Instead, we will be looking at the areas where a pipeline model proves hardest to implement, particularly focusing on the interface between the computer and the person who’s trying to use it productively.
The intelligence and skill of the craftsman remains as important today as it has ever been. This skill is often in evidence in resolving conflicts – in deciding which of two sets of edits should be accepted for further processing. To handle this, our aesthetically-pleasing pipeline must grow unsightly nodules, out-of-process manual interventions which are often inefficient and difficult to audit. For example, consider edits to a legal document made by two reviewers; where either of them deletes text, or adds it, and the other makes no change, a common rule is to allow the edit to pass. Only where both have edited the same section, or where one has edited it and the other marked it for deletion, do we have a definite conflict.
Crucially, the definition of what constitutes a “section” depends upon context. For the legal and translation teams, a paragraph may be the appropriate level at which to identify and sync changes – typesetting, though, may require sync at the word or even the character level. This is the “sync granularity”, tuning this is critical for optimal results.
DeltaXML Sync is our commercial product which addresses these issues, providing generic synchronization – tree-structured three-way merge – for XML, reconciling changes between a base document and two edited versions. Based on the DeltaXML Core technology [DXCore], and sync research presented at XML Europe 2002 [DXSync], DeltaXML Sync automates the identification of differences and produces as output a “conflicted” document, which contains the original XML with unconflicted edits merged in and with annotations to show where there are conflicting changes.
Using DeltaXML Sync to identify these conflict situations, how can we help automate their resolution? To take a practical example: the current practice in legal teams worldwide involves a sequential review process, with each reviewer taking the document in turn and using word processor “Track Changes” features, or even email quoting hierarchies – horribly inefficient, hard to audit, and work that most people regard as uninspiring, but still this appears to be the best available option when using unstructured documents.
But a legal document is not unstructured – it’s highly ordered and regular, and when the structure is expressed in XML we can apply generic tools to the task of synchronization. Instead of asking each reviewer to edit the document in turn, multiple changes can be made concurrently, with conflicts identified automatically and passed on for resolution.
This is clearly a great advance over synchronizing by hand, eyeballing changes and deciding which to keep. To be really useful, though, the manual intervention needs to be part of a larger workflow, with improved efficiency (increased parallelism) and simplified auditing. To synchronize concurrent edits to our XML legal document, for example, we want to identify changes and represent them in XML, identify and apply any non-conflicted edits and then pass on the conflicted ones for eyeballing and manual selection. Once this resolution step is complete, the automated workflow should resume, perhaps submitting a document to a content management system or committing synchronized database updates.
XML forms technologies appear to fit this paradigm very well, allowing us to integrate user interface logic into more extensive workflows and to build pipelines which include both server-side processing and client-side interaction, all using XML. Well-formedness and validity checks can be applied to user input just as they would be to any other step in the pipeline, simplifying the architecture and potentially producing richer interfaces.
Common to all the XML forms offerings are some appealing features:
How does this plug into our workflow? Typically, a conflicted output document is transformed (perhaps using XSLT) into an XML form. This contains the data, together with markup that indicates where conflicts must be resolved, and code for UI controls to select between available options. Validation constraints may be included in the same document, which can be processed by a “forms processor” of some kind to generate a user interface, perhaps in a web browser.
That’s the theory; we should be able to use forms to deliver the best features of automated sync without requiring complex setup or XML training for end users. Do XML forms implementations currently live up to their promise? To find out, we looked at some of the leading variants, concentrating on those that:
Our initial list included:
We also looked at some proprietary technologies, particularly
Since we are building forms for general use, though, rather than for a closed community we trimmed these “proprietary” players from our list, removing Microsoft InfoPath – which, though hugely impressive, does not deploy over HTTP and requires a Microsoft Windows™-specific client – and XAML, which with the Avalon presentation layer will be key components of Longhorn, Microsoft’s next iteration of their OS. Macromedia Flex was dropped since it requires a proprietary server. Adobe XML Forms looks promising, with client-side interaction through the widely deployed Adobe Acrobat 6 and with a rich UI, but relies on the proprietary FormCalc language for scripting and appears to offer little support for “pipelined” information flows. That left us with XForms, Cocoon Forms and Mozilla XUL.
We also excluded a raft of lighter-weight tools, such as Thinlets and SwiXML, since they are not yet well established and hence have a rather more uncertain future – a crucial concern since we are looking to the longer term. Commercial products from Bambookit, Laszlo and PureEdge are feature-rich and might usefully be included together with the other proprietary systems mentioned above in an evaluation for a larger project.
Finally, the work of the WHATWG [WHATWG] on WebForms, though interesting, has been excluded since it is largely nascent. In time, this approach to extending forms functionality through incremental improvements to browsers may prove very effective.
There is plenty of room for confusion over terms here. Some authors refer to every “XML forms” technology with the catchall term XUL – XML User Interface Language. The Mozilla community, however, clearly regard this as a term referring only to the Mozilla variant – a convention we follow here.
The XForms spec defines two levels of conformance, XForms Basic Profile and full XForms. The WHATWG [WHATWG] WebForms initiative was at one stage called “XForms Basic” and was fortunately renamed to avoid confusion.
Cocoon has its own set of naming peculiarities – a defunct “XML Forms” package has been deprecated and “JXForms” offered in its stead, both are now being replaced by the preferred CForms package – which was previously called “Woody” and is often called simply “Cocoon Forms”. More on this in the Cocoon section below.
Our challenge was to construct a UI for an XML Sync Conflict Resolution package. This would allow
The ability manually to enter a preferred fragment is, according to our informal research, highly valued – but clearly also adds complexity. Ability to configure sync granularity is essential, but need not necessarily be represented in the UI (could be a server-side configuration, for example).
So how did they perform?
Mozilla XUL is one component of the Mozilla application development suite, currently being used to to develop the integrated Mozilla application, the Firefox and Camino browsers, the Thunderbird e-mail client and the Nvu HTML editor. These represent some of the most advanced tools available in their respective categories. The architecture depends upon a core library (XPCOM - cross platform COM) written in C++, with applications developed using XUL to define user interfaces and JavaScript to control application flow. As well as lowering the bar for required development skills, this has proved to be a flexible and remarkably performant design.
Here’s a XUL code snippet from [MozillaXUL]. Note the emphasis on visual appearance, with precise control over the UI controls and use of CSS and JavaScript.
<?xml version="1.0"?> <?xml-stylesheet href="chrome://global/skin"type="text/css"?>
<!DOCTYPE window>
<window title="Hello xFly" xmlns:html="http://www.w3.org/1999/xhtml"
xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul"
style="background-color: white;" width="300" height="215"
onload="centerWindowOnScreen()">
<script type="application/x-javascript" src="chrome://global/content/dialogOverlay.js" />
<vbox align="left"> <label style="font-weight: bold;" value="Hello, Welcome to the xFly">
<image src="http://books.mozdev.org/xfly.gif" />
<button label="hello xFly"
oncommand="alert('Hello World');" />
</vbox>
</window>
|
For our purposes though, there is insufficient emphasis within the XUL framework on managing XML. Although the user interfaces are defined in XML, the separation between model and a view - that is, between the data to be processed and the physical controls on screen to manipulate the data - is less clearly defined than in, say, XForms. The emphasis for Mozilla is on developing rich client applications which build upon a well designed framework with lightweight XML and JavaScript so that new application functionality can be quickly developed and deployed. The functionality that we required was relatively basic, and since choosing this option would currently have limited us to a single browser we did not pursue it.
The first draft specification for W3C XForms appeared in April 2000. The original intent was to find a replacement for the much maligned forms functionality in HTML, since processing of forms is essential in many corporate applications. On its appearance in October 2003 at recommendation stage, XForms has become a rich and relatively complex standard with a surprisingly large number of implementations – at least on paper.
XForms follows a classic Model/View/Controller - MVC - architecture. The Model is an XML document containing the data to be collected or modified, the View is a collection of user interface widgets also defined in XML. The Controller is again defined in XML, both by using XML Events and by specifying constraints upon the model using XPath expressions.
Its original conception, though, was as a client side implementation within a browser. Practical issues with XForms functionality across a distributed environment include issues of validation and user feedback – field-level validation and error messages involve server hops for every input field. Adoption of XForms seems most successful in controlled environments such as intranets where a common client platform can be established and suitable browser plugins deployed consistently. In these environments, XForms is definitely a great improvement over its predecessor.
In order to distribute XForms functionality over HTTP using existing technology, a number of projects are converting an XForms document into a server side model with which the user communicates using conventional HTTP GETs and POSTs, updating the server side model and causing a new page to be sent back to the browser for each update. This is the approach taken up by both the Chiba and OXF processors and while not without problems, is currently perhaps our best hope for distributing realistic XForms functionality to the wider internet.
XForms came in for criticism for a number of reasons. First, it has inter-dependencies with a number of of other W3C standards. XForms depends upon XML (surprise!) including namespaces, XPath (no problem there), and more controversially upon W3C XML Schema for defining datatypes. It is intended to be a part of the specification for XHTML 2.0
Second, it is sometimes accused of bloat. Is it over-featured? That will, unsurprisingly, depend upon how you intend to use it. Certainly for simple user registration forms and the like the technology overhead required to set up an XForms implementation can be pretty daunting. Until server-side implementations are more complete and better integrated into existing frameworks, or client implementations are so widely available that deployment is no longer an issue, XForms can be a sledgehammer to crack a rather small nut. After all, even vanilla HTML forms have been used successfully for complex projects for years.
For our purposes though, we found that we wanted to use the more advanced pure-XML MVC features of XForms, since this would allow us to work in XML without conversion or flattening of our model. In particular, storing the conflicted document directly in the form as an XForms model instance theoretically allows for some elegant solutions. In practice, though, this depends on features which have made it into the spec but are currently not supported by any implementations – see Section 8.5 for details. In the balance between features and complexity, those wanting upgraded HTML forms find XForms too complex, while we found ourselves wanting more. This suggests that the XForms WG made a reasonable job of judging the 80/20 points.
Here's a simple snippet of XForms as a taster; meaningful sync examples are too large for this paper, so we're using a two-field entry form with minimal validation to illustrate the syntax.
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/2002/06/xhtml2"
xmlns:xf="http://www.w3.org/2002/xforms"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:ev="http://www.w3.org/2001/xml-events">
<head>
<title>hello clouds hello sky</title>
<xf:model id="main">
<xf:submission id="capture"
action="http://localhost/snark/rawcapture"
method="post" replace="all"/>
<xf:instance id="namedata" xmlns="">
<data>
<name userid=""/>
</data>
</xf:instance>
<xf:bind id="b-name" nodeset="name" required="true()"/>
<xf:bind id="b-userid" nodeset="name/@userid"
required="true()"
constraint="string-length(.) = 6"/>
</xf:model>
</head>
<body>
<xf:group id="maingroup" appearance="minimal">
<xf:label>A very simple data collection form.</xf:label>
<xf:input id="name-input" bind="b-name">
<xf:label>Name: </xf:label>
<xf:alert id="T7">You have left your name blank</xf:alert>
<xf:hint id="T8">The keyboard is quite useful for entering your name</xf:hint>
</xf:input>
<xf:input id="name-userid" bind="b-userid">
<xf:label>User ID </xf:label>
<xf:alert id="T9">You have entered an invalid user ID</xf:alert>
<xf:hint id="T10">User ID must be 6 characters</xf:hint>
</xf:input>
<xf:group>
<xf:trigger>
<xf:label>Update...</xf:label>
<xf:refresh ev:event="DOMActivate"/>
</xf:trigger>
</xf:group>
<xf:group>
<xf:output>Current name is:</xf:output>
<xf:output bind="b-name"/>
<xf:output>Current user ID is:</xf:output>
<xf:output bind="b-userid"/>
</xf:group>
</xf:group>
</body>
</html>
|
Chiba is an open-source (Artistic License) project that delivers XForms functionality using a Java server-side component (servlet) over HTTP name/value pairs and standard GET/POST technology. It works with the widest range of clients without requiring client plugins.
An XForms document is used to create a server-side DOM (document object model) in memory and to generate an HTML page with standard HTML form controls. When the page is completed and submitted, IDs attached to the controls are used to map the flattened set of name/value pairs back to the DOM, which is updated, possibly triggering validations and other events, and resulting in an updated page being returned to the client.
Here's Chiba displaying the simple XForms document we saw earlier:
Chiba runs as a web application, a typical invocation would be
http://localhost:8080/chiba/XFormsServlet?form=/forms/demo.html. Here Chiba is acting
as an “interpreter” for the XForms document demo.html, mapping controls
such as <xf:submission> and <xf:refresh>, to standard HTML
forms and buttons, so that even a simple XForms document can generate some pretty hairy HTML. The
HTML forms post back to the Chiba XFormsServlet, which parses the submission and generates as a
response a new (usually very similar) page, with updated controls depending on the
submission. Despite the impedance introduced into the process by the HTTP hops, though, Chiba is
well enough designed that once the initial hurdles are overcome it is possible to build
sophisticated forms very quickly.
Chiba’s major weakness is with integration. For example, for our pipeline we are dynamically generating an XForms document based upon the conflicted sync output; since Chiba expects input documents to be on the filesystem, we cannot build SAX pipelines which dynamically process the output of the DeltaXML Sync pipeline, but must first serialize to disk.
OXF - Orbeon XML Forms - takes a similar approach to Chiba, but is part of a larger Orbeon Presentation Server product. As this paper went for publication Orbeon announced that the complete framework is being released under an open source (LGPL) license.
Key strengths of the Orbeon package include excellent user documentation, simple set-up and rich functionality for controlling page flows defined in XML. As with Cocoon, the emphasis is on providing business application development features beyond simple XForms support. Documentation and initial complexity been frequently expressed as problems with Cocoon, it seems likely that OXF will help fill a gap in the market, certainly for simpler applications. Orbeon believe they offer other advantages over Cocoon, there is clearly room for both.
As part of their avowed commitment to XForms (they sponsored a Working Group member), IBM released [IBMXforms] in June 2003, a Java server-side implementation with a client side component as a Microsoft Internet Explorer plugin. This plugin communicates with the XML forms server to help resolve issues with a purely server based approach. In the longer term this division of responsibilities looks a valuable one. As of this writing, however, this project appears to be stagnating, and Microsoft Internet Explorer (MSIE) is the only supported browser.
Until XHTML 2.0, with full XForms support, is widely implemented in standard browsers, an XForms application will continue to require either a specific client side application or a browser plug-in to add XForms capability. For our requirements, delivering XML sync functionality with minimal footprint on client machines, we decided to avoid custom applications – though we list them here for reference.
These implementations come closest to realizing the vision of the XForms designers. The browser opens an XForms document, loads the model and renders UI widgets to allow interaction with that model. Page logic defined as constraints or as event triggers and actions are applied in real time, with immediate field-level validation, popup hints, the interactivity we have come to expect of desktop apps. When the form is submitted, XML is sent over the wire and can be directly processed by the receiving application.
Again, that’s the theory…
From the Helsinki University of Technology, this Java-based XML browser supports XHTML 2.0 (including XForms), SVG and CSS and uses other projects to add JavaScript, XSLT, XML Signature and more. Available under an Apache-style license, it was one of the two initial implementations that allowed XForms to progress from Candidate Recommendation to Final Recommendation. It is a largely complete implementation (features such as deep copy are still unsupported), currently only available as a standalone application.
Also written in Java, the Novell XForms Technology Preview is part of the exteNd application development suite, and is purely an XForms processor. That is, it does not support rendering of either XHTML or SVG - which is something of a problem since the XForms spec makes it explicit that an XForms document may be embedded within a container document. Nonetheless, this is a slick and apparently reasonably complete implementation – which may have a bearing on the Mozilla project (see below).
A freely available plugin for Microsoft Internet Explorer® Internet Explorer,
formsPlayer was the other implementation to pass the compliance tests necessary for XForms-REC in
October 2003. Although again there are problems with deep copy - <xf:copy> - the
integration with the browser is excellent, giving an excellent user experience. An active community
helps make this the preferred choice for many intranet deployments where cross-browser support is
lower priority.
An open source (GPL) Flash application that runs in any browser with Flash 6 enabled,
DENG was initially implemented by Claus Wahlers and is being extended by a small team to complete
XForms support. A significant portion of the spec is implemented (yet again, not
<xf:copy> ).
Load performance for larger documents is unsurprising sluggish, this is forgiveable with support for CSS3, XForms, XFrames and SVG. The GPL license is a potential problem for commercial deployment, a commercial offering may be forthcoming. Since professional XForms development is very much helped by the availability of CSS3, this processor is definitely one to watch, and was our preferred choice on the client side.
The Mozilla community have been discussing XForms for some time – at least since August 2001. Many key figures – David Hyatt of IBM, Ian Hickson of Opera, Micah Dubinko from the XForms WG and Brendan Eich of Mozilla amongst them – debated the merits of implementing an XForms processor within the standard Mozilla builds, or as an extension. Despite strong feelings that XForms did not represent the sort of incremental development that has driven the web, votes for this feature continued to flood in until it was #2 on the “most popular” list. Then on 11 August 2004 Mozilla announced that IBM and Novell were collaborating with them to implement XForms for Mozilla, and a project is currently underway .
If code for the Novell exteNd software, or for the IBM XML Forms Package, were to be made available, and given the speed of development which the Mozilla team have demonstrated to be possible, this project may well be bearing fruit before this paper is published. Mark Birbeck of x-port has also invited use of the formsPlayer components within Mozilla.
As well as the recent developments at Mozilla and the open-sourcing of OXF, there is a possibility of a Chiba Java applet (Convex, would allow XForms to run in any Java-enabled browser) and news from Ben Nolan that his JavaScript XForms processor nForms is being released as open source (LGPL). Between them, and with continued improvements to the current offerings, XForms looks likely to move successfully from intranet to extranet over the next 12 months.
There are now plenty of processors available. Micah Dubinko’s list of “Ten Favorite XForms Engines ”, though a year old, is still relevant. The results of a poll in June 2004 showed DENG to be most popular - with good reason, it is highly conformant, performs well and is available under a GPL license for all platforms that support Macromedia Flash 6. Chiba is excellent on the server-side though it needs work to integrate smoothly into a web app. The formsPlayer Microsoft Internet Explorer (MSIE) plugin is great for that browser– so XForms development is now viable for a much wider range of applications than a year ago. After initial tests, we decided we could suspend review of implementations and concentrate on design. We learnt the hard way that drawing up a design solely according to the spec can be rather dangerous.
Our initial design for the Sync Conflict Resolution app specified:
<xf:submit> element, so that it can be pulled straight back into the processing
pipeline;On paper, this looked possible by:
<xf:select1> element with
<xf:itemset> and <xf:copy > to have the XForms processor
iterate over the conflicts, offering choices and dynamically deep-copying the selected fragments to
the output tree.Here's a code snippet that we believe should work – as Mark Birbeck of x-port was kind enough to verify. Again, it's simplified proof-of concept code rather than an implementation of sync resolution.
<html
xmlns="http://www.w3.org/1999/xhtml"
xmlns:xf="http://www.w3.org/2002/xforms">
<head> <title>Attack Ships On Fire</title>
<xf:model id="jotter">
<xf:instance>
<root xmlns="">
<level1>
<level2></level2>
</level1>
<chosenName/>
</root>
</xf:instance>
</xf:model>
<xf:model id="lookup">
<xf:instance>
<root xmlns="">
<bod><name>Rutger</name><initials>RH</initials></bod>
<bod><name>Ridley</name><initials>RS</initials></bod>
</root>
</xf:instance>
</xf:model>
</head>
<body>
<p>Off the Shoulders Of Orion</p>
<xf:select1 ref="chosenName" appearance="compact" model="jotter">
<xf:label>Choose Initials:</xf:label>
<xf:itemset nodeset="bod" model="lookup">
<xf:label ref="initials/text()"/>
<xf:copy ref="."/>
</xf:itemset>
</xf:select1>
<xf:input ref="level1/level2" model="jotter" incremental="true">
<xf:label>Description:</xf:label>
</xf:input>
<p>
<xf:output ref="chosenName/bod/name" model="jotter"/>
is a <xf:output ref="level1/level2" model="jotter"/>
</p>
</body>
</html>
|
In practice, though, this proved a hopelessly over-engineered solution. In particular,
none of the processors we tested successfully handled the deep copy. Fortunately, the flexibility of
XML allowed a much simpler solution that doesn’t require such comprehensive XForms support.
At the cost of some extra post-processing, we moved to a design using a single copy of the instance
data in the XForms model; instead of iterating across conflicts within the XForms processor, we did
more work in the initial XSLT that generated XForms from the sync output – so that each
conflict generates a simple <xf:group> containing a couple of
<xf:textarea> s with <xf:trigger> s to update an annotation on
the instance data. This annotation – an attribute @chosen in our own namespace
– starts with a value indicating no selection has been made and is set using
<xf:setvalue> when a selection is made. On submission, the conflicts are
processed, with the @chosen attribute on a conflict showing us whether a conflict has
been resolved, and which option to select. The generated XForms code is too verbose for
inclusion here.
“Open” selections, with direct entry of alternative text, are possible
because the <xf:textarea> elements do not need to be readonly. We have some
unpleasant XML escaping to handle, but overall this is straightforward to implement, even if it does
generate somewhat verbose XForms documents.
This solution was becoming rather less elegant than our original “single model” design, but it was a good deal easier to implement, and a huge step forward over HTML forms. The problem, though, was orchestration – for which we wanted a simple framework. We wanted straightforward deployment, the ability to roll out our sync conflict resolution kit as easily as possible. We wanted XML pipelines, since the pipeline model is central to our vision for XML change control. We wanted a focus on XML, minimizing coding in procedural or scripting languages. We wanted all the usual good stuff, resilience, scalability, flexibility and so on.
Two contenders led the field – Cocoon [Cocoon] and OXF Section 8.3.2. OXF was proprietary (now LGPL) and did not support the huge range of functionality and connectivity that Cocoon had to offer, with everything from simple SQL through LDAP and IMAP to SOAP and Flow. Despite the learning curve, we chose Cocoon.
Apache Cocoon is an XML application development framework written as a Java servlet and released under the Apache license with a very active developer community. A detailed discussion of Cocoon is out of scope here – see the Cocoon site for more detail.
In overview, Cocoon takes Model-View-Controller distinctions and adds an emphasis on Separation of Concerns (SoC). The concept here is that different teams work simultaneously on the presentation, on the business logic, and on the content, with minimal friction. The current Cocoon “preferred architecture” hinges upon
Many projects will also add a persistence layer, perhaps Hibernate, OJB or iBATIS.
The sitemap provides a clean way to separate the physical layout of documents on the filesystem from the “URI space” seen by the end user. It can be used as the Controller, but this can also be delegated to the Flow system. This is the most compelling feature of Cocoon for webapp development – stateful programming over the Web. At its simplest, Flow uses server-side JavaScript to control the sequence of pages (URIs, each of which may trigger a pipeline) that is returned to the user. Complex validations and error recovery can be smoothly handled with a few lines of clear and maintainable code, and with JavaScript also enabled in the client browser this validation can be made highly interactive. Apart from the Flow definition in JavaScript (or, if you prefer, in Java) everything is defined in XML.
Standing between us and XML app dev Nirvana, however, are a few small problems: documentation; forms; – and complexity.
Cocoon began life as an XML document publishing framework, initial documentation has a clear focus on multi-channel content delivery and similar concerns. Cocoon became Cocoon 2, and began to focus more on developing applications. Cocoon 2.1 refactored, rendering more documentation obsolete. As the system became larger, coherent and authoritative documentation became harder to find.
Forms support began by looking to XForms, with an “XMLForms” component. Offering partial XForms support with server-side mappings and transport using HTTP name/value pairs, development on this slowed and in May 2003 XML Forms was deprecated, to be replaced by JXForms. This again was largely an XForms implementation, with the expectation that lessons had been learned. By February of 2004, however, it was clear that many difficulties resulted from the client-centric perspective of the original XForms WG, who were expecting these features to be implemented in browsers just as HTML support is today, and JXForms was deprecated.
Cocoon’s own forms implementation, originally called Woody and now just Cocoon Forms or CForms, is now the preferred (and only supported) forms technology. CForms currently requires many small, interconnected definitions – form template, form definition, sitemap, flowscript – as the price to pay for maximum maintainability and flexibility. The steep learning curve this produces is not, however, our principal problem. Crucially, CForms expects to be storing its “model” not as an XML document, but into Java beans. For Java developers, this is ideal – but for our purposes, we want access to the raw XML. While the CForms package can certainly be made to work, we have not found it straightforward.
The future or Cocoon looks bright. Once the documentation problems are more fully addressed (plenty of activity here currently) the benefits of Flow and the supremely flexible architecture will become accessible to a wider audience.
There are still problems, though. To use Cocoon in this configuration productively, you need to know XML pretty thoroughly, be familiar with several vocabularies (form template, form definition, JXTemplate, sitemap etc.) as well as with JavaScript, you need at least basic competency in Java, more than that in XSLT; and CSS will be pretty useful too. For large teams, the Separation of Concerns in Cocoon and the ability for different people to work concurrently on different aspects of the system is a real boon. For more lightweight projects, though, this is a pretty big elephant to swallow.
If the client-side situation continues to improve – most especially, if XHTML 2.0 is adopted as a standard and is supported in Microsoft Internet Explorer (MSIE) – we may move instead to delivering our XForms documents directly to the client. That’s a little way off at present.
The more incremental enhancements proposed by the WHATWG [WHATWG] – the Web Hypertext Application Technology Working Group – arise from proposals made by Mozilla, Opera and Apple. “ Web Forms 2.0 ”. The focus is less on “leveraging XML” and more on extending current browsers to make life easier for HTML forms authors. It may even be possible to extend MSIE using custom “behaviors” to meet this spec.
So where do we put our effort? We want the framework provided by OXF or Cocoon, the XML-centric forms processing of XForms, the minimal client browser requirements of HTTP name/value pairs, the interactivity of client-side XForms and the flexibility of Flow - and we’d like all of these things now. A pipedream?
Two projects look as though they could turn a significant part of this dream into reality. One is OXF, which is missing Flow and the client-side interactivity, but has the rest. The other is Chicoon, an adapter project which integrates Chiba with Cocoon. At the time of this writing Chicoon does not support the latest version of Chiba, but development is active. The Cocoon CForms project has shown what can be done when delivering JavaScript to the client to perform validation with minimal server hops. If this can be duplicated in Chicoon we have a viable option.
Why is this so painful? Micah Dubinko suggests that the current situation with XML forms is an example of Clayton Christensen’s Innovator’s Dilemma, with the WHATWG WebForms 2.0 being a “sustaining” technology, and XForms being “disruptive”. Others regard XForms as simply missing the 80/20 mark, as being too large a spec and too tightly coupled with other large specs to be of real value. Certainly an old-school HTML forms webapp would not have been significantly more difficult for us to develop – though it would have been more expensive to maintain, and with with much of the pain behind us subsequent development will be easier.
As ever, it’s horses for courses. If standards compliance is imperative, XForms is the way to go. Where you need maximum interactivity, the audience is defined and can be required to use custom software, X-Smiles, DENG and formsPlayer are worth considering, For the richest experience over HTTP, with dependency only on JavaScript (and pretty good functionality without it) Cocoon CForms with Flow is attractive. For a shallow learning curve and pretty comprehensive server-side XForms (and if the LGPL license is not a problem for you) try OXF.
We are developing using both Cocoon and Chiba – the Chicoon integration is pretty useful. For some projects CForms is great – for the XML pipeline processing we’re doing here, though, XForms is a better fit.
In our processing model, we receive input XML documents, synchronize them and generate a conflicted document. From this we create an XForms document which is processed by Chiba to handle the user interaction. By the time the completed submission has made its way out through the pipelines, we have an XML document containing the unconflicted data together with the user’s selections.
We have been able to “iron out the bumps” in our pipeline – but it isn’t yet easy.
Some factors to consider when choosing a forms technology:
And remember – KISS!
NOTE: We’d be very interested to hear your experiences with using XML forms; what projects are you using it on? What works for you? |
XHTML rendition made possible by SchemaSoft's Document Interpreter™ technology.