The ability to add annotations to content is a requirement within many content management systems, often those in which there is a need for some kind of formal or informal review process. Our software development team was tasked with developing a lightweight and scalable annotation system that would be accessible via multiple interfaces and would integrate flexibly into the workflow of the client's Content Management System (CMS). Increasing budgetary constraints on the part of the client also put a premium on minimizing the time and resources required to develop and integrate the solution.

The development team consisted of three developers. Two team members were on the project from conception to deployment, and the third was involved in the middle of the project to assist with the rendering of the markup into a human-readable form. The project as described took place over a period of two months in the fall of 2001, and was installed at the client site at the end of the development period.

In order to better understand the implementation decisions made during this project, it is useful to first explore the problem of annotation and annotation management in a more general sense. Annotations in general are a specific subset of meta-information with three characteristics:

Annotation management has a potential use case in any system in which metadata is important, and there are a wide variety of real-world examples of annotation management systems already in existence. Applications include compiler information, geospatial information, document management systems, and issue tracking systems.^[1] Several on-line documentation systems use annotations as well, particularly in the open-source community where a premium is placed on end-user interaction, information, and involvement with the product as opposed to central control of the product.^[2]

XML lends itself particularly well to annotation thanks to the structure that it imposes on information, and the linking technologies that have been built around it. This makes the association of annotations with particular parts of the annotated information particularly easy. One of the use cases for XML annotation occurs within large content management environments that require a lightweight feedback mechanism to communicate detailed information regarding specific aspects of the content from one workflow stage to another, usually as a part of a review process.

The authoring system being used by the client was based around Documentum 4i^[3] as a CMS and Arbortext Epic 4.2^[4] as the primary authoring tool, with Solaris 8 as a target server operating system and Microsoft Windows NT and 2000 being the target for clients. Any additions to the system also had to be compatible with an ISOGEN-provided Link Management System (LMS)that had already been developed. ^[5]The client did not require any cross-platform capability beyond the targeted systems.

Figure 1. The general system environment for the annotation manager.

Although the system itself is not particularly complicated, the requirements it had to fulfill were involved:

Although Documentum 4 does have annotation capability, it was deemed insufficient for the needs of the client.^[7] Thus, we set out to design a custom solution that would satisfy the requirements listed above.

For a generic annotation manager system such as the one described in the System Requirements Section 3section, there are only two hard problems that need to be solved:

The other problems involved in satisfying the requirements are significantly less difficult. Actually creating, storing, and manipulating annotations are relatively well-explored problems that can be solved with existing tools, as are rendering the annotatable XML documents for display and building user interfaces to manipulate the annotations.

In this sense, the problem of annotating documents can be seen as a subset of a more general problem, that of linking two pieces of information and managing those links. Once the linking problem has been solved, the creation and management of annotations outside of any system workflow falls into place rapidly. The workflow problem remains, but is less easily generalized.

With this in mind, it was realized that the custom link manager that had already been integrated into the system solved most of the difficult parts of annotation without any additional effort on the part of the team. Although a detailed discussion of it is beyond the scope of this paper, it is not necessarily difficult to build a link management system that will fulfill these basic requirements, depending on what tools are available and the CMS being used. There are three requirements that have to be met by a link manager in order to serve an annotation management function:

Java was determined to be the language of choice for implementing the project. Java was the easiest language with which to share code between the web application for reviewers, the integrated application for editors, and with the already existing CMS customizations that were written mostly in Java.

The system was divided into three packages, each addressing a separate group of requirements:

Figure 2. Annotation manager package listing.

5.1. Design and Implementation of Annotation Manager Core Package

The two driving principles behind the design of the core components for the annotation manager were a desire to separate interface from implementation as much as possible to create a portable implementation, and to make the interface as simple as possible. During the initial design phase, it was determined that a non-implementation-specific interface would allow us to develop the three major components of the system in parallel, further reducing the amount of time it would take the team to deliver the solution. The opportunity to provide a limited interface using a system other than Documentum, which can be complicated to integrate with, meant that limited testing on other components could be done before the core component was fully functional. In this case, the team developed a limited implementation that used Java DataBase Connectivity (JDBC) to access a Microsoft Access database for preliminary testing and development purposes. This prototyping allowed the team to sort out deficiencies in the interface itself quickly, before the more difficult work of creating the full Documentum implementation.

Figure 3. Class diagram of objects in the core package.

The implementation of the annotation management interface in both cases used a relational database to store annotations and manage them. Storing the annotations as XML within the content management system was considered, but was rejected for several reasons. Storing the annotations as XML was an unnecessary complexity with little apparent value, since there was no requirement to be able to access annotations independently of the tool. Storing the annotations in a relational database had the advantage of being significantly faster than storing them as XML within the CMS, and made querying annotations much simpler. Because of the simplicity of the relational database structure and queries for annotations, it was also much faster to implement.

With a more generalized interface, the actual information contained within the annotation is not really important in a general sense, only for specific implementations. Unfortunately, that is less the case with this interface as certain aspects of Annotation objects were placed directly into the AnnotationManager interface.

The Core interface consists of three main abstract objects, implementations of those objects, and some miscellaneous helper classes and exceptions. The AnnotationManager interface contains all of the basic operations of annotation management. The Annotation abstract class defines basic fields that must be present in an annotation. The AnnotationSearch interface defines a basic interface to searching functions. This architecture had the potential to keep the AnnotationManager object itself relatively independent of the structure of annotations.

The problem that this architecture does not solve, at least not very well, is the problem of integrating workflow. This is the point in the design where the team ran up against the second hard problem in the construction of an annotation management solution, and suffered for it.

The original implementation of the annotation manager ended up with an interface heavily polluted by annotation specifics and workflow. Since the aim of the project was a custom annotation manager done quickly and cheaply, a decision was made to not refactor the interface.

Workflow ended up being the most damaging addition to the interface, and caused the most problems in actual implementation. The actual workflow required was quite simple in concept: Reviewers would receive a document in their inbox, make annotations on it, and promote it to the next stage of the workflow when finished. Authors would receive a document in their inbox with annotations attached. They would then be responsible for resolving those annotations and promoting the document to the next stage of the workflow when it was ready.

Implementing this workflow within the Documentum system was less than simple, particularly since the team was inexperienced with Documentum workflow at that time. The implementation of the system reflects this with added clutter that is too Documentum-specific to be as generic as we desired. Promotion methods and other workflow-specific concepts leaked into the AnnotationManager interface and were never factored out into a separate class.

Work began on the annotation manager in mid-September of 2001 with the two original team members starting development work on the Core packages and web interface. Work on these proceeded in parallel. Around the first of October an additional developer was added to create the SAX transformation code used within the web interface. Development against the Documentum implementation of the core package, rather than against an Access-based mockup, began in mid-October round. The SAX transformation code was completed around the same time, although client requests would cause it to be tweaked throughout the construction of the system. At this point the team shrunk back to the two core developers.

An initial review of the system with the client was held on the 17th of October, and at that point the system was functional enough for a nearly complete demo. A second customer review took place on November 9th. Both of the reviews were remote demonstrations rather than client installations. Development plus testing were complete on November 19, and the system was installed at the client site a week later. In all, the development of the entire system took roughly two months for a team of two developers, plus another two weeks from a third developer for the SAX XML to HTML transformations.

Much of the development time was spent learning how to implement what we knew we needed to do within Documentum. Had we known what we do now, or if we were working with a CMS in which we had more experience, it is probable that another one to two weeks could have been saved.

Adding a customized annotation management system to an existing CMS solution is not necessarily an extraordinarily difficult task, as long as the two fundamental problems of annotation management are kept firmly in mind while designing the system and the implementation of the system is kept as simple as requirements allow. The interface may not be extremely polished for the initial delivery, but can be refined later if the annotation manager architecture is designed in a flexible manner. Customized annotation management can add a significant amount of value to a content management system for a relatively small cost, particularly if a general link management solution is available or has already been integrated.