XSLT and Scripting Languages

1. About XSLT

XSLT is the only programming language standardized specifically for processing XML. Although it does not look like a programming language, it is in fact one.

1.1. Programming Language Features in XSLT

In order to demonstrate that XSLT is in fact a programming language, it will be helpful to show how features from other programming languages manifest themselves in XSLT.

1.1.1. Looping

Consider the following input document:

<people>
<person><name>Paul Prescod</name>
<role>Snake Charmer</role>
</person>
<person><name>Gisle Aas</name>
<role>Camel Herder</role>
</person>
</people>

Now consider an XSLT transformation to transform it into a table model similar to HTML's.

<xsl:template match="people">
   <TABLE>
       <xsl:for-each select="person">
           <TR>
               <TD><xsl:value-of select="name"/></TD>
               <TD><xsl:value-of select="role"/></TD>
           </TR>
       </xsl:for-each>
   </TABLE>
</xsl:template>

For each person element in the people element, there is a new table row (TR) containing two table cells (TD). This form of looping is very limited because it can only be driven by data in the input document, not data that is the result of an internal computation. To do other kinds of looping, you need to use recursion. To do recursion you need to know how to do function calls so we will cover that next.

1.1.2. Function Calls in XSLT

Functions are the most basic unit of code in most programming languages. A function takes input, calculates a value and returns output. XSLT's basic code unit is a template. Templates also take input and return output, but XSLT is unusual in that the most important inputs and outputs to the function are implicit: the current node is automatically passed to every template and the result is automatically returned to the caller (or written to the output stream if the caller does not use it).

XSLT has two ways of calling functions. One is based on a template's name. The other is based on pattern matching against the current node. Calling based on name is most like function calls in most other languages.

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
     <xsl:template name="a" match="/">
									<xsl:message>In templae a</xsl:message>
         <xsl:call-template name="b"/>
     </xsl:template>
     
     <xsl:template name="b">
         <xsl:message>In template b</xsl:message>
         <xsl:call-template name="c"/>
     </xsl:template>
	  
     <xsl:template name="c">
         Blah
         <xsl:message>In template c</xsl:message>
     </xsl:template>
     
</xsl:stylesheet>

This code will start with the rule "a", continue to "b" and on to "c" just as in a programming language with functions named "a", "b" and "c". This is not the usual way one does XSLT programming but it is useful in implementing more complex algorithms. To implement recursion, you merely call a template from itself, or set up a pair of templates that are mutually recursive.

1.1.3. Parameters and Variables

XSLT templates may also take parameters and return text values.

<xsl:template name="a">
  <xsl:variable name="year" select="2001"/>
  <!-- Set up a "year" variable -->
  <xsl:call-template name="b">
    <!-- pass it as a parameter named "the-year" -->
    <xsl:with-param name="the-year" select="$year"/>
   </xsl:call-template>
</xsl:template>

<xsl:template name="b">
  <xsl:param name="the-year"/>
  <xsl:message>In template <xsl:value-of select="$the-year"/></xsl:message>
</xsl:template>

The xsl:with-param instruction passes a named parameter. The xsl:param instruction declares the parameter in the template that uses it.

1.1.4. XSLT and Turing Completeness

This demonstration of looping, variables, function calls and parameters may convince you that XSLT is similar to a programming language, but what about the computer science definition of a programming language: Turing completeness? If a language is Turing complete then you can write functions to compute anything that could be computed in any other programming language, such as C++ and Java.

XSLT does in fact meet the requirements for Turing completeness. That is to say that it can do any computation that any other programming language can do. A formal proof is outside the scope of this paper but informally this could be demonstrated by implementing the Turing "tape" as a long XSLT string and using XSLT's string manipulation functions (substr, concat) to write to the tape. Recursion must be used to advance or reverse the tape.

Turing completeness means that if you are using XSLT and you can you can envision a solution to a problem in some other language then there also exists an XSLT solution. This solution may be extremely difficult to program in XSLT and it may take an inordinate amount of time to execute. Nevertheless, it is useful to keep the fact of XSLT's Turing-completeness in mind. As you are learning XSLT, you should not give up on XSLT when a solution to a problem takes a little bit of extra thought. Sometimes the XSLT solution will actually turn out to be efficient and elegant. Keep in mind the power of the string manipulation functions and XSLT's ability to call templates recursively.

1.2. Weaknesses

Turing completeness means that in theory you could use XSLT to implement an encryption algorithm or (heaven forbid!) a weather simulation. Unfortunately, coding these sorts of highly algorithmic applications in XSLT would be excruciatingly painful. The performance would also be very poor.

XSLT also has a severe limitation when it comes to input and output. General purpose programming languages tend to have dozens of channels for input and output: files, pipes, sockets, GUIs, relational databases, character mode devices and so forth. Standard XSLT has only a few inputs: the stylesheet, the input file, parameters and "the Web" (the universe of URL-addressable resources). Admittedly, the Web is a pretty powerful and rich input, but even so, not all of the information in the world is on the Web in forms that can be readily processed by XSLT.

XSLT is not good at handling information that is not already in XML. Although it does have some primitive text processing features, they are quite inconvenient to use. Therefore it is very difficult to use XSLT to handle input that is not XML. Output is not quite as difficult but even there, XSLT is strongly biased towards XML or HTML output. Raw text output is not as easy.

2. Scripting Languages

Scripting languages have, over the last ten years, taken on an increasing portion of the software development burden. For our purposes we will define a scripting language as a language with very high level data types and data structures, dynamic runtime type checking, open source implementations and a strong set of features for the delivery of web pages and the administration of server machines.

Languages in this category include Perl, Python, PHP, Tcl and Ruby. Unlike scripting languages of ten years ago, these languages are featureful and designed to scale to large problems. Most of them have built-in object orientation and the only one that doesn't (Tcl) has object orientation as an optional extensions. They also have modules, namespaces and exception handling, just like traditional programming languages.

2.1. Compared To Other Programming Languages

One of the distinguishing features of these languages is that they are dynamically type checked. This allows programmers to construct programs with fewer lines of code but moves some of the burden for correctness checking from the compiler to the test suite. The reduced compile-time visibility also hurts the performance of these languages when compared to statically type-checked languages. Dynamicity also allows these languages to easily deal with heterogeneous data sources such as complex XML documents or dynamically discovered web services.

Many scripting languages have built-in support for XML but they typically require extension libraries to rival the XML-handling power of XSLT.

3. Advantages of XSLT

XSLT would not exist if its developers did not think it offered advantages beyond those already available from scripting languages. Although this point has been controversial, I think it is safe to say that some problems are much easier solved in XSLT than in a traditional scripting language unless the scripting language has been massively extended with sophisticated libraries. We will discuss these libraries later.

XSLT's central advantage is that it was designed for use with XML. This has several aspects. These will be explored in the next few sections.

3.1. Reliable XML Model

XSLT has a well defined XML data model that requires all compliant XSLT engines to parse XML in exactly the same way. Scripting languages tend to depend on the DOM or on a non-standard API. Even when they depend on the DOM, they do so in a variety of ways. Some normalize text nodes when the document is loaded. Some do not. Some support CDATA nodes. Some do not. Some discard entity boundaries. Some preserve them.

Although XSLT processors are much more predictable and reliable in their parsing, there are two variants of XSLT processors that you should watch out for. Those that use validating parsers can differ somewhat from those that use non-validating ones. Scripting languages and traditional programming languages also have this problem. In fact the problem stems from XML itself. Portable XML-processing code should not rely on the existence of the DTD.

3.2. Tree Walking

XML documents are trees. XSLT is organized around the notion of walking down trees from the root nodes to the leaves and from the start of the document to the end. XSLT is therefore very convenient for processing documents from top to bottom as is often necessary for conversion and formatting tasks.

The default behaviour of even an empty XSLT transformation is to walk down the XML tree from the start of the document to the end, examining each element and outputting each text node. Many simple transformations consist merely of adding some functionality to that process. Because XSLT by default does the document traversal that most people want, traversal code is minimized.

On the other hand, if you want to do some form of document traversal that is not like the one built into XSLT, your task will get harder. For instance if you wanted to process the document backwards or traverse from link references to their referents, this would be a little more difficult.

3.3. Templates

XSLT documents are XML. This means that there is a very natural syntax for representing the XML elements to be generated in the output: XML elements! An XML-generating program in a scripting language template would typically look something like this:

$var = setup_some_variable();
output("""
<somexmlelement attribute="$var">
    <etcetc>
 """)
do_some_processing()
output("""
    </etcetc>
</somexmlelement>
 """)

XSLT is designed to move back and forth between code and data more seamlessly. For example, computing an attribute value can be done right in a template for the attribute value. As another example, XSLT has a fairly natural built-in way of representing namespaces. It just uses the XML namespaces declared in the XSLT document. An XSLT equivalent of the code above is all in XML syntax.

<somexmlelement attribute="{calculate/some/value}">
   <etcetc>
	   <xsl:apply-templates/>
	</etcetc>
</somexmlelement>

With one important exception, scripting languages are not as well set up to do templating. The exception is PHP. PHP has excellent support for templating. It was designed as an HTML templating language but also works fine for XML. Most other scripting languages do have special templating extensions that can be used to approximate the templating ease of PHP.

XSLT's XML heritage is not a pure advantage, however. There are times when it can be inconvenient. For one thing it is always somewhat verbose. For another, controlling the generation of whitespace can be somewhat tedious for output formats where that matters. XSLT has no direct way to generate entity references, including named character references.

3.4. Pattern Matching

XSLT templates are triggered based on patterns expressed as XPaths. [XPath]XPath is an extremely rich and sophisticated language for node matching and selection. Scripting languages can be extended with libraries that incorporate XPath engines. But it is not common for scripting languages to have XPath engines (or anything of equivalent power) until somebody has implemented an XSLT engine in that language. XSLT seems to always be the impetus to improve a language's XML processing power to the point where the language competes with XSLT.

3.5. Optimization Opportunities

A language like XSLT offers both opportunities and difficulties in terms of performance. The primary opportunity is that XSLT is much more constrained and less flexible than a general purpose language (especially a scripting language). For instance XSLT disallows one template from overwriting a variable that is potentially in use by another template. That means that a variable can be looked up once and have its value cached somewhere (even in compiled code for the template itself) rather than having the value be looked up over and over again in some lexically scoped namespace. Also, XSLT is much easier to implement than a scripting language because its domain of application is so much more constrained. That leaves an implementor more time to pay to attention to optimization techniques.

Scripting languages can also have performance advantages in some circumstances. Where XSLT puts performance tweaking into the hands of the engine implementor, scripting languages can give the transformation programmer more leeway for performance optimizations in a particular transformation. All else equal, it is better to do performance optimizations once in an XSLT engine instead of over and over again in each individual transformation. Practically speaking, however, the transformation programmer has more knowledge about the performance requirements of the code they are writing than the engine implementor does. It is therefore appropriate in some cases for the transformation programmer to want to take on responsibility for performance.

3.6. Ubiquity

Being embedded in an important system is a surefire way for a language to become a success. Languages like DOS batch files, Unix shell scripts, JavaScript and Microsoft's Basic variants surely did not become popular based purely on their technical merits. It was through being integrated into an important system (DOS, Unix, Netscape and Visual Basic) that these languages became important.

XSLT is similarly embedded throughout the XML infrastructure. Databases embed XSLT, Internet Explorer and Windows XP ship with XSLT implementations. Most Linux distributions probably ship with at least one by now. This ubiquity is a remarkable achievement that has not yet been matched by any general purpose scripting language (which is to exclude JavaScript).

4. Advantages of Scripting Languages

Scripting languages are general purpose programming languages. Through libraries you can use them talk to databases, create graphical user interfaces, communicate with SOAP-based web services and do almost anything else that can be done with computers. That is why they are considered general purpose.

XSLT is designed for only one purpose: transforming XML documents. Of course almost any computational problem can be framed as a transformation between XML documents. But even so, XSLT is completely dependent on other programming languages to take the diverse sources of information in the world such as keystrokes, mouse clicks and database records and render them as XML. In the end, this is what will help most people to choose between XSLT and scripting languages for a particular project. If the input is XML, XSLT may well be a good choice. Otherwise, scripting languages are usually better, if only to translate the input into XML for handling by XSLT.

Another advantage of scripting languages is their rich ability to create complex data structures which can be acted upon in complex algorithms. XSLT 1.0 is extremely limited in its set of data structures. The input document is of course an XML tree, but the only data structure that can be generated at runtime is a string of characters! Variants of XSLT allow the construction of trees and node lists at runtime. Future versions of XSLT will likely do so also.

Moving from one data structure to two is a big step forward but still does not compare to a language with pointers or references where any data structure can be built with very efficient links from place to place. Making a linked datastructure in XSLT requires the use of inefficient text-based hyperlinks.

5. Integrating the Technologies

For any particular bit of code it is of course necessary to choose between XSLT and a scripting language, based on the strengths and weaknesses discussed earlier. But in many projects it is worthwhile to combine the two and thus get some of the advantages of each.

5.1. Preprocessing

Scripting languages may be used to generate XML for use in XSLT. One very basic way to do this puts the scripting language in the driver's seat:

tempfile.write("<xml …>")
run("xslttrans conv.xsl tempfile outfile")
data = outfile.read()

Another more elegant way is to give the script a web address (e.g. by using CGI or Zope) and have it generate XML on the fly. This is more elegant because it means that any XML consuming program (XSLT or otherwise) can access the data as XML. In buzzword-speak the scripting language becomes a kind of legacy->XML middleware. XSLT transforms can fetch data from many URLs and integrate them so the XSLT can serve as a data integration technology.

5.2. Scripting in XSLT

It is also possible to put script code right into an XSLT transformation. Most XSLT implementations have specific extensions that allow this sort of embedding.

<lxslt:component prefix="my-ext" functions="getdate"> 
  <lxslt:script lang="javascript">  
  function getdate(numdays){
	 var d = new Date();
		…
		return d.toLocaleString(); 
  }  
		</lxslt:script> 
</lxslt:component>

Using these extensions, it is easy to let the script code do what script code is good for and the XSLT do what XSLT is good for. The primary problem with these extensions is that each XSLT implementation must choose which languages to embed and what syntax to use for the embedding. This leads to transformations that are not portable between XSLT implementations. XSLT 2.0 will likely standardize JavaScript and Java bindings but it is not clear whether XSLT engines implemented in C++ and Python will actually support scripts in JavaScript and Java.

It is also worth noting that there is an impassioned group of XSLT implementors and users who are against the embedding of scripts in transformations. Their points are strong and forcefully made and thus deserve consideration. [Petition to remove xsl:script from XSLT 1.1]

5.3. Cross Dressing

Once XSLT has been implemented in a particular scripting language, it becomes apparent that it would be possible to structure the implementation so that rules could be written in the scripting language rather than in XSLT syntax. One example of this is EventDOM (Python).

Here is an example of an EventDOM rule that gets triggered based on an XPath:

def handle_spam( textNode ):
    "figure/title/text()"
    print "Figure title:"+`textNode`
		  

In this way it is possible to write Python code in a style that closely emulates XSLT. On the other hand, this would not likely be implemented if it were not for the existence of XSLT processing libraries in Python. There have also been similar toolkits for Perl and other languages, but none have become as popular as XSLT. This may be because they require scripting language programmers to think in a new mode and at the same time are less standardized than XSLT. So XSLT-ish APIs alienate some portion of scripting language advocates without attracting those without a predefined language allegiance.

6. Conclusions

There do seem to be certain tasks that XSLT excels at and others that are best handled by scripting languages or traditional statically typed programming languages. Over time, it seems that XSLT and scripting languages will grow closer and closer together as XSLT implementations in scripting languages mature and more and more scripting languages are embedded into XSLT. The opportunities for cooperation are vast but the areas of overlap are also growing. Over time, the choice of XSLT embedding scripting versus scripting embedding XSLT will likely become more a question of one's community and background than a decision about technology.

Design and Development by deepX Ltd.