Typing in transformations

Introduction

Of all the new features in XSLT [Extensible Stylesheet Language Transformations] 2.0, the ones that have caused the most contention — and worry — amongst existing XSLT authors are those involving typing: strong typing, static typing, and validation against schemas. These features have been introduced due to two general requirements: the requirement to align with W3C XML Schema, which entails using XML Schema’s type system and validation methods; and the requirement to support analysis of queries prior to their execution and warn authors of potential problems with them.

When XSLT authors express their concern about these changes, they are generally assured that, at the end of the day, if they don’t want to use the new type-related features that XPath 2.0 brings, they can always ignore them by using processors that are not schema aware and do not carry out static type checking.

In this paper, I’ll look at the extent to which this assurance is true: to what extent is it possible to ignore the new typing features in XPath 2.0? If they can’t be ignored, what are the kinds of changes that XSLT authors will have to get used to? And most importantly, do these changes offer users any real benefits?

XSLT 2.0 offers two conformance levels for XSLT processors: Basic, which does not require support for understanding schemas, and Schema-Aware, which does. For the purposes of this paper, I am going to focus on XSLT authors who are not using schemas to validate the XML documents that they work with, either because such schemas do not exist or because they want their stylesheets to be portable to Basic XSLT processors. The built-in types defined in XPath 2.0, including all the built-in types from XML Schema, are still available in these stylesheets, but without schema validation, nodes are all untyped and no user-defined types are available. This means that schema-unaware XSLT 2.0 stylesheets do not benefit from some features that are undoubtedly useful in XSLT 2.0, in particular, the ability to create templates that match, with a simple pattern, groups of elements based on either their name (via a substitution group) or type.

I will also constrain the discussion to behavior without the XPath 1.0 compatibility mode set. When set, XPath 1.0 compatibility mode alters the way that values are converted to each other, the aim being that expressions that were valid in XPath 1.0 work in the same way as they did in XPath 1.0. While XPath 1.0 compatibility mode will doubtless prove useful in migrating from XSLT 1.0 to XSLT 2.0, I want to focus here on the end-point of that journey, particularly since some portions of a stylesheet (in particular, the code within a function definition) are never run under the backwards compatibility rules.

This paper will start with a survey of the typing-related differences between XPath 1.0 and XPath 2.0, to explain why XPath 1.0 is characterized as a weakly typed language while XPath 2.0 is characterized as a strongly typed language. The differences fall into three categories, which I’ll go on to look at in more detail. First, they differ in the type system that’s used: the XPath 2.0 type system is based on the XML Schema type system, and I’ll discuss how to use these types in XSLT 2.0. Second, XPath 1.0 and XPath 2.0 differ in the rules used when converting between types; I’ll survey what these rules mean in terms of the expressions that you can use. Thirdly, in XPath 2.0 variables and parameters are typed whereas in XPath 1.0 they are not; again, I’ll discuss the impact of this change on XSLT authors. Finally, I’ll draw some conclusions about the relative ease of writing stylesheets in XSLT 2.0 due to these changes.

Weak and strong typing

So what are the type-related differences between XPath 1.0 and XPath 2.0? Perhaps the biggest difference is that XPath 1.0 is said to be a weakly typed language while XPath 2.0 is said to be a strongly typed language. The term “strongly typed language” is fairly nebulous, meaning different things to different people. A good list of possible overlapping definitions for “strongly typed language” can be found at http://c2.com/cgi/wiki?StronglyTyped:

Let’s look at each of these in turn to see which characteristics of XPath 2.0 make it a “strongly typed language”.

In XPath 2.0, type annotations are associated with both variable names and with values. Both atomic values (scalars) and nodes are annotated with types. When they are declared (using XSLT¹), variables and parameters can be assigned types using the as attribute. If you try to assign a value to a variable or parameter and the type of the value does not match the type of the variable or parameter, then you will get a type error. This differs from XPath 1.0 in which variables do not have types but values do.

XPath 2.0 does define how to carry out compile-time checks of type constraint violations, otherwise known as static type checking. Static type checking is an optional feature in XPath 2.0, and is not required at either XSLT conformance level, but the language and the type system has been designed specifically to enable processors to support it if they wish. In particular, XPath 2.0 ensures that it is always possible to identify the type of the result of an expression during analysis even when that expression is polymorphic (returns different results for operands of different types). If the processor does not support static type checking, it will report type errors at run time instead.

In XPath 1.0, there are fairly few situations when there are type constraints to violate (since most types can be converted to each other on demand) — checking that a predicate is actually being applied to a node set (rather than to a string or number) is one example of type checking in XPath 1.0. There is no statement in XPath 1.0 about when type checking occurs, so as with XPath 2.0, it’s really up to the processor. However, since variables are not typed, it can be quite hard work to do type checking during the analysis phase.

Both XPath 1.0 and XPath 2.0 allow you to convert between (some) different types. As we’ll see in more detail later, in XPath 2.0 most conversions must be carried out explicitly using the constructor functions (such as xs:date() and xs:double()) or a cast as expression whereas in XPath 1.0 most conversions are carried out implicitly.

Both XPath 1.0 and XPath 2.0 allow you to make explicit casts of atomic values to different types. XPath 2.0 also has a treat as expression to evade the static type checking of sequences and nodes, so under this definition both versions of XPath would be characterized as weakly typed.

XPath 2.0 certainly has a complex, fine-grained type system, based on that defined in XML Schema, while XPath 1.0 only has four types (six if you include XSLT 1.0’s result tree fragments and the “foreign object” type). On the compound-type front, it’s notable that while you can define the type of the items in a sequence, a sequence is not itself annotated with a type. For example, you can declare that a variable holds a sequence of xs:string items, but the sequence itself hasn’t got a type, such as “list of strings”.

In a way, this definition does not apply in XPath because XPath is a functional language in which data cannot change; in any case, it is certainly true that the the type of a piece of data is fixed in both versions of XPath.

The following table summarizes these characteristics:

This summary shows that while XPath 1.0 is (by most definitions) a weakly typed language and XPath 2.0 is (by most definitions) a strongly typed language, this distinction actually arises because of three main differences in XPath 2.0:

• typing of variables
• explicit casting required in most situations
• complex type system

In the rest of this paper, I will look at each of these new features in XPath 2.0, the impact that it has on the stylesheets that you create and the benefits (and costs) that it brings.

XPath 2.0 type system

Before we launch into the details of using XPath 2.0’s type system, let’s have a quick look at the way it works.

In XPath 2.0, everything is a sequence of zero or more items. Sequences cannot hold other sequences, and a singleton sequence, containing only one item, is equivalent to the single item that it contains. Items come in two flavors, nodes and atomic values.

Nodes are familiar from XPath 1.0 and represent logical structures within an XML document, such as elements and attributes. In XPath 1.0, we are used to using the name and string value of a node. In XPath 2.0, we can also use the type of a node and use the typed value of a node rather than its string value. In the kind of documents that we’re looking at in this paper — those that have not been validated against a schema or DTD² — element nodes will have the type xs:anyType, and attribute nodes will have the type xs:anySimpleType. The typed value of all nodes in these circumstances is the same as the string value of the node, but with the type xdt:untypedAtomic.

Atomic values are scalars like the strings, numbers, and booleans in XPath 1.0. In XPath 2.0, every atomic value has a type from amongst the atomic types that are built in to XML Schema, plus a few extra that have been defined for the purposes of XPath 2.0. The following diagram shows the hierarchy of atomic types that are built in to XPath 2.0.

Figure 1: Built-in atomic types in XPath 2.0

[Link to open this graphic in a separate page]

This hierarchy is daunting for those used to the three atomic types of XPath 1.0. While there’s no need to use a particular type if you don’t need it, authors will need to have some awareness of what the type hierarchy does and does not contain when writing their own functions, and need to know how it’s organized in order to use the casting rules that we look at later. In fact, the vast majority of these 45 types can be avoided. The most commonly used types will probably be:

• xdt:untypedAtomic, the type of the typed value of untyped nodes
• xs:string, a string in which whitespace is significant, equivalent to a string in XPath 1.0
• xs:token, a string in which leading, trailing, and runs of whitespace are not significant
• xs:boolean, a boolean value, equivalent to a boolean in XPath 1.0
• xs:QName, a qualified name with a namespace URI and a local part
• xs:double, a double-precision floating point number, more or less equivalent to a number in XPath 1.0
• xs:integer, an integer
• xs:dateTime, an date and time (with timezone)
• xs:date, a date (with timezone)
• xs:time, a time (with timezone)
• xdt:yearMonthDuration, a duration made up of a number of years and a number of months
• xdt:dayTimeDuration, a duration made up of a number of days, hours, minutes, and seconds

These are the types for which there is good support in the built-in function library in XPath 2.0. There are some types that should be avoided solely because of the lack of support for them in the built-in function library. For example:

• Rather than using xs:duration, use the appropriate one of xdt:yearMonthDuration and xdt:dayTimeDuration, which have been introduced because there is no reliable way to compare full durations. If you have to use a duration that involves both years or months and days, hours, minutes or seconds, then you will have to cast the duration to xdt:yearMonthDuration and xdt:dayTimeDuration values and compare or carry out arithmetic with them. For example, to add P1Y1D (a year and a day) to the current date, you have to add the year and the day separately, as follows:

  (current-date() + xdt:yearMonthDuration('P1Y')) + xdt:dayTimeDuration('P1D')

• Most of the functions that you would think accept or return xs:anyURI values as arguments actually require or return xs:string values. This makes it easy to call such functions with string literals, for example:

  collection('docs.xar')

• The only date/time types that have real support are xs:dateTime, xs:date and xs:time. For example, you can’t test whether one xs:gYearMonth is greater than another or extract the day component from a xs:gMonthDay.

These limitations aside, the date/time, duration and qualified name support in XPath 2.0 in particular will allow us to carry out some transformations with a lot more ease, portability, and accuracy than currently. Rather than employing extensions or complicated templates to work out the difference between two times, for example, we can just subtract them from each other; rather than having to parse dates in consecutive <xsl:sort> elements in order to sort by them, we can just point to the date and let the processor do the work:

<xsl:for-each select="Trans">
  <xsl:sort select="xs:date(@Period)" />
  ...
</xsl:for-each>

Only some of the types (namely strings and the numeric types) have corresponding literals that allow you to create values easily. To create values of most of the types or to cast to a particular type, you will need to use the constructor functions that we’ll look at in the next section. To use those functions, you will need to declare the xs namespace of http://www.w3.org/2001/XMLSchema. So the first impact of typing on XSLT 2.0 is that most stylesheets will need to have an extra namespace declaration and an exclude-result-prefixes attribute to ensure that the namespace isn’t carried through into the result document:

<xsl:stylesheet version="2.0"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:xs="http://www.w3.org/2001/XMLSchema"
                exclude-result-prefixes="xs">
  ...
</xsl:stylesheet>

If the stylesheet refers to xdt:yearMonthDuration or xdt:dayTimeDuration, then it may also need to declare the xdt namespace of http://www.w3.org/2003/05/xpath-datatypes³:

<xsl:stylesheet version="2.0"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:xs="http://www.w3.org/2001/XMLSchema"
                xmlns:xdt="http://www.w3.org/2003/05/xpath-datatypes"
                exclude-result-prefixes="xs xdt">
  ...
</xsl:stylesheet>

These additional namespace declarations are a little bit tedious, but the document element of a stylesheet is usually generated automatically or copied en masse anyway, so they are not too much of a burden for XSLT 2.0 authors.

Converting between types

As we’ve seen, one of the major differences between XPath 1.0 and XPath 2.0 is that while in XPath 1.0 type conversions are done implicitly (where they make sense), in XPath 2.0 they have to be done explicitly. In this section, we’ll look at what that means in practice for an XSLT author.

@date cast as xs:date?

xs:date('1/4/2003')

xs:date(concat(@Period, '-01'))

if (concat(@Period, '-01') castable as xs:date)
then xs:date(concat(@Period, '-01'))
else ()

<xsl:for-each select="Invoice">
  <xsl:value-of select="concat(position(), ': ', count(LineItem), ' items.')" />
</xsl:for-each>

<xsl:for-each select="Invoice">
  <xsl:value-of select="concat(string(position()), ': ', 
  string(count(LineItem)), ' items.')" />
</xsl:for-each>

<Trans AccNo="401000" TransType="ACT" Period="2000-05" Amount="-691126.97" />

floor((substring(@Period, 6, 2) - 1) div 3) * 3 + 1

floor((xs:integer(substring(@Period, 6, 2)) - 1) div 3) * 3 + 1

((xs:integer(substring(@Period, 6, 2)) - 1) idiv 3) * 3 + 1

substring(@Period, 6, 2)

(@value * 2) idiv 2

xs:integer(@value * 2) idiv 2

string-pad(' ', @indent)

<dress sizes="6 8 10 12 14 16">
  ...
</dress>

<xsl:function name="my:dress-fits">
  <xsl:param name="sizes" as="xs:integer+" />
  ...
</xsl:function>

my:dress-fits(dress/@sizes)

my:dress-fits(for $s in tokenize(dress/@sizes, '\s+')
              return xs:integer($s))

@indent ne 0

xs:integer(@indent) ne 0

@indent != 0

@width > @height

number(@width) > number(@height)

@date - current-date()

xs:date(@date) - current-date()

string-pad(' ', @indent + 2)

string-pad(' ', xs:integer(@indent) + 2)

Typing of variables and parameters

In XSLT 2.0, the variable-binding elements <xsl:variable>, <xsl:param>, and <xsl:with-param> all take an as attribute that specifies the type of the value of the variable or parameter. The value of the as attribute is a SequenceType, which is a pattern that matches values of particular types, for example:

• xs:date — a single value of type xs:date
• xs:integer+ — a sequence of one or more xs:integer values
• node()? — an optional node
• element(Trans,*) — a <Trans> element (of any type)
• attribute(@id,*) — an id attribute (of any type)
• item()* — any number of items of any kind

When you declare the type of a variable or parameter, then the value that you supply for that variable or parameter is cast to that type as long as it can be cast implicitly using the rules described above. In other words, if the type of the value is a subtype of the type declared for the variable or if the value is untyped, then the value will be cast as you’d expect, for example:

<xsl:variable name="date" select="current-date()" as="xs:date" />

<xsl:variable name="date" select="@date" as="xs:date" />

<xsl:variable name="date" select="'2003-04-01'" as="xs:date" />

<xsl:variable name="date" select="xs:date('2003-04-01')" as="xs:date" />

<xsl:variable name="date" as="xs:date">2003-04-01</xsl:variable>

The value of the text node is untyped and can, therefore, be cast to a date implicitly.

If you don’t specify an as attribute then no conversion takes place. In the following cases, the date variable is still recognisable as an xs:date because the type of the selected value is an xs:date:

<xsl:variable name="date" select="current-date()" />
<xsl:variable name="date" select="xs:date('2003-04-01')" />

To set the date attribute to the xs:date value of an untyped date attribute, the following are equivalent:

<xsl:variable name="date" select="@date" as="xs:date" />
<xsl:variable name="date" select="xs:date(@date)" />

So in fact, when setting variables or passing values to parameters with <xsl:with-param>, there is no need to use the as attribute unless the variable or parameter’s value is being set using the content of the variable-binding element, although doing so might help an XSLT processor to perform static analysis of the stylesheet.

Specifying the type of parameters when they are declared is another matter, however. On the <xsl:param> element, the as attribute specifies the required type of the value that’s specified for a parameter. If the wrong type of value is passed for the parameter, then a type error will be raised. Since the declaration of the parameter and the setting of its value occur in different places, this is a good thing — a place where type checking can help ensure that the functions and named templates that you define are being called with the correct types of arguments. What’s more, declaring the type of the parameters simplifies the code within the function definition or template because you won’t need to use explicit casting to use the parameter value (assuming, that is, you declare it with the correct type).

On the other hand, you need to be careful about the types that you use for parameters in order to make the function or template easy for people to call. For example, if you were writing a function with an argument that has to be a positive integer, you might be tempted to declare the type of that parameter as xs:positiveInteger. If you did so, however, those using your function (including yourself!) would have to use explicit casts to create values of the correct type. The general rule of being generous in what you accept and strict in what you produce apply here: using general types, such as xs:string and xs:double, for arguments and specific types, such as xs:language and xs:positiveInteger, for return values will make your functions and templates a lot easier to call and use.

Conclusions

This paper has reviewed the type-related changes that XPath 2.0 brings and their impact on XSLT authors. There are three type-related changes in XPath 2.0:

• complex type system
• explicit casting required in most situations
• typing of variables

The complexity of the type system is largely a consequence of XML Schema’s datatype specification, with a few extra types (notably, xdt:yearMonthDuration and xdt:dayTimeDuration) added to adapt it for computation rather than validation. However, even with these changes, the relationships of the types within the XML Schema type hierarchy is not well-suited for the strong typing rules that XPath 2.0 employs. This is evident in several places where the general rules of the XPath 2.0 type system are broken to make life easier for users, for example:

• the special casing of casting between the various numeric primitive types
• the use of xs:string, rather than xs:anyURI, as the type of arguments that have to be URIs

As we’ve seen, explicit casting makes life harder for the XSLT author. When working with typed data, or with data-oriented XML in which values fit nicely onto the type system, the implicit casting rules work reasonably well. However, there are four not uncommon situations where XSLT authors working with untyped data will find themselves having to explicitly cast values:

• when manipulating a non-string value, such as a date/time or a number, as a string
• when using the results of string manipulation of a value, such as a substring or the result of regular-expression processing
• when performing date/time arithmetic
• when performing integer arithmetic

These casts add to the length and complexity of the code and require the XSLT author to know, for example, which functions require xs:integer arguments and which, xs:double arguments.

One argument for the strong typing introduced to XPath 2.0 is that it should help the processor identify errors at analysis time and, thus, help the author create code that does the right thing. The kinds of errors that users are interested in are those where they have made mistakes in their algorithm or function calls. But XPath 2.0 processors will raise errors when there is nothing wrong with the algorithm used. An example we looked at earlier was:

((substring(@Period, 6, 2) - 1) idiv 3) * 3 + 1

There is nothing wrong with the algorithm being used here. When the XSLT 2.0 processor raises a type error because a xs:string can’t be an operand of a subtraction, it’s more likely to be an irritation to the XSLT author than an assistance. Even if an XSLT 2.0 processor doesn’t implement the static typing feature of XPath 2.0, and can recognize dynamically that the result of the substring() function can be cast to an xs:integer, it still has to raise the type error. And even when the cast is added:

((xs:integer(substring(@Period, 6, 2)) - 1) idiv 3) * 3 + 1

if (substring(@Period, 6, 2) castable as xs:integer)
then ((xs:integer(substring(@Period, 6, 2)) - 1) idiv 3) * 3 + 1
else xs:double('NaN')

The requirements for explicit casting in XPath 2.0 can be ameliorated by taking two steps.

First, when designing markup languages, the values of nodes should have a format that corresponds to one of the well-supported XPath 2.0 types. If the Period attribute in the example above was in the format of an xs:date rather than an xs:gYearMonth, then the code:

((get-month-from-date(@Period) - 1) idiv 3) * 3 + 1

Second, XML documents should be validated so that nodes are annotated with their type. If the width and height attributes are annotated as having xs:integer values, for example, then the comparison:

@width > @height

But these are steps that few XSLT authors will be able to take. XSLT authors generally have no control over the format of the XML that they deal with, which means that string parsing of element and attribute values is common place. Getting typed data is even rarer, especially given that there is no requirement for XSLT 2.0 processors to support the annotation of nodes, even with just the built-in types.

To conclude, the strong typing of XPath 2.0 offers few benefits, and a fair burden, to those XSLT authors working with untyped data.

The additional date/time and duration types make date/time arithmetic a lot easier to carry out, and the xs:QName type might also prove useful in some transformations, but as a whole the type hierarchy contains a great many more types than are required or actively supported with XPath 2.0 functions. The size of the type hierarchy is daunting for new users and creates a barrier to learning XPath 2.0.

The greatest burden on XSLT authors, especially those used to the weak typing of XPath 1.0, comes in the form of the requirement for explicit casting. The implicit casting of untyped nodes to a required type is a great help but does not address common scenarios in which explicit casting is required. In particular, there is no implicit casting of strings (which are often generated following string manipulation of a node) to a required type or of values to strings. And when untyped values are used with arithmetic and comparison operators, the results are sometimes not what an author might expect.

It remains to be seen whether these features of XPath 2.0 prove to limit its uptake amongst authors dealing with untyped data, or are simply irritations that XSLT 2.0 authors will learn to live with.