All About XML

XML Spy

2011-12-16T15:53:00.000Z

XML Spy is described by the company developing it as a an advanced XML editor for modeling, editing, transforming, and debugging XML-related technologies." From this description, it is obvious that use case scenarios for this program are numerous and it would be impossible to provide an in-depth review without spending a lot of time using it, while having at least some degree of familiarity with the technologies it works with.

This review will focus on XML Spy's capacity of aiding XML document authoring. Lately, I accumulated quite a few XML files, due to the fact that I'm writing my blog entries for this module as XML files, which I then use an XSLT template to transform into HTML. Then I post the HTML code to my blog, and paste it into the Word document which is my report. These files will provide sufficient material to compare XML Spy with my current setup - the GVim editor, with a number of plugins to aid with XML editing, and the XML Star command-line utitlies which I use from within GVim to perform tasks such as validating and automatic escaping of characters like < and &.

XML Spy is not free software; to evaluate the program, one is required to give his email address to receive a key used to actvate the program. After receiving the confirming email, I proceded do download the software. The first thing I found out about it during this process is that it is not lightweight; the installation package weights in at a hefty 55MB; installed, it takes up 250MB and during installation requires at least double that amount, for storing temporary files.

Ather going through the throes of installing this beast, I fired it up, eager to put it to the test. The first impression I got was that the interface was bloated and outdated. Many of the buttons are too small, and the editor has a lot of windows opened by default. Fortunatelly, they can be closed - but the size of the buttons cannot be increased, which makes them a little bit harder to click. These are small greivances, but they do contribute signifficantly to the overall impression the program makes.

The first thing I did after taking a close look at the user interface and menu options, was opening a couple of my XML documents. I opened the combine.xsl and one of the posts for the blog, post_35.xml. After browsing the contents of the files and making small changes for a while, to get an impression of how the editor handles such tasks, I was under the impression that the editor is too wastefull with screen space, and has too many little details which make it harder to concentrate on the code.

I was curious to see how does XML Spy treat illegal characters in XML files - in the editor I use, they are simply hilighted in red. But when I tried to type illegal characters in XML Spy, it simply ignored the keystrokes. I think that is a bad way of handling such issues - even though the character is illegal, the editor should not make decisions for the user, without asking for permission, or even notifying the user. I tried typing such characters later, and the editor accepted the keystrokes, I'm not sure why. When typing an amersand, the editor will show up a pop-up menu, whith the available entities - such as & - but if the user keeps typing, the menu will not update itselft to reflect the new keystrokes.

XML Spy does offer convenient ways of checking for validity and well-formedness, from the XML menu; it also has keyboard shortcuts for these probably quite common operations - to check for well-formedness, all the user needs to do is press F7. For validity testing, the keyboard shortcut is F8.

Error messages XML Spy produces when incorrect syntax is encountered can be confusing. For example, after removing the > character which closed a tag, I pressed F7. XML Spy came up with the followin error:

Although the message is correct, it could be more succint; for example, here is the output generated when validating the same document with the xmlstar command line utility:

What I did like about XML Spy, however, is that it automatically checked the xsl file for validity after I fixed the error, and saved it. It wasn't expecting that, but it was helpfull since I actually forgot to check for validity/well-formedness before saving. However, for my other open document - the post_35.xml - when I made some small changes and saved it, it did not perform the validity test; XML Spy does not check the validity of documents which do not declare a DTD/Schema. But it could have performed a well-formedness test. However, it did not do it automatically.

XML Spy has very user-friendly XPath tools. The "XML -> Evaluate XPaht..." menu option brings up the XPath window, shown below. XPath expressions can be written in the textbox, and the results will be printed in the window below. This window is very convenient, and also features auto-completion of both elements in the document, as well as XPath functions. For example, the value of the'title' element can be accessed like this:

Pressing "Enter" will result in the value of the title being printed in the "Output" window - in this case, ".doc versus .docx". Clicking the result will result in the window moving to the corresponding element, and hilighting it. Another powerfull feature is that XML Spy allows users to select the version of XPath used, and supports both XPath 1.0 and XPath 2.0.

Although XML Spy is quite heavy, not portable ( only available on Windows ) and not free, it is also provides a very capable XML editing environment. However, students such as myself are probably not the target audience of this package; it's costly, and complex. Freely available tools do a good enough job for basic XML authoring scenarios, while IDE's such as XML Spy can provide the editing power when it is really needed.

.doc versus .docx

2011-12-16T15:50:00.001Z

Ever since Microsoft Office 2007 was launched by Microsoft, a new headache began to plague office workers all around the world. The culprit was the new default format used by Microsoft's office suite, .docx. This format is intended to supersede the .doc format, which was perhaps perceived as obsolete by Microsoft.

The transition could have been smoother, if not for the fact that Microsoft Office versions older than 2007 cannot open the new format in their default configuration. It is possible for Office 2003 users to open .docx documents, after installing the Compatibility Kit, but they will be warned that the document might not be displayed as it would in later Office versions, and that some elements not supported by the older Word might not be present. This can lead to users making changes to a document, only to find out later that the document looks different, or has elements missing, when viewed with a different version of the Office suite.

I decided to perform a dissection of a .docx file, and get a glimpse at it's internal workings. I had a .docx, which was a BBC article I downloaded and converted into a .docx file for printing as part of an assignment. I changed it's extension to .zip, and indeed, the archive contains a number of folders, and a [Content_Types].xml file at it's root.

The [Content_Types].xml describes the contents of the other .xml files, scattered throughout the folder hierarchy. For the 'xml' extension, by default the ContentType is set to "application/xml", but certain files override this setting via Override elements. For example, "/word/document.xml" has the "application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml" content type.

The _rels folder for this document contains a single file, .rels - which doesn't have an xml extension, but is xml because it has the xml declaration as the first line, and is a well-formed xml document. It's root element is Relationships, which contains a number of Relationship elements, each having the Id, Type and Target attributes. I couldn't exactly figure out how are these used.

The docProps folder is a bit more interesting; it has two documents inside it, core.xml and app.xml. The core.xml is quite simple, and it is easy to see that it contains various meta-data about the document ( coreProperties, as the root element is named ).

I found out a number of interesting things - for example, the document had a dc:creator element ( dc is the namespace alias for "http://purl.org/dc/elements/1.1/" ), which has the value of my Middlesex University ID used for logging in to computers on the campus; I did not know Word automatically stores this information inside the file itself. I also noted that this file stores the dates the file was created and modified - therefore, .docx documents store this information independent of the file system. The file also contains information about who last modified the file.

The app.xml file is a bit of a mixed bag - I was expecting it to contain information about the version of Microsoft Word the .docx document was created with - which it did, inside the AppVersion element - but it also contains metadata about the document's contents; such as the number of paragraphs, pages, words and characters. In addition, it has a CompanyName element, which was bestowed the value 'Middlesex University'.

The documents inside the word folder contain the document data. The roles of most of the files inside it are not hard to guess; for example, the fontTable.xml file stores information about each of the fonts used in the document. The settings.xml file contains settings such as the zoom level set when the document was last edited, and the decimal symbol used. Images and other media will be stored inside the 'media' folder; in this case, it contained an image named image1.gif. This hierarchy resembles an XHTML webpage - the document written in a markup language, accompanied by resources ( such as images ) and stylesheets.

The document.xml file is at the core of the .docx folder hierarchy, containing the text and layout information. All the other .xml files are used to describe certain aspects of this file, and to support Word in deciding how to process or display it.

I then saved the file as a simple .doc; after which I renamed it to .zip and tried to open it, but the archiving software gave me an error. I then opened the document with a hex editor ( 010 Editor ), and the reason became clear: the .doc file is a binary format. Some of the text was readable while browsing the hex data, but most of the file was difficult to make sense of.

Is AJAX both a success and a failure?

2011-12-16T15:48:00.002Z

Since it's emergence as a mainstream technology, AJAX has proven it's usefulness. AJAX technologies are now employed by companies such as Microsoft and Google, and are used by millions of web pages.

AJAX does, however have drawbacks which take away from it's shine. One of the main concerns with AJAX technologies is with it's impact on usability - certain screen readers and other accessibility devices might not respond well to dynamically updated web pages.

Important issues emerge when a fast internet connection is not available. This can lead to websites being very difficult to use, because AJAX is often used for functionality which requires responsiveness - such as responding to the keys the user types in a search box, by updating a list of matching items. On a slow internet connection, this might lead to serious usability issues.

Another problem can arise due to the fact that AJAX relies on JavaScript. But many users don't like JavaScript, and all modern browsers provide some mechanism of preventing JavaScript code from executing. If the user has JavaScript disabled, the website will not be dynamically updated. Although improbable, it is also possible that some users visit the web page with browsers which do not support JavaScript - in which case, again, dynamic content will not be updated.

On the server side, employing AJAX technologies can lead to very high number of requests to serve, which might be difficult to cope for the servers. But companies usually take this into account, and purchase hardware which can cope with the increased load; this is rarely an issue.

Is there something wrong with XML ?

2011-12-14T13:29:00.000Z

It seems somewhat inappropriate to state that there is something wrong with XML; it is, after all, just a technology designed for simple purposes: to provide an extensible markup language, which could be used on it's own, or be used to create other, specialized markup languages. The burgeoning scene of XML-based languages is a testament to XML's success, and therefore it fulfilled its purpose.

But that is not to say that XML doesn't come without drawbacks, though. One of the most important issues with XML pertains to it's verbosity - XML files can become quite large. This is inherent to XML, and is one of the trade-offs consciously made by it's designers; integrity was deemed more important than file size. This issue can be mitigated to some degree by using compression - an XML file is, after all, just text so it lends itself to compression algorithms. However, this negates one of XML's advantages, since if the archive is damaged it would hardly be possible to salvage any of the data.

Another issue consists of the relative complexity of an XML parser, as compared to alternatives available under certain conditions. For example, XML is often being used for storing configuration settings for applications. The application then has to include an XML parser, which could significantly increase the size of the application. This is less of an issue with large applications, such as Visual Studio or Microsoft Office, but simpler means of storing configuration files - such as .ini or .conf files - are available for smaller applications.

XML files can also be cumbersome to author without a specialized editor, again due to the explicit nature of XML. This not a concern when small changes need to be made - for example, to change the resolution for a game in it's config.xml. But authoring can be an extremely tedious process when large chunks of XML need to be added, or when a document is created from scratch. This problem is compounded by the fact that XML documents should be well-formed, and in many cases valid; and small mistakes can render a document unacceptable to parsers due to well-formedness or validity issues. This issue can make XML documents 'read-only' for humans.

XML Security

2011-12-14T13:27:00.000Z

XML is becoming increasingly common, and therefore security issues are also having an increased importance. Given that XML is frequently used in e-commerce, businesses need ways to ensure that stored data is secure, and prevent unauthorized access. This is especially important due to the textual nature of XML - a simple text editor can be used to view the contents and extract sensitive data.

A number of security-related technologies have emerged, one of the most important ones being XACML - the eXtensible Access Control Markup Language. XACML allows for controlling access to information via rules and policies. Other significant initiatives are XML signature and XML encryption.

The concept of digital signatures is not new, and mature standards have been in place for quite some time. But existing technologies only allow signing individual files. Given the hierarchical nature of XML, more granularity is highly desirable - in other words, the ability to sign portions of an XML document. This is the role fulfilled by XML signature. The high-level algorithm is quite simple - the element to be signed is hashed, the hash being stored inside a DigestValue element. Another digest ( usually a hash ) is produced from this element and is cryptographically signed. The XML signature is then inserted in the signature element.

XML encryption operates on similar principles to XML signatures - it allows the encryption of XML documents with a high granularity, enabling the encryption of portions of a document. An additional advantage is that different encryption keys and algorithms can be used for each encrypted portion, allowing precise control over who can use which portions of the document.

XML encryption is especially important in a business environment. For example, a delivery company might define an XML document type which contains one element which contains information about the client, and billing information, and one element containing information about the contents of the delivered package. The driver will have the key for viewing the client information, while the client might be sent via email the key for him to be able to view information about the content and check if anything is missing.

Is the semantic web a failure ?

2011-12-14T13:25:00.001Z

Although counter-intuitive at first sight, the concept of a failed semantic web takes shape at a closer examination. But first, a definition of the semantic web is in order; as it is accepted now, the semantic web is the next evolutionary step from 'Web 1.0', in which web pages generally lack semantic meaning and web pages are written in HTML which mixes presentation with content.

The advent of technologies such as XML and the XML version of HTML, XHTML were promising, since the line between content and presentation was clearly drawn. However, the fact that the semantic web was possible and desirable did not necessarily translate into it becoming a reality - there were still billions of web pages written in older versions of HTML, which prevented browsers from adopting a tougher stance towards badly written HTML code.

This pressure to assure backwards compatibility resulted in the continuing proliferation of web pages which do not respect the tenets of the semantic web, since there is little pressure to do so - the browsers are still forgiving. Even with pages which have the XHTML DTD, browsers will be less strict since many of them are not well-formed XML documents.

In conclusion, the semantic web remains an idea, a concept, rather than an accomplished fact. The realities of the web make it nearly impossible to achieve in practice, but nevertheless a useful concept which encourages best practices.

XUL - what it is, and how is it used.

2011-12-14T13:15:00.000Z

XUL - XML User Interface Language, pronounced "Zool" is a framework employed by Mozilla in it's applications, such as the Firefox web browser. It is based on XML while also making use of CSS and HTML features.

XUL is used in the context of Mozilla applications, which consist of components like buttons, labels and text boxes - "widgets". The rendering and handling of user actions within the widget are under the control of the widget itself, while the layout of the widgets and their interactions is specified in the XUL document.

XUL makes use of numerous other technologies, such as JavaScript - which can be used to program individual widgets, XBL - the eXtensible Bindings Language - used to define "bindings" for widgets, which also allows for event handlers, and adding new interface elements and properties.

Another technology which can be employed in the context of XUL is XPConnect and XPCOM. XPCOM ( Cross Platform Component Object Model ) is a framework for writing cross-platform, modular software components in native languages, such as C or C++. XPCOM components can be used form within C, C++, JavaScript, Python, Java and Perl. XPConnect enables such XPCOM objects to be accessed from JavaScript.

It is similar, in some aspects, to XIML since it also aims to provide a way of describing user interfaces, and is based on XML. But there are many differences, one of the most significant ones being that XIML has a broader scope, aiming for a generalization and standardization of UI elements and interactions, while XUL has a more practical perspective, providing a working solution at the expense of standardization, and having a narrower scope.

XIML - what is it, and how it is used.

2011-12-14T13:08:00.001Z

XIML ( eXtensible Interface Markup Language ) is an XML-based markup language which aims to provide a standard way of representing interaction data - data pertaining to the user interface. XIML allows for the representation and manipulation of such data, which defines and relates all the relevant elements of a user interface.

XIML aims to provide comprehensive lifecycle support - in other words, the same XIML file could be used during the design of the user interface, as well as at runtime and during the evaluation phases. Tools used for GUI ( Graphical User Interface ) design could output XIML files, which will be used by the application at runtime. The same XIML files could then be used for usability engineering.

Designing a language such as XIML is also complicated by the fact that it needs to be able to represent both abstract, and concrete elements of a user interface. One of the abstract elements consists of the context in which the respective interaction takes place, and the particular task that is accomplished. Concrete elements are much easier to envision, consisting of the various controls or widgets which are to be displayed by the application.

XIML allows for the definition of relations between the various elements which can be represented. For example, XIML could be used to state that "Type A objects will be displayed with element E", where E is a presentational element, while A is a domain-component.

What is the value of XML in e-commerce?

2011-12-09T14:49:00.003Z

It is very common today to see XML-based technologies employed, for various purposes, in e-commerce. It’s popularity is increasing due to the advantages XML offers over other traditional database technologies.

One of the main reasons for increasing XML adoption consists of the fact that XML is more future-proof – data stored in XML does not necessarily depend on a specific application to enable users to make use of it. If a company stores it’s data in a SQL database, for example, retrieving the data a few years from now might not be a simple task – the format supported by SQL database applications might change, or the particular SQL application used by the company might not be available anymore due to bankruptcy or other reasons. This is an unlikely scenario, but nevertheless possible; XML avoids these potential issues because it’s data is stored as text, which can be searched; therefore, a simple text editor can be used to retrieve data from an XML document, or edit it.

Additionally, there are many XML processors available, which are relatively easy to adapt for a particular XML type of document. This is a direct result of XML being extensible by definition.

Another benefit resulting from XML’s textual nature is that corrupt data is likely to be more salvageable than a database file; a few corrupt bytes will not render the XML file useless – most of the data can still be accessed – this is likely to be an important issue for companies.

XML is also being increasingly used for communications between applications, services and web-servers, via the XML-RPC ( Remote Procedure Call ) protocol.

Parsing XML using SAX

2011-11-30T09:52:00.003Z

SAX, the Simple API for XML, is another take on parsing XML files. But, as opposed to the DOM approach, a SAX parser will not re-construct the entire document tree in memory, but instead operates in an event-driven manner. The SAX API allows clients to register callback functions for the items they are interested in ( which can be text nodes, element nodes, processing instructions or comments ). When the SAX parser will encounter these items, an event will be triggered and the callback function ( if any are registered ) will be called, allowing the client application to take action. Another event will be triggered when an end of any of those XML features is encountered, and eventual functions called.

Another difference between SAX and DOM is that there is no standard for SAX, and therefore the API is not guaranteed to be the same for all parsers. In practice, differences between various implementations are quite significant, requiring users to familiarize themselves with the particular flavor of SAX which the parser employs.

For example, a SAX parser is described by Microsoft’s ISAXXMLReader (http://msdn.microsoft.com/en-us/library/aa924174.aspx ); callback functions can be set using the ISAXContentHandler::startElement method. But fortunately other implementations outside COM ugliness are available for C++; one of the most popular choices is Xerces (http://xerces.apache.org/index.html ), by Apache. It can be used like this:

void foobar_sax_handler::startElement(const XMLCh* const name,

AttributeList& attributes)

{

char* m_name = XMLString::transcode(name);

cout << "Element encountered: "<< m_name << endl;

XMLString::release(&m_name);

}

This assumes that we have defined a class named foobar_sax_handler, which inherits from HandlerBase and has a startElement method. We can set this class to handle XML events with the setDocumentHandler method of the SaxParser class. Our customized parser can then be invoked using the parse( xml_file ) method of the SaxParser class, where xml_file represents the name of an XML file.

When the parse method is called, the startElement method will be called for each element, printing it’s name. Of course, inside this method we can perform more specific actions, for example only printing or modifying certain elements.

SAX bindings are also available for other languages as well, for example the jssaxparser ( http://code.google.com/p/jssaxparser/ ) is for JavaScript; a SAX parser is also available for Java ( javax.xml.parsers.SAXParser ). The API varies slightly between all these implementations, but the same principles apply; if one is familiar with using SAX with one programming language, switching to a different one will be relatively easy.

The Document Object Model

2011-11-29T21:13:00.000Z

The DOM is an interface which provides a way of accessing and modifying structured documents, such as XML, XHTM and SVG. The DOM is usually accessed via it’s public API, in a platform- and operating system-independent manner.

The DOM has a troubled history, due to the fact that the early browsers market was very competitive, which reflected negatively on the willingness of the companies producing them to cooperate, develop and implement standards. W3C later managed to get companies such as Netscape and Microsoft to collaborate and develop a standard for a scripting language ( ECMAScript ), and a DOM afterwards, in late 1998.

The concept of a DOM is used by browsers, which expose an API to enable JavaScript code to access and modify the DOM. For example, the following code will find an element which has the ‘foo’ id and make it’s background colour green:

document.getElementById('foo').backgroundColor="green";

This example illustrates a few important concepts about the DOM – instead of being a monolithic specification, it is divided into several separate documents, each describing a specific area. For Level 2 DOM, it comprises a ‘Core’ ( http://www.w3.org/TR/2000/REC-DOM-Level-2-Core-20001113/ ), and the other recommendations extend on it. For example, the Level 2 HTML DOM extends some components which are part of the ‘Core’ specification, specializing them according to the needs of HTML. The HTML DOM ‘HTMLDocument’ interface, for example ( http://www.w3.org/TR/2003/REC-DOM-Level-2-HTML-20030109/html.html#ID-26809268 ) was derived from the core ‘Document’ interface, and similarly the HTMLElement interface was derived from the core ‘Element’ interface.

Such extensions of the ‘Core’ DOM allow for a wider range of application, and therefore more uses. The aforementioned HTML DOM allows for HTM-specific manipulation – for example, hiding an element with a certain ID. The DOM Level 2 Style specification allows scripts to dynamically access and update the content of stylesheets, therefore affecting how the document looks.

SVG Images

2011-11-29T19:54:00.002Z

Scalable Vector Graphics ( SVG ) relies on text for describing an image. It also supports the inclusion of raster data, however the mainstay of SVG is textual description of graphic data. For example, the following SVG document contains a red circle with a black outline:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<svg

xmlns:dc="http://purl.org/dc/elements/1.1/"

xmlns:cc="http://creativecommons.org/ns#"

xmlns:svg="http://www.w3.org/2000/svg"

xmlns="http://www.w3.org/2000/svg"

version="1.1"

width="100"

height="100"

id="svg2">

transform="translate(0,-952.36218)"

id="layer1">

<path

d="m 90,50 a 40,40 0 1 1 -80,0 40,40 0 1 1 80,0 z"

transform="translate(0,952.36218)"

id="path2987"

style="fill:#ff0000;

fill-rule:evenodd;

stroke:#000000;

stroke-width:1px;

stroke-linecap:butt;

stroke-linejoin:miter;

stroke-opacity:1"

</g>

</svg>

As it is obvious from the document above, SVG is an application of XML – valid SVG documents are also validate as XML. It also showcases how graphical data is stored inside the SVG file – as XML elements, with graphical properties described via attributes. The circle is represented by the path element – the ‘d‘ element is actually short for ‘Path Data’, and describes it’s shape. ( http://www.w3.org/TR/SVG11/paths.html#PathData ). Other properties of the circle, such as it’s fill colour and stroke width/colour are described via attributes, as visible in the above example.

The SVG specification has multiple profiles; for example, the SVG Mobile Recommendation ( http://www.w3.org/TR/SVGMobile/ ) defines the SVG Tiny and SVG Basic mobile profiles. SVG Tiny is targeted at highly restricted mobile devices, while SVG Basic is meant to be suitable for higher level mobile devices.

One of the major issues concerning SVG is a reduced browser adoption rate. Although the most recent versions of all major browsers have at least some level of support for SVG images, some older, but still widely-used versions do not support SVG; Internet Explorer fully supports the SVG Basic specification only since version 9, released on March 14^th 2011.

XLink

2011-11-29T00:02:00.002Z

XLink is the XML way of tackling the issue of creating links between documents. It is very similar to it’s HTML counterpart, the <a> tag. But having a dedicated element for handling linking is not in XML’s spirit ( it would impose unnecessary restrictions that would hamper flexibility ), therefore this functionality is implemented in a separate namespace ( xmlns:xlink=http://www.w3.org/1999/xlink ), and attributed to elements with by setting certain properties.

It can be used for the roles fulfilled by <a> in HTML – to enable the user to navigate to a different document; but they’re more versatile, and have a number of attributes that control it’s behaviour.

An XLink can be of two types: simple and extended. The simple XLink has similar similar capabilities to traditional HTML links: they provide a uni-directional connection between two resources.

Example of simple XLink usage:

1 <?xml version="1.0"?>

2 <links xmlns:xlink="http://www.w3.org/1999/xlink">

4 <blog xlink:type="simple"

5 xlink:href="http://mihairotaru.blogspot.com">Ram's blog</blog>

7 <blog xlink:type="simple"

8 xlink:href="http://native-dev.blogspot.com">Second blog</blog>

10 </links>

Transforming XML documents with XSLT

2011-11-28T09:06:00.003Z

The XSLT transformation pipeline involves four elements: the source XML document(s) with the XSLT stylesheets, an XSLT processor ( template processing engine ), and the resulting document(s).

The output format of an XSLT transformation ( actually, in the context of XSLT a transformation doesn’t actually transform the source XML document – but instead uses it as input for creating other documents ) can range from PDF files to plain text files; this is due to XSLT’s powerful templating mechanism and versatile XSL Formatting Objects.

Here’s an XSL template which will create an XML file based on movies.xml, and add a column which will represent the ‘value’ of the movie – it’s rating divided by it’s price:

1 <?xml version="1.0" encoding="ISO-8859-1"?>

2 <xsl:stylesheet version="1.0"

3 xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

5 <xsl:output method="xml" version="1.0" indent="yes"/>

6 <xsl:template match="/">

7 <movies>

8 <xsl:for-each select="movies/movie">

9 <movie>

10 <xsl:copy-of select="./*"/>

11 <value><xsl:value-of select="rating div price"/></value>

12 </movie>

13 </xsl:for-each>

14 </movies>

15 </xsl:template>

16 </xsl:stylesheet>

Notes

- this solution is not ideal, since some elements are hard-coded ( ‘movies’ and ‘movie’ ); these should be somehow deduced

- xsl:for-each is used on L08 to select each ‘movie’ node in turn ( in the order they appear in the original XML file – ‘document order’ )

- xsl:copy-of is used on L10 to create a copies of all the child nodes of the current node. This statement will copy the ‘title’, ‘year’, etc elements for each ‘movie’ element.

- on L11, the value element is created. xsl:value-of is used to insert the value resulting from dividing the value of the ‘rating’ element with the value of the ‘price’ element of the current node.

Microsoft’s msxsl tool (http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=21714 ) can be used to transform the original XML file into a new one, with the added <value> element:

msxsl movies.xml transform.xsl -o new_movies.xml

Sorting is accomplished by using the xsl:sort element ( http://www.w3.org/TR/xslt#sorting ). xsl:sort can only appear as a child of an xsl:apply-templates or an xsl:for-each element. To sort the list of movies by the name of the title, we can simply add a line after L08 in the previous XSLT stylesheet:

9 <xsl:sort select="title">

The resulting XML document will have it’ elements sorted corresponding to the alphabetical order of their ‘title’ elements. So, ‘Apocalypto’ will appear the first, and ‘Year One’ as the last one.

Styling XML documents with XSL

2011-11-27T19:13:00.001Z

This blog post will describe the process of employing XSLT stylesheets for styling an XML document. It is very similar to using CSS – the stylesheet must be created, and the XML document should be told about the stylesheet. But that’s where the similarities stop; the two methods are very different in their approach.

Central to XSLT-based styling is the concept of a `template` - it is XSLT’s equivalent for CSS rules. XSLT will try to match the template’s pattern with the XML document; when it finds a match, it will perform the transformations described in the template body.

Here’s an XSL which would display the movies.xml file as a table:


<?xml version="1.0"?>
<xsl:stylesheet 
version="1.0" 
xmlns:xsl="http://www.w3.org/1999/XSL/Transform" >
<xsl:template match="/">
<html>
    <body style="font:12px Georgia;">
        <table style="border: 1px outset; background-color: lightgray;">
            <!--Generate table headings-->
            <tr bgcolor="#9acd32">
                <th>Title</th>
                <th>Year</th>
                <th>Rating</th>
            </tr>

            <!--Generate table rows-->
            <xsl:for-each select="movies/movie">
                <tr>
                    <td><xsl:value-of select="title"/></td>
                    <td><xsl:value-of select="year"/></td>
                    <td><xsl:value-of select="rating"/></td>
                </tr>
            </xsl:for-each>
        </table>
    </body>
</html>
</xsl:template>
</xsl:stylesheet>

The XML file needs to know about this XSL stylesheet; so this line needs to be added to the XML file, assuming the XSL has the ‘movies-xsl-style.xsl’:

2 <?xml-stylesheet type="text/xsl" href="movies-xsl-style.xsl"?>

This is how the XML file was rendered by Firefox 8.0:

Which is exactly how I wanted it to look. This example illustrates the advantages and potential drawbacks of using XSL fro styling XML documents.

Among the drawbacks, the most obvious one is the learning curve – although XSL stylesheets are XML documents themselves, they have numerous tags with a specific role ( more precisely, those inside the http://www.w3.org/1999/XSL/Transform namespace ), which means that it is still a technology that one would need to spend considerable amounts of time to learn. It can be considered a fully-fledged programming language, with loops, if’s and variables.

Browser support is also quite limited for XSLT 2.0; however, this doesn’t diminish it’s usefulness on the server side. Instead of serving the XML + XSL documents and relying on the client’s browser to perform the transformations, the transformations could be run on the server ( using tools such as Saxon or AltovaXML ), generating XHTML files which would then be served to the clients.

One of the most important advantages of using XSL to style XML documents is the fact that it respects the principle of keeping the data and presentation separate, while providing the same styling capabilities as with CSS stylesheets, and much more powerful constructs.

Styling XML documents with CSS

2011-11-26T22:42:00.000Z

XML can be styled using CSS in a similar way to (X)HTML; in fact, the same CSS file could be used by both XML and XHTML files. In this post, I will demonstrate, using the XML file ( Pastebin link ) created in the previous post, how to add CSS styling to an XML document. This blog post is based on the short tutorial in section 2.2 of the CSS2.1 W3C Recommendation ( http://www.w3.org/TR/CSS2/intro.html#xml-tutorial ).

Before telling the XML file about the CSS, obviously the CSS file needs to be created. I wanted the table to have a gray background, a serif font, and green background for the table headers. I ended up creating this short CSS file:

1 body

2 {

3 font: 12px Georgia;

4 }

6 table

7 {

8 border-width: 1px;

9 border-style: outset;

10 background-color: lightgray;

11 }

13 th

14 {

15 background-color: #9acd32;

16 }

Then, I simply added a line to the XML file:

2 <?xml-stylesheet type="text/css" href="style.css"?>

Not a very smart approach, as I soon realized. Instead of rendering a lovely table, the browser simply printed all the information about the movies, in a single row. Which makes perfect sense; the CSS states that table headers should have a green background. But how is the browser supposed to know which part of the XML file should be the headers ? Or where is the table ? All it sees is a well-formed XML file.

After massaging the CSS file further – while not touching the XML, in keeping with the ‘separate data from style’ principle – I managed to get to something resembling a table:

1 movies

2 {

3 font: 12px Georgia;

4 margin: 3px;

5 width: 260px;

6 border-width: 1px;

7 border-style: outset;

8 background-color: lightgray;

9 }

11 title { background-color: lightcyan; }

13 movie { display: block; background-color: honeydew }

15 title { display: inline-block; width: 100px; border-right: 2px solid gray; }

16 rating { display: inline-block; width: 50px; border-right: 1px solid gray; }

17 price { display: inline-block; width: 50px; border-right: 1px solid gray; }

18 year { display: inline-block; width: 40px; }

How the XML was rendered by Firefox 8.0:

Looks like a table, sure, but my dreams of green table header backgrounds were shattered; it cannot be accomplished without modifying the XML file. And I really don’t want to do that – because then I would be using the XML file as (X)HTML. Obviously, XHTML is XML by definition; but it’s not wrong – in fact, it is required – for XHTML to have tags like ‘body’, ‘table’, ‘td’, ‘tr’, etc.

XML, on the other hand, I intend to use it for it’s intended purpose – storing data in a future-proof way. What if I later decide to write another parser that would use my XML files with data about movies ? Had I decided to change the XML to accommodate CSS styling, my parser would have to look for and parse elements such as ‘body’, ‘h1’, etc – and I would much rather have to deal with ‘movie’ elements, which have the ‘title’, ‘year’, etc elements. In other words, the semantics of the XML document would be preserved, without being contaminated with presentational tags – or simply tags that don’t make sense in the context of the data which is stored.

I think this highlights very well the problems with styling XML with CSS – while it can work, the desired effects might be difficult to achieve without contaminating the XML document with additional data. This is not desirable; the presentational document should accommodate the data, not the other way around. While attempting this, the CSS file might end up bloated, and using unintuitive work-arounds.

However, styling XML documents with CSS might work very well for simpler document types, which do not require the control that my example did. For example, if the document only contains a few textual elements, then basic presentational properties can be set easily – font, colors, etc.

But probably the most enticing feature of CSS over the alternative ( XSL ) is the fact that it leverages knowledge of a widespread, almost ubiquitous technology, since CSS has been used extensively for decades. It has a very large user base, and these programmers would be able to style XML documents using CSS with little learning required. XSL, on the other hand, can be quite daunting to learn.

Ultimately, I think a decision should be taken on a per-case basis. In simple cases, CSS works just fine; but when XML documents have a more complex structure, the power of XSL should probably be invoked instead.

How to generate XHTML from XML using XSL

2011-11-26T20:34:00.000Z

The Extensible Stylesheet Language Family (XSL) can be used to transform XML documents into other types of XML documents ( such as XHTML ), or into completely different formats – such as PDF, or simple plain text.

One of the most common applications of XSL is to present documents stored inside XML files as XHTML, for consumption by browsers. The principle behind XSL is simple: take a well-formed XML document, find the needed elements/nodes/text ( using XPath ), and place them in a template. Assuming a valid XSL template and a well-formed XML file were used, the resulting document will also be a valid XHTML document.

Taking the document used in the previous blog post, which contains information about a few movies, I will assume the goal is to generate the XHTML code which will present the information inside the file.

To accomplish this, an XSL Style Sheet must be created. The name is slightly misleading, because XSL can be used not only to configure and customize how an XML document is presented, but transformations can be applied as well. This will not, of course, alter the XML document, but might not represent all the data inside it – some information can be filtered out, new information might be added by the XSL template.

I have come up with the following XSL file:

1 <?xml version="1.0"?>
2 <xsl:stylesheet
3     version="1.0"
4     xmlns:xsl="http://www.w3.org/1999/XSL/Transform" >
5     <xsl:template match="/">
6         
7         <html>
8             <body>
9                 <h2>A list of movies</h2>
10                 <p>This list comprises several movies</p>
11                 <table>

12
13                     
14                     <tr>
15                         <th>Title</th>
16                         <th>Year</th>
17                         <th>Rating</th>
18                     </tr>
19
20                     
21                     <xsl:for-each select="movies/movie">
22                         <tr>
23                             <td><xsl:value-of     select="title"/></td>
24                             <td><xsl:value-of select="year"/></td>
25                             <td><xsl:value-of select="rating"/></td>
26                         </tr>
27                     </xsl:for-each>
28                 </table>
29             </body>
30         </html>
31         
32     </xsl:template>
33 </xsl:stylesheet>

A few things to note:

- I intentionally omitted the price information, to underline how XSL might create the wrong impression of the information contained in the XML file. This information is available inside the XML, but the XSLT omits it.

1. XPath is used on L05, to define the elements to which the template should be applied. In this case, the ‘/’ means that it should be applied to the whole document. Again, very similar to paths in Unix – ‘/’ represents the root of the filesystem.

2. XPath is used again on L21; the statements contained by the for-each element will be executed for each node matching the XPath expression “movies/movie”.

3. Another important XSL element, the xsl:value-of is used on lines 23-25; it relies on the concept of the “current element”. In the context of a for-each, this means that each element matching for-each’s XPath ( in this case, "movies/movie" ) will become the “current element”, in sequence. We have five movie elements in our XML file, so each of these will become, in turn, the current element. xsl:value-of will extract the value of certain child elements - title, year and rating, and place these values instead of itself.

One more step is needed – we need to tell the XML file that we made an XSL stylesheet for it – otherwise, when opened in a browser, it will simply be displayed as before. To do this, a line needs to be added to the XML file:

2 <?xml-stylesheet type="text/xsl" href="movies-style.xsl"?>

When opened in an XSL-compliant browser ( all modern browsers are compatible with XSL ), the XML file will be processed using our XSL file ( movies-style.xsl ), and the result of this process is XHTML – which will then be displayed by the browser.

XPath

2011-11-26T17:24:00.000Z

XPath is a syntax used for addressing certain parts of an XML document. It can also be used to test whether a node matches a pattern ( the pattern being the XPath ) – this property is used by XSLT.

It is difficult to point out strengths and weaknesses, since XPath is a relatively simple technology, designed with a very specific purpose in mind. Over the years since it was introduced, XPath has proven it’s value, and is a very important part of XML. The most common usage is in the context of XSLT, to select nodes onto which to apply formatting. It is obvious that some mechanism of selecting nodes is necessary in this context, not simply desirable; and XPath is the natural consequence of this requirement.

Assuming this well-formed XML document:

1 <?xml version="1.0"?>

2 <movies>

3 <movie >

4 <title>Hunger</title>

5 <rating>7.6</rating>

6 <price>15.00</price>

7 <year>2008</year>

8 </movie>

9 <movie>

10 <title>Year One</title>

11 <rating>5.0</rating>

12 <price>12.00</price>

13 <year>2011</year>

14 </movie>

15 <movie>

16 <title>Pulp Fiction</title>

17 <rating>9.0</rating>

18 <price>10.00</price>

19 <year>1994</year>

20 </movie>

21 <movie>

22 <title>The Godfather</title>

23 <rating>9.2</rating>

24 <price>15.00</price>

25 <year>1972</year>

26 </movie>

27 <movie>

28 <title>Apocalypto</title>

29 <rating>7.8</rating>

30 <price>17.00</price>

31 <year>2006</year>

32 </movie>

33 </movies>

XPath can be used to select a particular element. For example, to view the titles of all the movie elements, the following XPath expression can be used:

movies/movie/title

To view only the tiles of movies released after year 2000:

/movies/movie[year > '2000']

To view the first two books:

/movies/movie[position() < '3']

Some of the inconveniences of XPath include the fact that it has it’s own syntax that one needs to learn; it’s a language within a language. XPath includes quite a number of functions ( http://www.w3.org/TR/xpath/#corelib ), which means that proficiency is not easy to achieve, and can become very difficult to read. Here’s an example from the XPath specification:

/child::doc/child::chapter[position()=5]/child::section[position()=2]

It selects the second section of the fifth chapter of the doc document element.

On the other hand, the syntax is purposefully designed to be similar to the way file and folder paths are described on Unix systems, and having some familiarity with the Unix command line can ease the process of becoming comfortable with XPath. For example, the ‘.’ has a conceptually similar significance – in Unix, it signifies the current directory, while in XPath the current element. In a similar vein, ‘..’ can be used on the Unix command line as a shortcut for the parent of the current folder, while in XPath it means the parent of the current element. Hierarchy is described using the slash in both cases.

XML namespaces

2011-11-25T23:57:00.000Z

Namespaces, a familiar concept to C++ and Java programmers, have a similar role in XML – to prevent name clashes. The problem of name clashes has the potential to be more acute with XML due to it’s versatility and flexibility. XML has a wider and more difficult to anticipate application domain, which makes namespaces a very important component to ensure a way of mitigating potential name conflicts.

For example, a file containing information about a module taught at a university might have the following structure:

1 <?xml version="1.0"?>

2 <module>

3 <ID>CMT3315</ID>

4 <tutor>Ray Adams</tutor>

5 </module>

And another file could contain information about a modules in a software system:

1 <?xml version="1.0"?>

2 <module>

3 <name>Graphical User Interface</name>

4 <priority>high</priority>

5 </module>

These documents could be included in the same XML document, via external entities – which would lead to two elements having the same name, but at the same time being completely unrelated. The name of XML elements is described as the element’s ‘type’ in the XML standard (http://www.w3.org/TR/REC-xml/#sec-starttags ), and all elements at the same hierarchical level are likely to be assumed as belonging to the same type.

Namespaces provide a way of differentiating between elements with the same name, but which are unrelated. The document containing both the above elements would end up looking like this:

1 <?xml version="1.0"?>

2 <root xmlns:t="teaching" xmlns:s="software_systems">

3 <t:module>

4 <ID>CMT3315</ID>

5 <tutor>Ray Adams</tutor>

6 </t:module>

7 <s:module>

8 <name>Graphical User Interface</name>

9 <priority>high</priority>

10 </s:module>

11 </root>

This is a well-formed document, and there are no ambiguities since the two different types of module elements are placed in different namespaces, therefore avoiding name conflicts.

Data modelling with XML

2011-11-25T12:03:00.000Z

Needs work:

Expand example ( w. code ), ID-REFS, XLink

http://www.ibm.com/developerworks/xml/library/x-xdm2m.html

Logical data modeling usually occurs largely at a high level, outside the constraints of particular technologies, such as XML or SQL. However, since the resulting data model will ultimately be implemented in one ( or more ) of the available technologies, the logical data model might be adjusted for the particular computing system on which the physical data model is to be implemented – in this case, XML.

A logical data model consists of three main components: entities, attributes, and relationships (http://web.archive.org/web/20080509063521/http://www.dbmsmag.com/9506d16.html ). XML provides a robust platform for implementing entities and attributes, but relationships can be more difficult to express ( http://www.tdan.com/view-articles/5538 ).

The hierarchical structure of XML can be used to convey basic relationships, such as one-to-one and one-to-many; for example, a document called ‘Module’ can contain one or more ‘Tutor’ elements. This can be thought of as expressing either an one-to-one relationship ( if the module only has one tutor ), or as one-to-many, if the module contains multiple ‘Tutor’ elements.

Encodings in XML

2011-11-24T23:13:00.006Z

The character set used throughout an XML document can be declared as part of the first line of the document, inside the xml declaration:

1 <?xml version="1.0"? encoding="UTF-8"?>

If no encoding is declared, parsers will generally try to guess which encoding is used, first by looking at the first bytes of the file. If characters outside the assumed encoding are detected, the parser will try to find an alternative encoding that will contain it. If the characters are not recognized, the parser should stop processing (http://www.w3.org/TR/REC-xml/#charencoding ).

XML encodings can be divided into three categories, the most common encodings being Unicode/ISO/IEC10646 encodings and transformations: "UTF-8", "UTF-16", "ISO-10646-UCS-2", and "ISO-10646-UCS-4". UTF-8 is the most used encoding (http://w3techs.com/technologies/overview/character_encoding/all ), accounting for 67% of websites which use a known character encoding.

The main advantage of UTF-8 is that it can encode any Unicode character, and any valid ASCII text file is also a valid UTF-8 document. This is due to the fact that UTF-8 characters do not have a fixed width ( in bytes ), but can occupy from one to six bytes; and since it was designed with backwards-compatibility with ASCII, the first 127 characters are the same for ASCII and UTF-8.

Among disadvantages, the major drawback of UTF-8 is that for certain languages, it will take up more space. Taking Romanian as an example – the word “ampulă” cannot be represented with ASCII because of the “ă” character ( http://www.fileformat.info/info/unicode/char/103/index.htm ); however, the ISO/IEC 8859-16, informally known as “Latin-10” or “South-Eastern European” encoding defines the character I need as 0xE3, so I would use this encoding for my document. ISO-8859-16 is a single-byte encoding, so it is very efficient and suited for situations when I don’t need other non-ASCII characters.

However, to represent “ă” using UTF-8, two bytes are needed: 0xC4 and 0x83, because it is decomposed into 'LATIN SMALL LETTER A' ( 0xC4 ) and 'COMBINING BREVE' ( 0x83 ), hence occupying double the amount of space. Therefore, if space were a concern, I would use the ISO-8859-16 encoding for Romanian texts.

However, it is unpractical to represent Chinese characters with any encodings which don’t support many characters. Single-byte encodings work for Romanian and other languages with a small number of characters, but not for languages which use ideograms, such as Chinese or Japanese. In this situation, UTF-8 becomes the better option.

XML Entities: possible issues

2011-11-24T18:02:00.002Z

XML entities can be used to store pre-defined text fragments, which can then be used throughout the XML document. The entities are replaced with the text they represent; such entities are known as ‘General Entities’.

Another type of entities are the ‘Parameter Entities’ – which are to be used only inside the DTD. It is important to distinguish between them because they occupy different namespaces, and are recognized only in their respective context. ( http://www.w3.org/TR/REC-xml/REC-xml-20081126.xml#sec-physical-struct )

Not distinguishing between the two types of entities can be a source of errors; for example, we might have the following XML document:

1 <?xml version="1.0"?>

2 <!DOCTYPE my_doc [

3 <!ENTITY % ParEntity "This is a parameter entity" >

4 <!ENTITY GeneralEntity "This is a general entity" >

5 <!ELEMENT my_doc (#PCDATA | item )*>

6 ]>

7 <my_doc>

8 &ParEntity;

9 </my_doc>

This document is not valid, because ParEntity is a parameter entity. Because general and parameter entities are in different namespaces, we can simply declare another general entity, with the name ParEntity:

4 <!ENTITY ParEntity "This is another general entity" >

The document would then be valid. Obviously, having two entities with the same name is generally not a good idea, since it can easily lead to confusion, and is best avoided when possible.

General entities can hold data other than text, for example images or sounds. But these powerful capabilities open the door for a range of mistakes, normally of no concern to XML.

One example is the location of the resource, specified after SYSTEM or PUBLIC inside the external entity declaration; it is known as the system identifier, and is meant to be converted to a URI reference. Relative paths are normally relative to the document within which the entity is declared, but the final URI might be affected by the requirements of particular DTD’s, or might be interpreted in a non-conventional manner by applications. This can lead to the inability to correctly resolve the URI. Additionally, the incorrect NDATA might be specified, which will lead to the file being interpreted as the wrong type of file.