diff options
| author | Mike Hommey <mh@glandium.org> | 2004-03-25 06:59:32 +0000 |
|---|---|---|
| committer | Mike Hommey <mh@glandium.org> | 2004-03-25 06:59:32 +0000 |
| commit | d09ab089457ae3c20cc98f9afa03379c6ebf9598 (patch) | |
| tree | f34702d634972abbc1b478a4529149b548a1cd4c /doc/xml.html | |
| download | libxml2-d09ab089457ae3c20cc98f9afa03379c6ebf9598.tar.gz | |
[svn-inject] Installing original source versionupstream/2.6.8
Diffstat (limited to 'doc/xml.html')
| -rw-r--r-- | doc/xml.html | 4452 |
1 files changed, 4452 insertions, 0 deletions
diff --git a/doc/xml.html b/doc/xml.html new file mode 100644 index 0000000..909f76a --- /dev/null +++ b/doc/xml.html @@ -0,0 +1,4452 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" + "http://www.w3.org/TR/html4/loose.dtd"> +<html> +<head> + <title>The XML C parser and toolkit of Gnome</title> + <meta name="GENERATOR" content="amaya 5.1"> + <meta http-equiv="Content-Type" content="text/html"> +</head> + +<body bgcolor="#ffffff"> +<h1 align="center">The XML C parser and toolkit of Gnome</h1> + +<h1>Note: this is the flat content of the <a href="index.html">web +site</a></h1> + +<h1 style="text-align: center">libxml, a.k.a. gnome-xml</h1> + +<p></p> + +<p +style="text-align: right; font-style: italic; font-size: 10pt">"Programming +with libxml2 is like the thrilling embrace of an exotic stranger." <a +href="http://diveintomark.org/archives/2004/02/18/libxml2">Mark +Pilgrim</a></p> + +<p>Libxml2 is the XML C parser and toolkit developed for the Gnome project +(but usable outside of the Gnome platform), it is free software available +under the <a href="http://www.opensource.org/licenses/mit-license.html">MIT +License</a>. XML itself is a metalanguage to design markup languages, i.e. +text language where semantic and structure are added to the content using +extra "markup" information enclosed between angle brackets. HTML is the most +well-known markup language. Though the library is written in C <a +href="python.html">a variety of language bindings</a> make it available in +other environments.</p> + +<p>Libxml2 is known to be very portable, the library should build and work +without serious troubles on a variety of systems (Linux, Unix, Windows, +CygWin, MacOS, MacOS X, RISC Os, OS/2, VMS, QNX, MVS, ...)</p> + +<p>Libxml2 implements a number of existing standards related to markup +languages:</p> +<ul> + <li>the XML standard: <a + href="http://www.w3.org/TR/REC-xml">http://www.w3.org/TR/REC-xml</a></li> + <li>Namespaces in XML: <a + href="http://www.w3.org/TR/REC-xml-names/">http://www.w3.org/TR/REC-xml-names/</a></li> + <li>XML Base: <a + href="http://www.w3.org/TR/xmlbase/">http://www.w3.org/TR/xmlbase/</a></li> + <li><a href="http://www.cis.ohio-state.edu/rfc/rfc2396.txt">RFC 2396</a> : + Uniform Resource Identifiers <a + href="http://www.ietf.org/rfc/rfc2396.txt">http://www.ietf.org/rfc/rfc2396.txt</a></li> + <li>XML Path Language (XPath) 1.0: <a + href="http://www.w3.org/TR/xpath">http://www.w3.org/TR/xpath</a></li> + <li>HTML4 parser: <a + href="http://www.w3.org/TR/html401/">http://www.w3.org/TR/html401/</a></li> + <li>XML Pointer Language (XPointer) Version 1.0: <a + href="http://www.w3.org/TR/xptr">http://www.w3.org/TR/xptr</a></li> + <li>XML Inclusions (XInclude) Version 1.0: <a + href="http://www.w3.org/TR/xinclude/">http://www.w3.org/TR/xinclude/</a></li> + <li>ISO-8859-x encodings, as well as <a + href="http://www.cis.ohio-state.edu/rfc/rfc2044.txt">rfc2044</a> [UTF-8] + and <a href="http://www.cis.ohio-state.edu/rfc/rfc2781.txt">rfc2781</a> + [UTF-16] Unicode encodings, and more if using iconv support</li> + <li>part of SGML Open Technical Resolution TR9401:1997</li> + <li>XML Catalogs Working Draft 06 August 2001: <a + href="http://www.oasis-open.org/committees/entity/spec-2001-08-06.html">http://www.oasis-open.org/committees/entity/spec-2001-08-06.html</a></li> + <li>Canonical XML Version 1.0: <a + href="http://www.w3.org/TR/xml-c14n">http://www.w3.org/TR/xml-c14n</a> + and the Exclusive XML Canonicalization CR draft <a + href="http://www.w3.org/TR/xml-exc-c14n">http://www.w3.org/TR/xml-exc-c14n</a></li> + <li>Relax NG, ISO/IEC 19757-2:2003, <a + href="http://www.oasis-open.org/committees/relax-ng/spec-20011203.html">http://www.oasis-open.org/committees/relax-ng/spec-20011203.html</a></li> + <li>W3C XML Schemas Part 2: Datatypes <a + href="http://www.w3.org/TR/2001/REC-xmlschema-2-20010502/">REC 02 May + 2001</a></li> +</ul> + +<p>In most cases libxml2 tries to implement the specifications in a +relatively strictly compliant way. As of release 2.4.16, libxml2 passes all +1800+ tests from the <a +href="http://www.oasis-open.org/committees/xml-conformance/">OASIS XML Tests +Suite</a>.</p> + +<p>To some extent libxml2 provides support for the following additional +specifications but doesn't claim to implement them completely:</p> +<ul> + <li>Document Object Model (DOM) <a + href="http://www.w3.org/TR/DOM-Level-2-Core/">http://www.w3.org/TR/DOM-Level-2-Core/</a> + it doesn't implement the API itself, gdome2 does this on top of + libxml2</li> + <li><a href="http://www.cis.ohio-state.edu/rfc/rfc959.txt">RFC 959</a> : + libxml2 implements a basic FTP client code</li> + <li><a href="http://www.cis.ohio-state.edu/rfc/rfc1945.txt">RFC 1945</a> : + HTTP/1.0, again a basic HTTP client code</li> + <li>SAX: a minimal SAX implementation compatible with early expat + versions</li> + <li>DocBook SGML v4: libxml2 includes a hackish parser to transition to + XML</li> +</ul> + +<p>A partial implementation of <a +href="http://www.w3.org/TR/2001/REC-xmlschema-1-20010502/">XML Schemas Part +1: Structure</a> is being worked on but it would be far too early to make any +conformance statement about it at the moment.</p> + +<p>Separate documents:</p> +<ul> + <li><a href="http://xmlsoft.org/XSLT/">the libxslt page</a> providing an + implementation of XSLT 1.0 and common extensions like EXSLT for + libxml2</li> + <li><a href="http://www.cs.unibo.it/~casarini/gdome2/">the gdome2 page</a> + : a standard DOM2 implementation for libxml2</li> + <li><a href="http://www.aleksey.com/xmlsec/">the XMLSec page</a>: an + implementation of <a href="http://www.w3.org/TR/xmldsig-core/">W3C XML + Digital Signature</a> for libxml2</li> + <li>also check the related links section below for more related and active + projects.</li> +</ul> +<!----------------<p>Results of the <a +href="http://xmlbench.sourceforge.net/results/benchmark/index.html">xmlbench +benchmark</a> on sourceforge February 2004 (smaller is better):</p> + +<p align="center"><img src="benchmark.png" +alt="benchmark results for Expat Xerces libxml2 Oracle and Sun toolkits"></p> +--------------> + + +<p>Logo designed by <a href="mailto:liyanage@access.ch">Marc Liyanage</a>.</p> + +<h2><a name="Introducti">Introduction</a></h2> + +<p>This document describes libxml, the <a +href="http://www.w3.org/XML/">XML</a> C parser and toolkit developed for the +<a href="http://www.gnome.org/">Gnome</a> project. <a +href="http://www.w3.org/XML/">XML is a standard</a> for building tag-based +structured documents/data.</p> + +<p>Here are some key points about libxml:</p> +<ul> + <li>Libxml2 exports Push (progressive) and Pull (blocking) type parser + interfaces for both XML and HTML.</li> + <li>Libxml2 can do DTD validation at parse time, using a parsed document + instance, or with an arbitrary DTD.</li> + <li>Libxml2 includes complete <a + href="http://www.w3.org/TR/xpath">XPath</a>, <a + href="http://www.w3.org/TR/xptr">XPointer</a> and <a + href="http://www.w3.org/TR/xinclude">XInclude</a> implementations.</li> + <li>It is written in plain C, making as few assumptions as possible, and + sticking closely to ANSI C/POSIX for easy embedding. Works on + Linux/Unix/Windows, ported to a number of other platforms.</li> + <li>Basic support for HTTP and FTP client allowing applications to fetch + remote resources.</li> + <li>The design is modular, most of the extensions can be compiled out.</li> + <li>The internal document representation is as close as possible to the <a + href="http://www.w3.org/DOM/">DOM</a> interfaces.</li> + <li>Libxml2 also has a <a + href="http://www.megginson.com/SAX/index.html">SAX like interface</a>; + the interface is designed to be compatible with <a + href="http://www.jclark.com/xml/expat.html">Expat</a>.</li> + <li>This library is released under the <a + href="http://www.opensource.org/licenses/mit-license.html">MIT + License</a>. See the Copyright file in the distribution for the precise + wording.</li> +</ul> + +<p>Warning: unless you are forced to because your application links with a +Gnome-1.X library requiring it, <strong><span +style="background-color: #FF0000">Do Not Use libxml1</span></strong>, use +libxml2</p> + +<h2><a name="FAQ">FAQ</a></h2> + +<p>Table of Contents:</p> +<ul> + <li><a href="FAQ.html#License">License(s)</a></li> + <li><a href="FAQ.html#Installati">Installation</a></li> + <li><a href="FAQ.html#Compilatio">Compilation</a></li> + <li><a href="FAQ.html#Developer">Developer corner</a></li> +</ul> + +<h3><a name="License">License</a>(s)</h3> +<ol> + <li><em>Licensing Terms for libxml</em> + <p>libxml2 is released under the <a + href="http://www.opensource.org/licenses/mit-license.html">MIT + License</a>; see the file Copyright in the distribution for the precise + wording</p> + </li> + <li><em>Can I embed libxml2 in a proprietary application ?</em> + <p>Yes. The MIT License allows you to keep proprietary the changes you + made to libxml, but it would be graceful to send-back bug fixes and + improvements as patches for possible incorporation in the main + development tree.</p> + </li> +</ol> + +<h3><a name="Installati">Installation</a></h3> +<ol> + <li><strong><span style="background-color: #FF0000">Do Not Use + libxml1</span></strong>, use libxml2</li> + <li><em>Where can I get libxml</em> ? + <p>The original distribution comes from <a + href="ftp://rpmfind.net/pub/libxml/">rpmfind.net</a> or <a + href="ftp://ftp.gnome.org/pub/GNOME/sources/libxml2/2.6/">gnome.org</a></p> + <p>Most Linux and BSD distributions include libxml, this is probably the + safer way for end-users to use libxml.</p> + <p>David Doolin provides precompiled Windows versions at <a + href="http://www.ce.berkeley.edu/~doolin/code/libxmlwin32/ ">http://www.ce.berkeley.edu/~doolin/code/libxmlwin32/</a></p> + </li> + <li><em>I see libxml and libxml2 releases, which one should I install ?</em> + <ul> + <li>If you are not constrained by backward compatibility issues with + existing applications, install libxml2 only</li> + <li>If you are not doing development, you can safely install both. + Usually the packages <a + href="http://rpmfind.net/linux/RPM/libxml.html">libxml</a> and <a + href="http://rpmfind.net/linux/RPM/libxml2.html">libxml2</a> are + compatible (this is not the case for development packages).</li> + <li>If you are a developer and your system provides separate packaging + for shared libraries and the development components, it is possible + to install libxml and libxml2, and also <a + href="http://rpmfind.net/linux/RPM/libxml-devel.html">libxml-devel</a> + and <a + href="http://rpmfind.net/linux/RPM/libxml2-devel.html">libxml2-devel</a> + too for libxml2 >= 2.3.0</li> + <li>If you are developing a new application, please develop against + libxml2(-devel)</li> + </ul> + </li> + <li><em>I can't install the libxml package, it conflicts with libxml0</em> + <p>You probably have an old libxml0 package used to provide the shared + library for libxml.so.0, you can probably safely remove it. The libxml + packages provided on <a + href="ftp://rpmfind.net/pub/libxml/">rpmfind.net</a> provide + libxml.so.0</p> + </li> + <li><em>I can't install the libxml(2) RPM package due to failed + dependencies</em> + <p>The most generic solution is to re-fetch the latest src.rpm , and + rebuild it locally with</p> + <p><code>rpm --rebuild libxml(2)-xxx.src.rpm</code>.</p> + <p>If everything goes well it will generate two binary rpm packages (one + providing the shared libs and xmllint, and the other one, the -devel + package, providing includes, static libraries and scripts needed to build + applications with libxml(2)) that you can install locally.</p> + </li> +</ol> + +<h3><a name="Compilatio">Compilation</a></h3> +<ol> + <li><em>What is the process to compile libxml2 ?</em> + <p>As most UNIX libraries libxml2 follows the "standard":</p> + <p><code>gunzip -c xxx.tar.gz | tar xvf -</code></p> + <p><code>cd libxml-xxxx</code></p> + <p><code>./configure --help</code></p> + <p>to see the options, then the compilation/installation proper</p> + <p><code>./configure [possible options]</code></p> + <p><code>make</code></p> + <p><code>make install</code></p> + <p>At that point you may have to rerun ldconfig or a similar utility to + update your list of installed shared libs.</p> + </li> + <li><em>What other libraries are needed to compile/install libxml2 ?</em> + <p>Libxml2 does not require any other library, the normal C ANSI API + should be sufficient (please report any violation to this rule you may + find).</p> + <p>However if found at configuration time libxml2 will detect and use the + following libs:</p> + <ul> + <li><a href="http://www.info-zip.org/pub/infozip/zlib/">libz</a> : a + highly portable and available widely compression library.</li> + <li>iconv: a powerful character encoding conversion library. It is + included by default in recent glibc libraries, so it doesn't need to + be installed specifically on Linux. It now seems a <a + href="http://www.opennc.org/onlinepubs/7908799/xsh/iconv.html">part + of the official UNIX</a> specification. Here is one <a + href="http://www.gnu.org/software/libiconv/">implementation of the + library</a> which source can be found <a + href="ftp://ftp.ilog.fr/pub/Users/haible/gnu/">here</a>.</li> + </ul> + </li> + <li><em>Make check fails on some platforms</em> + <p>Sometimes the regression tests' results don't completely match the + value produced by the parser, and the makefile uses diff to print the + delta. On some platforms the diff return breaks the compilation process; + if the diff is small this is probably not a serious problem.</p> + <p>Sometimes (especially on Solaris) make checks fail due to limitations + in make. Try using GNU-make instead.</p> + </li> + <li><em>I use the CVS version and there is no configure script</em> + <p>The configure script (and other Makefiles) are generated. Use the + autogen.sh script to regenerate the configure script and Makefiles, + like:</p> + <p><code>./autogen.sh --prefix=/usr --disable-shared</code></p> + </li> + <li><em>I have troubles when running make tests with gcc-3.0</em> + <p>It seems the initial release of gcc-3.0 has a problem with the + optimizer which miscompiles the URI module. Please use another + compiler.</p> + </li> +</ol> + +<h3><a name="Developer">Developer</a> corner</h3> +<ol> + <li><em>Troubles compiling or linking programs using libxml2</em> + <p>Usually the problem comes from the fact that the compiler doesn't get + the right compilation or linking flags. There is a small shell script + <code>xml2-config</code> which is installed as part of libxml2 usual + install process which provides those flags. Use</p> + <p><code>xml2-config --cflags</code></p> + <p>to get the compilation flags and</p> + <p><code>xml2-config --libs</code></p> + <p>to get the linker flags. Usually this is done directly from the + Makefile as:</p> + <p><code>CFLAGS=`xml2-config --cflags`</code></p> + <p><code>LIBS=`xml2-config --libs`</code></p> + </li> + <li><em>xmlDocDump() generates output on one line.</em> + <p>Libxml2 will not <strong>invent</strong> spaces in the content of a + document since <strong>all spaces in the content of a document are + significant</strong>. If you build a tree from the API and want + indentation:</p> + <ol> + <li>the correct way is to generate those yourself too.</li> + <li>the dangerous way is to ask libxml2 to add those blanks to your + content <strong>modifying the content of your document in the + process</strong>. The result may not be what you expect. There is + <strong>NO</strong> way to guarantee that such a modification won't + affect other parts of the content of your document. See <a + href="http://xmlsoft.org/html/libxml-parser.html#xmlKeepBlanksDefault">xmlKeepBlanksDefault + ()</a> and <a + href="http://xmlsoft.org/html/libxml-tree.html#xmlSaveFormatFile">xmlSaveFormatFile + ()</a></li> + </ol> + </li> + <li>Extra nodes in the document: + <p><em>For a XML file as below:</em></p> + <pre><?xml version="1.0"?> +<PLAN xmlns="http://www.argus.ca/autotest/1.0/"> +<NODE CommFlag="0"/> +<NODE CommFlag="1"/> +</PLAN></pre> + <p><em>after parsing it with the function + pxmlDoc=xmlParseFile(...);</em></p> + <p><em>I want to the get the content of the first node (node with the + CommFlag="0")</em></p> + <p><em>so I did it as following;</em></p> + <pre>xmlNodePtr pnode; +pnode=pxmlDoc->children->children;</pre> + <p><em>but it does not work. If I change it to</em></p> + <pre>pnode=pxmlDoc->children->children->next;</pre> + <p><em>then it works. Can someone explain it to me.</em></p> + <p></p> + <p>In XML all characters in the content of the document are significant + <strong>including blanks and formatting line breaks</strong>.</p> + <p>The extra nodes you are wondering about are just that, text nodes with + the formatting spaces which are part of the document but that people tend + to forget. There is a function <a + href="http://xmlsoft.org/html/libxml-parser.html">xmlKeepBlanksDefault + ()</a> to remove those at parse time, but that's an heuristic, and its + use should be limited to cases where you are certain there is no + mixed-content in the document.</p> + </li> + <li><em>I get compilation errors of existing code like when accessing + <strong>root</strong> or <strong>child fields</strong> of nodes.</em> + <p>You are compiling code developed for libxml version 1 and using a + libxml2 development environment. Either switch back to libxml v1 devel or + even better fix the code to compile with libxml2 (or both) by <a + href="upgrade.html">following the instructions</a>.</p> + </li> + <li><em>I get compilation errors about non existing + <strong>xmlRootNode</strong> or <strong>xmlChildrenNode</strong> + fields.</em> + <p>The source code you are using has been <a + href="upgrade.html">upgraded</a> to be able to compile with both libxml + and libxml2, but you need to install a more recent version: + libxml(-devel) >= 1.8.8 or libxml2(-devel) >= 2.1.0</p> + </li> + <li><em>XPath implementation looks seriously broken</em> + <p>XPath implementation prior to 2.3.0 was really incomplete. Upgrade to + a recent version, there are no known bugs in the current version.</p> + </li> + <li><em>The example provided in the web page does not compile.</em> + <p>It's hard to maintain the documentation in sync with the code + <grin/> ...</p> + <p>Check the previous points 1/ and 2/ raised before, and please send + patches.</p> + </li> + <li><em>Where can I get more examples and information than provided on the + web page?</em> + <p>Ideally a libxml2 book would be nice. I have no such plan ... But you + can:</p> + <ul> + <li>check more deeply the <a href="html/libxml-lib.html">existing + generated doc</a></li> + <li>have a look at <a href="examples/index.html">the set of + examples</a>.</li> + <li>look for examples of use for libxml2 function using the Gnome code. + For example the following will query the full Gnome CVS base for the + use of the <strong>xmlAddChild()</strong> function: + <p><a + href="http://cvs.gnome.org/lxr/search?string=xmlAddChild">http://cvs.gnome.org/lxr/search?string=xmlAddChild</a></p> + <p>This may be slow, a large hardware donation to the gnome project + could cure this :-)</p> + </li> + <li><a + href="http://cvs.gnome.org/bonsai/rview.cgi?cvsroot=/cvs/gnome&dir=gnome-xml">Browse + the libxml2 source</a> , I try to write code as clean and documented + as possible, so looking at it may be helpful. In particular the code + of xmllint.c and of the various testXXX.c test programs should + provide good examples of how to do things with the library.</li> + </ul> + </li> + <li>What about C++ ? + <p>libxml2 is written in pure C in order to allow easy reuse on a number + of platforms, including embedded systems. I don't intend to convert to + C++.</p> + <p>There is however a C++ wrapper which may fulfill your needs:</p> + <ul> + <li>by Ari Johnson <ari@btigate.com>: + <p>Website: <a + href="http://libxmlplusplus.sourceforge.net/">http://libxmlplusplus.sourceforge.net/</a></p> + <p>Download: <a + href="http://sourceforge.net/project/showfiles.php?group_id=12999">http://sourceforge.net/project/showfiles.php?group_id=12999</a></p> + </li> + <!-- Website is currently unavailable as of 2003-08-02 + <li>by Peter Jones <pjones@pmade.org> + <p>Website: <a + href="http://pmade.org/pjones/software/xmlwrapp/">http://pmade.org/pjones/software/xmlwrapp/</a></p> + </li> + --> + </ul> + </li> + <li>How to validate a document a posteriori ? + <p>It is possible to validate documents which had not been validated at + initial parsing time or documents which have been built from scratch + using the API. Use the <a + href="http://xmlsoft.org/html/libxml-valid.html#xmlValidateDtd">xmlValidateDtd()</a> + function. It is also possible to simply add a DTD to an existing + document:</p> + <pre>xmlDocPtr doc; /* your existing document */ +xmlDtdPtr dtd = xmlParseDTD(NULL, filename_of_dtd); /* parse the DTD */ + + dtd->name = xmlStrDup((xmlChar*)"root_name"); /* use the given root */ + + doc->intSubset = dtd; + if (doc->children == NULL) xmlAddChild((xmlNodePtr)doc, (xmlNodePtr)dtd); + else xmlAddPrevSibling(doc->children, (xmlNodePtr)dtd); + </pre> + </li> + <li>So what is this funky "xmlChar" used all the time? + <p>It is a null terminated sequence of utf-8 characters. And only utf-8! + You need to convert strings encoded in different ways to utf-8 before + passing them to the API. This can be accomplished with the iconv library + for instance.</p> + </li> + <li>etc ...</li> +</ol> + +<p></p> + +<h2><a name="Documentat">Developer Menu</a></h2> + +<p>There are several on-line resources related to using libxml:</p> +<ol> + <li>Use the <a href="search.php">search engine</a> to look up + information.</li> + <li>Check the <a href="FAQ.html">FAQ.</a></li> + <li>Check the <a href="http://xmlsoft.org/html/libxml-lib.html">extensive + documentation</a> automatically extracted from code comments.</li> + <li>Look at the documentation about <a href="encoding.html">libxml + internationalization support</a>.</li> + <li>This page provides a global overview and <a href="example.html">some + examples</a> on how to use libxml.</li> + <li><a href="examples/index.html">Code examples</a></li> + <li>John Fleck's libxml2 tutorial: <a href="tutorial/index.html">html</a> + or <a href="tutorial/xmltutorial.pdf">pdf</a>.</li> + <li>If you need to parse large files, check the <a + href="xmlreader.html">xmlReader</a> API tutorial</li> + <li><a href="mailto:james@daa.com.au">James Henstridge</a> wrote <a + href="http://www.daa.com.au/~james/gnome/xml-sax/xml-sax.html">some nice + documentation</a> explaining how to use the libxml SAX interface.</li> + <li>George Lebl wrote <a + href="http://www-106.ibm.com/developerworks/library/l-gnome3/">an article + for IBM developerWorks</a> about using libxml.</li> + <li>Check <a href="http://cvs.gnome.org/lxr/source/gnome-xml/TODO">the TODO + file</a>.</li> + <li>Read the <a href="upgrade.html">1.x to 2.x upgrade path</a> + description. If you are starting a new project using libxml you should + really use the 2.x version.</li> + <li>And don't forget to look at the <a + href="http://mail.gnome.org/archives/xml/">mailing-list archive</a>.</li> +</ol> + +<h2><a name="Reporting">Reporting bugs and getting help</a></h2> + +<p>Well, bugs or missing features are always possible, and I will make a +point of fixing them in a timely fashion. The best way to report a bug is to +use the <a href="http://bugzilla.gnome.org/buglist.cgi?product=libxml2">Gnome +bug tracking database</a> (make sure to use the "libxml2" module name). I +look at reports there regularly and it's good to have a reminder when a bug +is still open. Be sure to specify that the bug is for the package libxml2.</p> + +<p>For small problems you can try to get help on IRC, the #xml channel on +irc.gnome.org (port 6667) usually have a few person subscribed which may help +(but there is no garantee and if a real issue is raised it should go on the +mailing-list for archival).</p> + +<p>There is also a mailing-list <a +href="mailto:xml@gnome.org">xml@gnome.org</a> for libxml, with an <a +href="http://mail.gnome.org/archives/xml/">on-line archive</a> (<a +href="http://xmlsoft.org/messages">old</a>). To subscribe to this list, +please visit the <a +href="http://mail.gnome.org/mailman/listinfo/xml">associated Web</a> page and +follow the instructions. <strong>Do not send code, I won't debug it</strong> +(but patches are really appreciated!).</p> + +<p>Check the following <strong><span style="color: #FF0000">before +posting</span></strong>:</p> +<ul> + <li>Read the <a href="FAQ.html">FAQ</a> and <a href="search.php">use the + search engine</a> to get information related to your problem.</li> + <li>Make sure you are <a href="ftp://xmlsoft.org/">using a recent + version</a>, and that the problem still shows up in a recent version.</li> + <li>Check the <a href="http://mail.gnome.org/archives/xml/">list + archives</a> to see if the problem was reported already. In this case + there is probably a fix available, similarly check the <a + href="http://bugzilla.gnome.org/buglist.cgi?product=libxml2">registered + open bugs</a>.</li> + <li>Make sure you can reproduce the bug with xmllint or one of the test + programs found in source in the distribution.</li> + <li>Please send the command showing the error as well as the input (as an + attachment)</li> +</ul> + +<p>Then send the bug with associated information to reproduce it to the <a +href="mailto:xml@gnome.org">xml@gnome.org</a> list; if it's really libxml +related I will approve it. Please do not send mail to me directly, it makes +things really hard to track and in some cases I am not the best person to +answer a given question, ask on the list.</p> + +<p>To <span style="color: #E50000">be really clear about support</span>:</p> +<ul> + <li>Support or help <span style="color: #E50000">requests MUST be sent to + the list or on bugzilla</span> in case of problems, so that the Question + and Answers can be shared publicly. Failing to do so carries the implicit + message "I want free support but I don't want to share the benefits with + others" and is not welcome. I will automatically Carbon-Copy the + xml@gnome.org mailing list for any technical reply made about libxml2 or + libxslt.</li> + <li>There is <span style="color: #E50000">no garantee of support</span>, if + your question remains unanswered after a week, repost it, making sure you + gave all the detail needed and the information requested.</li> + <li>Failing to provide information as requested or double checking first + for prior feedback also carries the implicit message "the time of the + library maintainers is less valuable than my time" and might not be + welcome.</li> +</ul> + +<p>Of course, bugs reported with a suggested patch for fixing them will +probably be processed faster than those without.</p> + +<p>If you're looking for help, a quick look at <a +href="http://mail.gnome.org/archives/xml/">the list archive</a> may actually +provide the answer. I usually send source samples when answering libxml2 +usage questions. The <a +href="http://xmlsoft.org/html/book1.html">auto-generated documentation</a> is +not as polished as I would like (i need to learn more about DocBook), but +it's a good starting point.</p> + +<h2><a name="help">How to help</a></h2> + +<p>You can help the project in various ways, the best thing to do first is to +subscribe to the mailing-list as explained before, check the <a +href="http://mail.gnome.org/archives/xml/">archives </a>and the <a +href="http://bugzilla.gnome.org/buglist.cgi?product=libxml2">Gnome bug +database</a>:</p> +<ol> + <li>Provide patches when you find problems.</li> + <li>Provide the diffs when you port libxml2 to a new platform. They may not + be integrated in all cases but help pinpointing portability problems + and</li> + <li>Provide documentation fixes (either as patches to the code comments or + as HTML diffs).</li> + <li>Provide new documentations pieces (translations, examples, etc + ...).</li> + <li>Check the TODO file and try to close one of the items.</li> + <li>Take one of the points raised in the archive or the bug database and + provide a fix. <a href="mailto:daniel@veillard.com">Get in touch with me + </a>before to avoid synchronization problems and check that the suggested + fix will fit in nicely :-)</li> +</ol> + +<h2><a name="Downloads">Downloads</a></h2> + +<p>The latest versions of libxml2 can be found on <a +href="ftp://xmlsoft.org/">xmlsoft.org</a> (<a +href="ftp://speakeasy.rpmfind.net/pub/libxml/">Seattle</a>, <a +href="ftp://fr.rpmfind.net/pub/libxml/">France</a>) or on the <a +href="ftp://ftp.gnome.org/pub/GNOME/MIRRORS.html">Gnome FTP server</a> either +as a <a href="ftp://ftp.gnome.org/pub/GNOME/sources/libxml2/2.6/">source +archive</a><!-- commenting this out because they seem to have disappeared or <a +href="ftp://ftp.gnome.org/pub/GNOME/stable/redhat/i386/libxml/">RPM +packages</a> --> + , Antonin Sprinzl also provide <a href="ftp://gd.tuwien.ac.at/pub/libxml/">a +mirror in Austria</a>. (NOTE that you need both the <a +href="http://rpmfind.net/linux/RPM/libxml2.html">libxml(2)</a> and <a +href="http://rpmfind.net/linux/RPM/libxml2-devel.html">libxml(2)-devel</a> +packages installed to compile applications using libxml.)</p> + +<p>Binary ports:</p> +<ul> + <li>Red Hat RPMs for i386 are available directly on <a + href="ftp://xmlsoft.org/">xmlsoft.org</a>, the source RPM will compile on + any architecture supported by Red Hat.</li> + <li><p><a href="mailto:igor@zlatkovic.com">Igor Zlatkovic</a></p> + is now the maintainer of the Windows port, <a + href="http://www.zlatkovic.com/projects/libxml/index.html">he provides + binaries</a>.</li> + <li><a href="mailto:Gary.Pennington@sun.com">Gary Pennington</a> provides + <a href="http://garypennington.net/libxml2/">Solaris binaries</a>.</li> + <li><a href="mailto:Steve.Ball@zveno.com">Steve Ball</a> provides <a + href="http://www.zveno.com/open_source/libxml2xslt.html">Mac Os X + binaries</a>.</li> + <li>The HP-UX porting center provides <a + href="http://hpux.connect.org.uk/hppd/hpux/Gnome/">HP-UX binaries</a></li> +</ul> + +<p>If you know other supported binary ports, please <a +href="http://veillard.com/">contact me</a>.</p> + +<p><a name="Snapshot">Snapshot:</a></p> +<ul> + <li>Code from the W3C cvs base gnome-xml <a + href="ftp://xmlsoft.org/cvs-snapshot.tar.gz">cvs-snapshot.tar.gz</a>.</li> + <li>Docs, content of the web site, the list archive included <a + href="ftp://xmlsoft.org/libxml-docs.tar.gz">libxml-docs.tar.gz</a>.</li> +</ul> + +<p><a name="Contribs">Contributions:</a></p> + +<p>I do accept external contributions, especially if compiling on another +platform, get in touch with me to upload the package, wrappers for various +languages have been provided, and can be found in the <a +href="contribs.html">contrib section</a></p> + +<p>Libxml2 is also available from CVS:</p> +<ul> + <li><p>The <a + href="http://cvs.gnome.org/bonsai/rview.cgi?cvsroot=/cvs/gnome&dir=gnome-xml">Gnome + CVS base</a>. Check the <a + href="http://developer.gnome.org/tools/cvs.html">Gnome CVS Tools</a> + page; the CVS module is <b>gnome-xml</b>.</p> + </li> + <li>The <strong>libxslt</strong> module is also present there</li> +</ul> + +<h2><a name="News">News</a></h2> + +<h3>CVS only : check the <a +href="http://cvs.gnome.org/lxr/source/gnome-xml/ChangeLog">Changelog</a> file +for a really accurate description</h3> + +<p>Items not finished and worked on, get in touch with the list if you want +to test those</p> +<ul> + <li>More testing on RelaxNG</li> + <li>Finishing up <a href="http://www.w3.org/TR/xmlschema-1/">XML + Schemas</a></li> +</ul> + +<h3>2.6.8: Mar 23 2004</h3> +<ul> + <li>First step of the cleanup of the serialization code and APIs</li> + <li>XML Schemas: mixed content (Adam Dickmeiss), QName handling fixes (Adam + Dickmeiss), anyURI for "" (John Belmonte)</li> + <li>Python: Canonicalization C14N support added (Anthony Carrico)</li> + <li>xmlDocCopyNode() extension (William)</li> + <li>Relax-NG: fix when processing XInclude results (William), external + reference in interleave (William), missing error on <choice> + failure (William), memory leak in schemas datatype facets.</li> + <li>xmlWriter: patch for better DTD support (Alfred Mickautsch)</li> + <li>bug fixes: xmlXPathLangFunction memory leak (Mike Hommey and William + Brack), no ID errors if using HTML_PARSE_NOERROR, xmlcatalog fallbacks to + URI on SYSTEM lookup failure, XInclude parse flags inheritance (William), + XInclude and XPointer fixes for entities (William), XML parser bug + reported by Holger Rauch, nanohttp fd leak (William), regexps char + groups '-' handling (William), dictionnary reference counting problems, + do not close stderr. </li> + <li>performance patches from Petr Pajas</li> + <li>Documentation fixes: XML_CATALOG_FILES in man pages (Mike Hommey)</li> + <li>compilation and portability fixes: --without-valid, catalog cleanups + (Peter Breitenlohner), MingW patch (Roland Schwingel), cross-compilation + to Windows (Christophe de Vienne), --with-html-dir fixup (Julio Merino + Vidal), Windows build (Eric Zurcher)</li> +</ul> + +<h3>2.6.7: Feb 23 2004</h3> +<ul> + <li>documentation: tutorial updates (John Fleck), benchmark results</li> + <li>xmlWriter: updates and fixes (Alfred Mickautsch, Lucas Brasilino)</li> + <li>XPath optimization (Petr Pajas)</li> + <li>DTD ID handling optimization</li> + <li>bugfixes: xpath number with > 19 fractional (William Brack), push + mode with unescaped '>' characters, fix xmllint --stream --timing, fix + xmllint --memory --stream memory usage, xmlAttrSerializeTxtContent + handling NULL, trying to fix Relax-NG/Perl interface.</li> + <li>python: 2.3 compatibility, whitespace fixes (Malcolm Tredinnick)</li> + <li>Added relaxng option to xmllint --shell</li> +</ul> + +<h3>2.6.6: Feb 12 2004</h3> +<ul> + <li>nanohttp and nanoftp: buffer overflow error on URI parsing (Igor and + William) reported by Yuuichi Teranishi</li> + <li>bugfixes: make test and path issues, xmlWriter attribute serialization + (William Brack), xmlWriter indentation (William), schemas validation + (Eric Haszlakiewicz), XInclude dictionnaries issues (William and Oleg + Paraschenko), XInclude empty fallback (William), HTML warnings (William), + XPointer in XInclude (William), Python namespace serialization, + isolat1ToUTF8 bound error (Alfred Mickautsch), output of parameter + entities in internal subset (William), internal subset bug in push mode, + <xs:all> fix (Alexey Sarytchev)</li> + <li>Build: fix for automake-1.8 (Alexander Winston), warnings removal + (Philip Ludlam), SOCKLEN_T detection fixes (Daniel Richard), fix + --with-minimum configuration.</li> + <li>XInclude: allow the 2001 namespace without warning.</li> + <li>Documentation: missing example/index.html (John Fleck), version + dependancies (John Fleck)</li> + <li>reader API: structured error reporting (Steve Ball)</li> + <li>Windows compilation: mingw, msys (Mikhail Grushinskiy), function + prototype (Cameron Johnson), MSVC6 compiler warnings, _WINSOCKAPI_ + patch</li> + <li>Parsers: added xmlByteConsumed(ctxt) API to get the byte offest in + input.</li> +</ul> + +<h3>2.6.5: Jan 25 2004</h3> +<ul> + <li>Bugfixes: dictionnaries for schemas (William Brack), regexp segfault + (William), xs:all problem (William), a number of XPointer bugfixes + (William), xmllint error go to stderr, DTD validation problem with + namespace, memory leak (William), SAX1 cleanup and minimal options fixes + (Mark Vadoc), parser context reset on error (Shaun McCance), XPath union + evaluation problem (William) , xmlReallocLoc with NULL (Aleksey Sanin), + XML Schemas double free (Steve Ball), XInclude with no href, argument + callbacks order for XPath callbacks (Frederic Peters)</li> + <li>Documentation: python scripts (William Brack), xslt stylesheets (John + Fleck), doc (Sven Zimmerman), I/O example.</li> + <li>Python bindings: fixes (William), enum support (Stéphane Bidoul), + structured error reporting (Stéphane Bidoul)</li> + <li>XInclude: various fixes for conformance, problem related to dictionnary + references (William & me), recursion (William)</li> + <li>xmlWriter: indentation (Lucas Brasilino), memory leaks (Alfred + Mickautsch),</li> + <li>xmlSchemas: normalizedString datatype (John Belmonte)</li> + <li>code cleanup for strings functions (William)</li> + <li>Windows: compiler patches (Mark Vakoc)</li> + <li>Parser optimizations, a few new XPath and dictionnary APIs for future + XSLT optimizations.</li> +</ul> + +<h3>2.6.4: Dec 24 2003</h3> +<ul> + <li>Windows build fixes (Igor Zlatkovic)</li> + <li>Some serious XInclude problems reported by Oleg Paraschenko and</li> + <li>Unix and Makefile packaging fixes (me, William Brack,</li> + <li>Documentation improvements (John Fleck, William Brack), example fix + (Lucas Brasilino)</li> + <li>bugfixes: xmlTextReaderExpand() with xmlReaderWalker, XPath handling of + NULL strings (William Brack) , API building reader or parser from + filedescriptor should not close it, changed XPath sorting to be stable + again (William Brack), xmlGetNodePath() generating '(null)' (William + Brack), DTD validation and namespace bug (William Brack), XML Schemas + double inclusion behaviour</li> +</ul> + +<h3>2.6.3: Dec 10 2003</h3> +<ul> + <li>documentation updates and cleanup (DV, William Brack, John Fleck)</li> + <li>added a repository of examples, examples from Aleksey Sanin, Dodji + Seketeli, Alfred Mickautsch</li> + <li>Windows updates: Mark Vakoc, Igor Zlatkovic, Eric Zurcher, Mingw + (Kenneth Haley)</li> + <li>Unicode range checking (William Brack)</li> + <li>code cleanup (William Brack)</li> + <li>Python bindings: doc (John Fleck), bug fixes</li> + <li>UTF-16 cleanup and BOM issues (William Brack)</li> + <li>bug fixes: ID and xmlReader validation, XPath (William Brack), + xmlWriter (Alfred Mickautsch), hash.h inclusion problem, HTML parser + (James Bursa), attribute defaulting and validation, some serialization + cleanups, XML_GET_LINE macro, memory debug when using threads (William + Brack), serialization of attributes and entities content, xmlWriter + (Daniel Schulman)</li> + <li>XInclude bugfix, new APIs and update to the last version including the + namespace change.</li> + <li>XML Schemas improvements: include (Robert Stepanek), import and + namespace handling, fixed the regression tests troubles, added examples + based on Eric van der Vlist book, regexp fixes</li> + <li>preliminary pattern support for streaming (needed for schemas + constraints), added xmlTextReaderPreservePattern() to collect subdocument + when streaming.</li> + <li>various fixes in the structured error handling</li> +</ul> + +<h3>2.6.2: Nov 4 2003</h3> +<ul> + <li>XPath context unregistration fixes</li> + <li>text node coalescing fixes (Mark Lilback)</li> + <li>API to screate a W3C Schemas from an existing document (Steve Ball)</li> + <li>BeOS patches (Marcin 'Shard' Konicki)</li> + <li>xmlStrVPrintf function added (Aleksey Sanin)</li> + <li>compilation fixes (Mark Vakoc)</li> + <li>stdin parsing fix (William Brack)</li> + <li>a posteriori DTD validation fixes</li> + <li>xmlReader bug fixes: Walker fixes, python bindings</li> + <li>fixed xmlStopParser() to really stop the parser and errors</li> + <li>always generate line numbers when using the new xmlReadxxx + functions</li> + <li>added XInclude support to the xmlReader interface</li> + <li>implemented XML_PARSE_NONET parser option</li> + <li>DocBook XSLT processing bug fixed</li> + <li>HTML serialization for <p> elements (William Brack and me)</li> + <li>XPointer failure in XInclude are now handled as resource errors</li> + <li>fixed xmllint --html to use the HTML serializer on output (added + --xmlout to implement the previous behaviour of saving it using the XML + serializer)</li> +</ul> + +<h3>2.6.1: Oct 28 2003</h3> +<ul> + <li>Mostly bugfixes after the big 2.6.0 changes</li> + <li>Unix compilation patches: libxml.m4 (Patrick Welche), warnings cleanup + (William Brack)</li> + <li>Windows compilation patches (Joachim Bauch, Stephane Bidoul, Igor + Zlatkovic)</li> + <li>xmlWriter bugfix (Alfred Mickautsch)</li> + <li>chvalid.[ch]: couple of fixes from Stephane Bidoul</li> + <li>context reset: error state reset, push parser reset (Graham + Bennett)</li> + <li>context reuse: generate errors if file is not readable</li> + <li>defaulted attributes for element coming from internal entities + (Stephane Bidoul)</li> + <li>Python: tab and spaces mix (William Brack)</li> + <li>Error handler could crash in DTD validation in 2.6.0</li> + <li>xmlReader: do not use the document or element _private field</li> + <li>testSAX.c: avoid a problem with some PIs (Massimo Morara)</li> + <li>general bug fixes: mandatory encoding in text decl, serializing + Document Fragment nodes, xmlSearchNs 2.6.0 problem (Kasimier Buchcik), + XPath errors not reported, slow HTML parsing of large documents.</li> +</ul> + +<h3>2.6.0: Oct 20 2003</h3> +<ul> + <li>Major revision release: should be API and ABI compatible but got a lot + of change</li> + <li>Increased the library modularity, far more options can be stripped out, + a --with-minimum configuration will weight around 160KBytes</li> + <li>Use per parser and per document dictionnary, allocate names and small + text nodes from the dictionnary</li> + <li>Switch to a SAX2 like parser rewrote most of the XML parser core, + provides namespace resolution and defaulted attributes, minimize memory + allocations and copies, namespace checking and specific error handling, + immutable buffers, make predefined entities static structures, etc...</li> + <li>rewrote all the error handling in the library, all errors can be + intercepted at a structured level, with precise information + available.</li> + <li>New simpler and more generic XML and HTML parser APIs, allowing to + easilly modify the parsing options and reuse parser context for multiple + consecutive documents.</li> + <li>Similar new APIs for the xmlReader, for options and reuse, provided new + functions to access content as const strings, use them for Python + bindings</li> + <li>a lot of other smaller API improvements: xmlStrPrintf (Aleksey Sanin), + Walker i.e. reader on a document tree based on Alfred Mickautsch code, + make room in nodes for line numbers, reference counting and future PSVI + extensions, generation of character ranges to be checked with faster + algorithm (William), xmlParserMaxDepth (Crutcher Dunnavant), buffer + access</li> + <li>New xmlWriter API provided by Alfred Mickautsch</li> + <li>Schemas: base64 support by Anthony Carrico</li> + <li>Parser<->HTTP integration fix, proper processing of the Mime-Type + and charset informations if available.</li> + <li>Relax-NG: bug fixes including the one reported by Martijn Faassen and + zeroOrMore, better error reporting.</li> + <li>Python bindings (Stéphane Bidoul), never use stdout for errors + output</li> + <li>Portability: all the headers have macros for export and calling + convention definitions (Igor Zlatkovic), VMS update (Craig A. Berry), + Windows: threads (Jesse Pelton), Borland compiler (Eric Zurcher, Igor), + Mingw (Igor), typos (Mark Vakoc), beta version (Stephane Bidoul), + warning cleanups on AIX and MIPS compilers (William Brack), BeOS (Marcin + 'Shard' Konicki)</li> + <li>Documentation fixes and README (William Brack), search fix (William), + tutorial updates (John Fleck), namespace docs (Stefan Kost)</li> + <li>Bug fixes: xmlCleanupParser (Dave Beckett), threading uninitialized + mutexes, HTML doctype lowercase, SAX/IO (William), compression detection + and restore (William), attribute declaration in DTDs (William), namespace + on attribute in HTML output (William), input filename (Rob Richards), + namespace DTD validation, xmlReplaceNode (Chris Ryland), I/O callbacks + (Markus Keim), CDATA serialization (Shaun McCance), xmlReader (Peter + Derr), high codepoint charref like &#x10FFFF;, buffer access in push + mode (Justin Fletcher), TLS threads on Windows (Jesse Pelton), XPath bug + (William), xmlCleanupParser (Marc Liyanage), CDATA output (William), HTTP + error handling.</li> + <li>xmllint options: --dtdvalidfpi for Tobias Reif, --sax1 for compat + testing, --nodict for building without tree dictionnary, --nocdata to + replace CDATA by text, --nsclean to remove surperfluous namespace + declarations</li> + <li>added xml2-config --libtool-libs option from Kevin P. Fleming</li> + <li>a lot of profiling and tuning of the code, speedup patch for + xmlSearchNs() by Luca Padovani. The xmlReader should do far less + allocation and it speed should get closer to SAX. Chris Anderson worked + on speeding and cleaning up repetitive checking code.</li> + <li>cleanup of "make tests"</li> + <li>libxml-2.0-uninstalled.pc from Malcolm Tredinnick</li> + <li>deactivated the broken docBook SGML parser code and plugged the XML + parser instead.</li> +</ul> + +<h3>2.5.11: Sep 9 2003</h3> + +<p>A bugfix only release:</p> +<ul> + <li>risk of crash in Relax-NG</li> + <li>risk of crash when using multithreaded programs</li> +</ul> + +<h3>2.5.10: Aug 15 2003</h3> + +<p>A bugfixes only release</p> +<ul> + <li>Windows Makefiles (William Brack)</li> + <li>UTF-16 support fixes (Mark Itzcovitz)</li> + <li>Makefile and portability (William Brack) automake, Linux alpha, Mingw + on Windows (Mikhail Grushinskiy)</li> + <li>HTML parser (Oliver Stoeneberg)</li> + <li>XInclude performance problem reported by Kevin Ruscoe</li> + <li>XML parser performance problem reported by Grant Goodale</li> + <li>xmlSAXParseDTD() bug fix from Malcolm Tredinnick</li> + <li>and a couple other cleanup</li> +</ul> + +<h3>2.5.9: Aug 9 2003</h3> +<ul> + <li>bugfixes: IPv6 portability, xmlHasNsProp (Markus Keim), Windows build + (Wiliam Brake, Jesse Pelton, Igor), Schemas (Peter Sobisch), threading + (Rob Richards), hexBinary type (), UTF-16 BOM (Dodji Seketeli), + xmlReader, Relax-NG schemas compilation, namespace handling, EXSLT (Sean + Griffin), HTML parsing problem (William Brack), DTD validation for mixed + content + namespaces, HTML serialization, library initialization, + progressive HTML parser</li> + <li>better interfaces for Relax-NG error handling (Joachim Bauch, )</li> + <li>adding xmlXIncludeProcessTree() for XInclud'ing in a subtree</li> + <li>doc fixes and improvements (John Fleck)</li> + <li>configure flag for -with-fexceptions when embedding in C++</li> + <li>couple of new UTF-8 helper functions (William Brack)</li> + <li>general encoding cleanup + ISO-8859-x without iconv (Peter Jacobi)</li> + <li>xmlTextReader cleanup + enum for node types (Bjorn Reese)</li> + <li>general compilation/warning cleanup Solaris/HP-UX/... (William + Brack)</li> +</ul> + +<h3>2.5.8: Jul 6 2003</h3> +<ul> + <li>bugfixes: XPath, XInclude, file/URI mapping, UTF-16 save (Mark + Itzcovitz), UTF-8 checking, URI saving, error printing (William Brack), + PI related memleak, compilation without schemas or without xpath (Joerg + Schmitz-Linneweber/Garry Pennington), xmlUnlinkNode problem with DTDs, + rpm problem on , i86_64, removed a few compilation problems from 2.5.7, + xmlIOParseDTD, and xmlSAXParseDTD (Malcolm Tredinnick)</li> + <li>portability: DJGPP (MsDos) , OpenVMS (Craig A. Berry)</li> + <li>William Brack fixed multithreading lock problems</li> + <li>IPv6 patch for FTP and HTTP accesses (Archana Shah/Wipro)</li> + <li>Windows fixes (Igor Zlatkovic, Eric Zurcher), threading (Stéphane + Bidoul)</li> + <li>A few W3C Schemas Structure improvements</li> + <li>W3C Schemas Datatype improvements (Charlie Bozeman)</li> + <li>Python bindings for thread globals (Stéphane Bidoul), and method/class + generator</li> + <li>added --nonet option to xmllint</li> + <li>documentation improvements (John Fleck)</li> +</ul> + +<h3>2.5.7: Apr 25 2003</h3> +<ul> + <li>Relax-NG: Compiling to regexp and streaming validation on top of the + xmlReader interface, added to xmllint --stream</li> + <li>xmlReader: Expand(), Next() and DOM access glue, bug fixes</li> + <li>Support for large files: RGN validated a 4.5GB instance</li> + <li>Thread support is now configured in by default</li> + <li>Fixes: update of the Trio code (Bjorn), WXS Date and Duration fixes + (Charles Bozeman), DTD and namespaces (Brent Hendricks), HTML push parser + and zero bytes handling, some missing Windows file path conversions, + behaviour of the parser and validator in the presence of "out of memory" + error conditions</li> + <li>extended the API to be able to plug a garbage collecting memory + allocator, added xmlMallocAtomic() and modified the allocations + accordingly.</li> + <li>Performances: removed excessive malloc() calls, speedup of the push and + xmlReader interfaces, removed excessive thread locking</li> + <li>Documentation: man page (John Fleck), xmlReader documentation</li> + <li>Python: adding binding for xmlCatalogAddLocal (Brent M Hendricks)</li> +</ul> + +<h3>2.5.6: Apr 1 2003</h3> +<ul> + <li>Fixed W3C XML Schemas datatype, should be compliant now except for + binHex and base64 which are not supported yet.</li> + <li>bug fixes: non-ASCII IDs, HTML output, XInclude on large docs and + XInclude entities handling, encoding detection on external subsets, XML + Schemas bugs and memory leaks, HTML parser (James Bursa)</li> + <li>portability: python/trio (Albert Chin), Sun compiler warnings</li> + <li>documentation: added --relaxng option to xmllint man page (John)</li> + <li>improved error reporting: xml:space, start/end tag mismatches, Relax NG + errors</li> +</ul> + +<h3>2.5.5: Mar 24 2003</h3> +<ul> + <li>Lot of fixes on the Relax NG implementation. More testing including + DocBook and TEI examples.</li> + <li>Increased the support for W3C XML Schemas datatype</li> + <li>Several bug fixes in the URI handling layer</li> + <li>Bug fixes: HTML parser, xmlReader, DTD validation, XPath, encoding + conversion, line counting in the parser.</li> + <li>Added support for $XMLLINT_INDENT environment variable, FTP delete</li> + <li>Fixed the RPM spec file name</li> +</ul> + +<h3>2.5.4: Feb 20 2003</h3> +<ul> + <li>Conformance testing and lot of fixes on Relax NG and XInclude + implementation</li> + <li>Implementation of XPointer element() scheme</li> + <li>Bug fixes: XML parser, XInclude entities merge, validity checking on + namespaces, + <p>2 serialization bugs, node info generation problems, a DTD regexp + generation problem.</p> + </li> + <li>Portability: windows updates and path canonicalization (Igor)</li> + <li>A few typo fixes (Kjartan Maraas)</li> + <li>Python bindings generator fixes (Stephane Bidoul)</li> +</ul> + +<h3>2.5.3: Feb 10 2003</h3> +<ul> + <li>RelaxNG and XML Schemas datatypes improvements, and added a first + version of RelaxNG Python bindings</li> + <li>Fixes: XLink (Sean Chittenden), XInclude (Sean Chittenden), API fix for + serializing namespace nodes, encoding conversion bug, XHTML1 + serialization</li> + <li>Portability fixes: Windows (Igor), AMD 64bits RPM spec file</li> +</ul> + +<h3>2.5.2: Feb 5 2003</h3> +<ul> + <li>First implementation of RelaxNG, added --relaxng flag to xmllint</li> + <li>Schemas support now compiled in by default.</li> + <li>Bug fixes: DTD validation, namespace checking, XInclude and entities, + delegateURI in XML Catalogs, HTML parser, XML reader (Stéphane Bidoul), + XPath parser and evaluation, UTF8ToUTF8 serialization, XML reader memory + consumption, HTML parser, HTML serialization in the presence of + namespaces</li> + <li>added an HTML API to check elements and attributes.</li> + <li>Documentation improvement, PDF for the tutorial (John Fleck), doc + patches (Stefan Kost)</li> + <li>Portability fixes: NetBSD (Julio Merino), Windows (Igor Zlatkovic)</li> + <li>Added python bindings for XPointer, contextual error reporting + (Stéphane Bidoul)</li> + <li>URI/file escaping problems (Stefano Zacchiroli)</li> +</ul> + +<h3>2.5.1: Jan 8 2003</h3> +<ul> + <li>Fixes a memory leak and configuration/compilation problems in 2.5.0</li> + <li>documentation updates (John)</li> + <li>a couple of XmlTextReader fixes</li> +</ul> + +<h3>2.5.0: Jan 6 2003</h3> +<ul> + <li>New <a href="xmlreader.html">XmltextReader interface</a> based on C# + API (with help of Stéphane Bidoul)</li> + <li>Windows: more exports, including the new API (Igor)</li> + <li>XInclude fallback fix</li> + <li>Python: bindings for the new API, packaging (Stéphane Bidoul), + drv_libxml2.py Python xml.sax driver (Stéphane Bidoul), fixes, speedup + and iterators for Python-2.2 (Hannu Krosing)</li> + <li>Tutorial fixes (john Fleck and Niraj Tolia) xmllint man update + (John)</li> + <li>Fix an XML parser bug raised by Vyacheslav Pindyura</li> + <li>Fix for VMS serialization (Nigel Hall) and config (Craig A. Berry)</li> + <li>Entities handling fixes</li> + <li>new API to optionally track node creation and deletion (Lukas + Schroeder)</li> + <li>Added documentation for the XmltextReader interface and some <a + href="guidelines.html">XML guidelines</a></li> +</ul> + +<h3>2.4.30: Dec 12 2002</h3> +<ul> + <li>2.4.29 broke the python bindings, rereleasing</li> + <li>Improvement/fixes of the XML API generator, and couple of minor code + fixes.</li> +</ul> + +<h3>2.4.29: Dec 11 2002</h3> +<ul> + <li>Windows fixes (Igor): Windows CE port, pthread linking, python bindings + (Stéphane Bidoul), Mingw (Magnus Henoch), and export list updates</li> + <li>Fix for prev in python bindings (ERDI Gergo)</li> + <li>Fix for entities handling (Marcus Clarke)</li> + <li>Refactored the XML and HTML dumps to a single code path, fixed XHTML1 + dump</li> + <li>Fix for URI parsing when handling URNs with fragment identifiers</li> + <li>Fix for HTTP URL escaping problem</li> + <li>added an TextXmlReader (C#) like API (work in progress)</li> + <li>Rewrote the API in XML generation script, includes a C parser and saves + more informations needed for C# bindings</li> +</ul> + +<h3>2.4.28: Nov 22 2002</h3> +<ul> + <li>a couple of python binding fixes</li> + <li>2 bug fixes in the XML push parser</li> + <li>potential memory leak removed (Martin Stoilov)</li> + <li>fix to the configure script for Unix (Dimitri Papadopoulos)</li> + <li>added encoding support for XInclude parse="text"</li> + <li>autodetection of XHTML1 and specific serialization rules added</li> + <li>nasty threading bug fixed (William Brack)</li> +</ul> + +<h3>2.4.27: Nov 17 2002</h3> +<ul> + <li>fixes for the Python bindings</li> + <li>a number of bug fixes: SGML catalogs, xmlParseBalancedChunkMemory(), + HTML parser, Schemas (Charles Bozeman), document fragment support + (Christian Glahn), xmlReconciliateNs (Brian Stafford), XPointer, + xmlFreeNode(), xmlSAXParseMemory (Peter Jones), xmlGetNodePath (Petr + Pajas), entities processing</li> + <li>added grep to xmllint --shell</li> + <li>VMS update patch from Craig A. Berry</li> + <li>cleanup of the Windows build with support for more compilers (Igor), + better thread support on Windows</li> + <li>cleanup of Unix Makefiles and spec file</li> + <li>Improvements to the documentation (John Fleck)</li> +</ul> + +<h3>2.4.26: Oct 18 2002</h3> +<ul> + <li>Patches for Windows CE port, improvements on Windows paths handling</li> + <li>Fixes to the validation code (DTD and Schemas), xmlNodeGetPath() , + HTML serialization, Namespace compliance, and a number of small + problems</li> +</ul> + +<h3>2.4.25: Sep 26 2002</h3> +<ul> + <li>A number of bug fixes: XPath, validation, Python bindings, DOM and + tree, xmlI/O, Html</li> + <li>Serious rewrite of XInclude</li> + <li>Made XML Schemas regexp part of the default build and APIs, small fix + and improvement of the regexp core</li> + <li>Changed the validation code to reuse XML Schemas regexp APIs</li> + <li>Better handling of Windows file paths, improvement of Makefiles (Igor, + Daniel Gehriger, Mark Vakoc)</li> + <li>Improved the python I/O bindings, the tests, added resolver and regexp + APIs</li> + <li>New logos from Marc Liyanage</li> + <li>Tutorial improvements: John Fleck, Christopher Harris</li> + <li>Makefile: Fixes for AMD x86_64 (Mandrake), DESTDIR (Christophe + Merlet)</li> + <li>removal of all stderr/perror use for error reporting</li> + <li>Better error reporting: XPath and DTD validation</li> + <li>update of the trio portability layer (Bjorn Reese)</li> +</ul> + +<p><strong>2.4.24: Aug 22 2002</strong></p> +<ul> + <li>XPath fixes (William), xf:escape-uri() (Wesley Terpstra)</li> + <li>Python binding fixes: makefiles (William), generator, rpm build, x86-64 + (fcrozat)</li> + <li>HTML <style> and boolean attributes serializer fixes</li> + <li>C14N improvements by Aleksey</li> + <li>doc cleanups: Rick Jones</li> + <li>Windows compiler makefile updates: Igor and Elizabeth Barham</li> + <li>XInclude: implementation of fallback and xml:base fixup added</li> +</ul> + +<h3>2.4.23: July 6 2002</h3> +<ul> + <li>performances patches: Peter Jacobi</li> + <li>c14n fixes, testsuite and performances: Aleksey Sanin</li> + <li>added xmlDocFormatDump: Chema Celorio</li> + <li>new tutorial: John Fleck</li> + <li>new hash functions and performances: Sander Vesik, portability fix from + Peter Jacobi</li> + <li>a number of bug fixes: XPath (William Brack, Richard Jinks), XML and + HTML parsers, ID lookup function</li> + <li>removal of all remaining sprintf: Aleksey Sanin</li> +</ul> + +<h3>2.4.22: May 27 2002</h3> +<ul> + <li>a number of bug fixes: configure scripts, base handling, parser, memory + usage, HTML parser, XPath, documentation (Christian Cornelssen), + indentation, URI parsing</li> + <li>Optimizations for XMLSec, fixing and making public some of the network + protocol handlers (Aleksey)</li> + <li>performance patch from Gary Pennington</li> + <li>Charles Bozeman provided date and time support for XML Schemas + datatypes</li> +</ul> + +<h3>2.4.21: Apr 29 2002</h3> + +<p>This release is both a bug fix release and also contains the early XML +Schemas <a href="http://www.w3.org/TR/xmlschema-1/">structures</a> and <a +href="http://www.w3.org/TR/xmlschema-2/">datatypes</a> code, beware, all +interfaces are likely to change, there is huge holes, it is clearly a work in +progress and don't even think of putting this code in a production system, +it's actually not compiled in by default. The real fixes are:</p> +<ul> + <li>a couple of bugs or limitations introduced in 2.4.20</li> + <li>patches for Borland C++ and MSC by Igor</li> + <li>some fixes on XPath strings and conformance patches by Richard + Jinks</li> + <li>patch from Aleksey for the ExcC14N specification</li> + <li>OSF/1 bug fix by Bjorn</li> +</ul> + +<h3>2.4.20: Apr 15 2002</h3> +<ul> + <li>bug fixes: file descriptor leak, XPath, HTML output, DTD validation</li> + <li>XPath conformance testing by Richard Jinks</li> + <li>Portability fixes: Solaris, MPE/iX, Windows, OSF/1, python bindings, + libxml.m4</li> +</ul> + +<h3>2.4.19: Mar 25 2002</h3> +<ul> + <li>bug fixes: half a dozen XPath bugs, Validation, ISO-Latin to UTF8 + encoder</li> + <li>portability fixes in the HTTP code</li> + <li>memory allocation checks using valgrind, and profiling tests</li> + <li>revamp of the Windows build and Makefiles</li> +</ul> + +<h3>2.4.18: Mar 18 2002</h3> +<ul> + <li>bug fixes: tree, SAX, canonicalization, validation, portability, + XPath</li> + <li>removed the --with-buffer option it was becoming unmaintainable</li> + <li>serious cleanup of the Python makefiles</li> + <li>speedup patch to XPath very effective for DocBook stylesheets</li> + <li>Fixes for Windows build, cleanup of the documentation</li> +</ul> + +<h3>2.4.17: Mar 8 2002</h3> +<ul> + <li>a lot of bug fixes, including "namespace nodes have no parents in + XPath"</li> + <li>fixed/improved the Python wrappers, added more examples and more + regression tests, XPath extension functions can now return node-sets</li> + <li>added the XML Canonicalization support from Aleksey Sanin</li> +</ul> + +<h3>2.4.16: Feb 20 2002</h3> +<ul> + <li>a lot of bug fixes, most of them were triggered by the XML Testsuite + from OASIS and W3C. Compliance has been significantly improved.</li> + <li>a couple of portability fixes too.</li> +</ul> + +<h3>2.4.15: Feb 11 2002</h3> +<ul> + <li>Fixed the Makefiles, especially the python module ones</li> + <li>A few bug fixes and cleanup</li> + <li>Includes cleanup</li> +</ul> + +<h3>2.4.14: Feb 8 2002</h3> +<ul> + <li>Change of License to the <a + href="http://www.opensource.org/licenses/mit-license.html">MIT + License</a> basically for integration in XFree86 codebase, and removing + confusion around the previous dual-licensing</li> + <li>added Python bindings, beta software but should already be quite + complete</li> + <li>a large number of fixes and cleanups, especially for all tree + manipulations</li> + <li>cleanup of the headers, generation of a reference API definition in + XML</li> +</ul> + +<h3>2.4.13: Jan 14 2002</h3> +<ul> + <li>update of the documentation: John Fleck and Charlie Bozeman</li> + <li>cleanup of timing code from Justin Fletcher</li> + <li>fixes for Windows and initial thread support on Win32: Igor and Serguei + Narojnyi</li> + <li>Cygwin patch from Robert Collins</li> + <li>added xmlSetEntityReferenceFunc() for Keith Isdale work on xsldbg</li> +</ul> + +<h3>2.4.12: Dec 7 2001</h3> +<ul> + <li>a few bug fixes: thread (Gary Pennington), xmllint (Geert Kloosterman), + XML parser (Robin Berjon), XPointer (Danny Jamshy), I/O cleanups + (robert)</li> + <li>Eric Lavigne contributed project files for MacOS</li> + <li>some makefiles cleanups</li> +</ul> + +<h3>2.4.11: Nov 26 2001</h3> +<ul> + <li>fixed a couple of errors in the includes, fixed a few bugs, some code + cleanups</li> + <li>xmllint man pages improvement by Heiko Rupp</li> + <li>updated VMS build instructions from John A Fotheringham</li> + <li>Windows Makefiles updates from Igor</li> +</ul> + +<h3>2.4.10: Nov 10 2001</h3> +<ul> + <li>URI escaping fix (Joel Young)</li> + <li>added xmlGetNodePath() (for paths or XPointers generation)</li> + <li>Fixes namespace handling problems when using DTD and validation</li> + <li>improvements on xmllint: Morus Walter patches for --format and + --encode, Stefan Kost and Heiko Rupp improvements on the --shell</li> + <li>fixes for xmlcatalog linking pointed by Weiqi Gao</li> + <li>fixes to the HTML parser</li> +</ul> + +<h3>2.4.9: Nov 6 2001</h3> +<ul> + <li>fixes more catalog bugs</li> + <li>avoid a compilation problem, improve xmlGetLineNo()</li> +</ul> + +<h3>2.4.8: Nov 4 2001</h3> +<ul> + <li>fixed SGML catalogs broken in previous release, updated xmlcatalog + tool</li> + <li>fixed a compile errors and some includes troubles.</li> +</ul> + +<h3>2.4.7: Oct 30 2001</h3> +<ul> + <li>exported some debugging interfaces</li> + <li>serious rewrite of the catalog code</li> + <li>integrated Gary Pennington thread safety patch, added configure option + and regression tests</li> + <li>removed an HTML parser bug</li> + <li>fixed a couple of potentially serious validation bugs</li> + <li>integrated the SGML DocBook support in xmllint</li> + <li>changed the nanoftp anonymous login passwd</li> + <li>some I/O cleanup and a couple of interfaces for Perl wrapper</li> + <li>general bug fixes</li> + <li>updated xmllint man page by John Fleck</li> + <li>some VMS and Windows updates</li> +</ul> + +<h3>2.4.6: Oct 10 2001</h3> +<ul> + <li>added an updated man pages by John Fleck</li> + <li>portability and configure fixes</li> + <li>an infinite loop on the HTML parser was removed (William)</li> + <li>Windows makefile patches from Igor</li> + <li>fixed half a dozen bugs reported for libxml or libxslt</li> + <li>updated xmlcatalog to be able to modify SGML super catalogs</li> +</ul> + +<h3>2.4.5: Sep 14 2001</h3> +<ul> + <li>Remove a few annoying bugs in 2.4.4</li> + <li>forces the HTML serializer to output decimal charrefs since some + version of Netscape can't handle hexadecimal ones</li> +</ul> + +<h3>1.8.16: Sep 14 2001</h3> +<ul> + <li>maintenance release of the old libxml1 branch, couple of bug and + portability fixes</li> +</ul> + +<h3>2.4.4: Sep 12 2001</h3> +<ul> + <li>added --convert to xmlcatalog, bug fixes and cleanups of XML + Catalog</li> + <li>a few bug fixes and some portability changes</li> + <li>some documentation cleanups</li> +</ul> + +<h3>2.4.3: Aug 23 2001</h3> +<ul> + <li>XML Catalog support see the doc</li> + <li>New NaN/Infinity floating point code</li> + <li>A few bug fixes</li> +</ul> + +<h3>2.4.2: Aug 15 2001</h3> +<ul> + <li>adds xmlLineNumbersDefault() to control line number generation</li> + <li>lot of bug fixes</li> + <li>the Microsoft MSC projects files should now be up to date</li> + <li>inheritance of namespaces from DTD defaulted attributes</li> + <li>fixes a serious potential security bug</li> + <li>added a --format option to xmllint</li> +</ul> + +<h3>2.4.1: July 24 2001</h3> +<ul> + <li>possibility to keep line numbers in the tree</li> + <li>some computation NaN fixes</li> + <li>extension of the XPath API</li> + <li>cleanup for alpha and ia64 targets</li> + <li>patch to allow saving through HTTP PUT or POST</li> +</ul> + +<h3>2.4.0: July 10 2001</h3> +<ul> + <li>Fixed a few bugs in XPath, validation, and tree handling.</li> + <li>Fixed XML Base implementation, added a couple of examples to the + regression tests</li> + <li>A bit of cleanup</li> +</ul> + +<h3>2.3.14: July 5 2001</h3> +<ul> + <li>fixed some entities problems and reduce memory requirement when + substituting them</li> + <li>lots of improvements in the XPath queries interpreter can be + substantially faster</li> + <li>Makefiles and configure cleanups</li> + <li>Fixes to XPath variable eval, and compare on empty node set</li> + <li>HTML tag closing bug fixed</li> + <li>Fixed an URI reference computation problem when validating</li> +</ul> + +<h3>2.3.13: June 28 2001</h3> +<ul> + <li>2.3.12 configure.in was broken as well as the push mode XML parser</li> + <li>a few more fixes for compilation on Windows MSC by Yon Derek</li> +</ul> + +<h3>1.8.14: June 28 2001</h3> +<ul> + <li>Zbigniew Chyla gave a patch to use the old XML parser in push mode</li> + <li>Small Makefile fix</li> +</ul> + +<h3>2.3.12: June 26 2001</h3> +<ul> + <li>lots of cleanup</li> + <li>a couple of validation fix</li> + <li>fixed line number counting</li> + <li>fixed serious problems in the XInclude processing</li> + <li>added support for UTF8 BOM at beginning of entities</li> + <li>fixed a strange gcc optimizer bugs in xpath handling of float, gcc-3.0 + miscompile uri.c (William), Thomas Leitner provided a fix for the + optimizer on Tru64</li> + <li>incorporated Yon Derek and Igor Zlatkovic fixes and improvements for + compilation on Windows MSC</li> + <li>update of libxml-doc.el (Felix Natter)</li> + <li>fixed 2 bugs in URI normalization code</li> +</ul> + +<h3>2.3.11: June 17 2001</h3> +<ul> + <li>updates to trio, Makefiles and configure should fix some portability + problems (alpha)</li> + <li>fixed some HTML serialization problems (pre, script, and block/inline + handling), added encoding aware APIs, cleanup of this code</li> + <li>added xmlHasNsProp()</li> + <li>implemented a specific PI for encoding support in the DocBook SGML + parser</li> + <li>some XPath fixes (-Infinity, / as a function parameter and namespaces + node selection)</li> + <li>fixed a performance problem and an error in the validation code</li> + <li>fixed XInclude routine to implement the recursive behaviour</li> + <li>fixed xmlFreeNode problem when libxml is included statically twice</li> + <li>added --version to xmllint for bug reports</li> +</ul> + +<h3>2.3.10: June 1 2001</h3> +<ul> + <li>fixed the SGML catalog support</li> + <li>a number of reported bugs got fixed, in XPath, iconv detection, + XInclude processing</li> + <li>XPath string function should now handle unicode correctly</li> +</ul> + +<h3>2.3.9: May 19 2001</h3> + +<p>Lots of bugfixes, and added a basic SGML catalog support:</p> +<ul> + <li>HTML push bugfix #54891 and another patch from Jonas Borgström</li> + <li>some serious speed optimization again</li> + <li>some documentation cleanups</li> + <li>trying to get better linking on Solaris (-R)</li> + <li>XPath API cleanup from Thomas Broyer</li> + <li>Validation bug fixed #54631, added a patch from Gary Pennington, fixed + xmlValidGetValidElements()</li> + <li>Added an INSTALL file</li> + <li>Attribute removal added to API: #54433</li> + <li>added a basic support for SGML catalogs</li> + <li>fixed xmlKeepBlanksDefault(0) API</li> + <li>bugfix in xmlNodeGetLang()</li> + <li>fixed a small configure portability problem</li> + <li>fixed an inversion of SYSTEM and PUBLIC identifier in HTML document</li> +</ul> + +<h3>1.8.13: May 14 2001</h3> +<ul> + <li>bugfixes release of the old libxml1 branch used by Gnome</li> +</ul> + +<h3>2.3.8: May 3 2001</h3> +<ul> + <li>Integrated an SGML DocBook parser for the Gnome project</li> + <li>Fixed a few things in the HTML parser</li> + <li>Fixed some XPath bugs raised by XSLT use, tried to fix the floating + point portability issue</li> + <li>Speed improvement (8M/s for SAX, 3M/s for DOM, 1.5M/s for + DOM+validation using the XML REC as input and a 700MHz celeron).</li> + <li>incorporated more Windows cleanup</li> + <li>added xmlSaveFormatFile()</li> + <li>fixed problems in copying nodes with entities references (gdome)</li> + <li>removed some troubles surrounding the new validation module</li> +</ul> + +<h3>2.3.7: April 22 2001</h3> +<ul> + <li>lots of small bug fixes, corrected XPointer</li> + <li>Non deterministic content model validation support</li> + <li>added xmlDocCopyNode for gdome2</li> + <li>revamped the way the HTML parser handles end of tags</li> + <li>XPath: corrections of namespaces support and number formatting</li> + <li>Windows: Igor Zlatkovic patches for MSC compilation</li> + <li>HTML output fixes from P C Chow and William M. Brack</li> + <li>Improved validation speed sensible for DocBook</li> + <li>fixed a big bug with ID declared in external parsed entities</li> + <li>portability fixes, update of Trio from Bjorn Reese</li> +</ul> + +<h3>2.3.6: April 8 2001</h3> +<ul> + <li>Code cleanup using extreme gcc compiler warning options, found and + cleared half a dozen potential problem</li> + <li>the Eazel team found an XML parser bug</li> + <li>cleaned up the user of some of the string formatting function. used the + trio library code to provide the one needed when the platform is missing + them</li> + <li>xpath: removed a memory leak and fixed the predicate evaluation + problem, extended the testsuite and cleaned up the result. XPointer seems + broken ...</li> +</ul> + +<h3>2.3.5: Mar 23 2001</h3> +<ul> + <li>Biggest change is separate parsing and evaluation of XPath expressions, + there is some new APIs for this too</li> + <li>included a number of bug fixes(XML push parser, 51876, notations, + 52299)</li> + <li>Fixed some portability issues</li> +</ul> + +<h3>2.3.4: Mar 10 2001</h3> +<ul> + <li>Fixed bugs #51860 and #51861</li> + <li>Added a global variable xmlDefaultBufferSize to allow default buffer + size to be application tunable.</li> + <li>Some cleanup in the validation code, still a bug left and this part + should probably be rewritten to support ambiguous content model :-\</li> + <li>Fix a couple of serious bugs introduced or raised by changes in 2.3.3 + parser</li> + <li>Fixed another bug in xmlNodeGetContent()</li> + <li>Bjorn fixed XPath node collection and Number formatting</li> + <li>Fixed a loop reported in the HTML parsing</li> + <li>blank space are reported even if the Dtd content model proves that they + are formatting spaces, this is for XML conformance</li> +</ul> + +<h3>2.3.3: Mar 1 2001</h3> +<ul> + <li>small change in XPath for XSLT</li> + <li>documentation cleanups</li> + <li>fix in validation by Gary Pennington</li> + <li>serious parsing performances improvements</li> +</ul> + +<h3>2.3.2: Feb 24 2001</h3> +<ul> + <li>chasing XPath bugs, found a bunch, completed some TODO</li> + <li>fixed a Dtd parsing bug</li> + <li>fixed a bug in xmlNodeGetContent</li> + <li>ID/IDREF support partly rewritten by Gary Pennington</li> +</ul> + +<h3>2.3.1: Feb 15 2001</h3> +<ul> + <li>some XPath and HTML bug fixes for XSLT</li> + <li>small extension of the hash table interfaces for DOM gdome2 + implementation</li> + <li>A few bug fixes</li> +</ul> + +<h3>2.3.0: Feb 8 2001 (2.2.12 was on 25 Jan but I didn't kept track)</h3> +<ul> + <li>Lots of XPath bug fixes</li> + <li>Add a mode with Dtd lookup but without validation error reporting for + XSLT</li> + <li>Add support for text node without escaping (XSLT)</li> + <li>bug fixes for xmlCheckFilename</li> + <li>validation code bug fixes from Gary Pennington</li> + <li>Patch from Paul D. Smith correcting URI path normalization</li> + <li>Patch to allow simultaneous install of libxml-devel and + libxml2-devel</li> + <li>the example Makefile is now fixed</li> + <li>added HTML to the RPM packages</li> + <li>tree copying bugfixes</li> + <li>updates to Windows makefiles</li> + <li>optimization patch from Bjorn Reese</li> +</ul> + +<h3>2.2.11: Jan 4 2001</h3> +<ul> + <li>bunch of bug fixes (memory I/O, xpath, ftp/http, ...)</li> + <li>added htmlHandleOmittedElem()</li> + <li>Applied Bjorn Reese's IPV6 first patch</li> + <li>Applied Paul D. Smith patches for validation of XInclude results</li> + <li>added XPointer xmlns() new scheme support</li> +</ul> + +<h3>2.2.10: Nov 25 2000</h3> +<ul> + <li>Fix the Windows problems of 2.2.8</li> + <li>integrate OpenVMS patches</li> + <li>better handling of some nasty HTML input</li> + <li>Improved the XPointer implementation</li> + <li>integrate a number of provided patches</li> +</ul> + +<h3>2.2.9: Nov 25 2000</h3> +<ul> + <li>erroneous release :-(</li> +</ul> + +<h3>2.2.8: Nov 13 2000</h3> +<ul> + <li>First version of <a href="http://www.w3.org/TR/xinclude">XInclude</a> + support</li> + <li>Patch in conditional section handling</li> + <li>updated MS compiler project</li> + <li>fixed some XPath problems</li> + <li>added an URI escaping function</li> + <li>some other bug fixes</li> +</ul> + +<h3>2.2.7: Oct 31 2000</h3> +<ul> + <li>added message redirection</li> + <li>XPath improvements (thanks TOM !)</li> + <li>xmlIOParseDTD() added</li> + <li>various small fixes in the HTML, URI, HTTP and XPointer support</li> + <li>some cleanup of the Makefile, autoconf and the distribution content</li> +</ul> + +<h3>2.2.6: Oct 25 2000:</h3> +<ul> + <li>Added an hash table module, migrated a number of internal structure to + those</li> + <li>Fixed a posteriori validation problems</li> + <li>HTTP module cleanups</li> + <li>HTML parser improvements (tag errors, script/style handling, attribute + normalization)</li> + <li>coalescing of adjacent text nodes</li> + <li>couple of XPath bug fixes, exported the internal API</li> +</ul> + +<h3>2.2.5: Oct 15 2000:</h3> +<ul> + <li>XPointer implementation and testsuite</li> + <li>Lot of XPath fixes, added variable and functions registration, more + tests</li> + <li>Portability fixes, lots of enhancements toward an easy Windows build + and release</li> + <li>Late validation fixes</li> + <li>Integrated a lot of contributed patches</li> + <li>added memory management docs</li> + <li>a performance problem when using large buffer seems fixed</li> +</ul> + +<h3>2.2.4: Oct 1 2000:</h3> +<ul> + <li>main XPath problem fixed</li> + <li>Integrated portability patches for Windows</li> + <li>Serious bug fixes on the URI and HTML code</li> +</ul> + +<h3>2.2.3: Sep 17 2000</h3> +<ul> + <li>bug fixes</li> + <li>cleanup of entity handling code</li> + <li>overall review of all loops in the parsers, all sprintf usage has been + checked too</li> + <li>Far better handling of larges Dtd. Validating against DocBook XML Dtd + works smoothly now.</li> +</ul> + +<h3>1.8.10: Sep 6 2000</h3> +<ul> + <li>bug fix release for some Gnome projects</li> +</ul> + +<h3>2.2.2: August 12 2000</h3> +<ul> + <li>mostly bug fixes</li> + <li>started adding routines to access xml parser context options</li> +</ul> + +<h3>2.2.1: July 21 2000</h3> +<ul> + <li>a purely bug fixes release</li> + <li>fixed an encoding support problem when parsing from a memory block</li> + <li>fixed a DOCTYPE parsing problem</li> + <li>removed a bug in the function allowing to override the memory + allocation routines</li> +</ul> + +<h3>2.2.0: July 14 2000</h3> +<ul> + <li>applied a lot of portability fixes</li> + <li>better encoding support/cleanup and saving (content is now always + encoded in UTF-8)</li> + <li>the HTML parser now correctly handles encodings</li> + <li>added xmlHasProp()</li> + <li>fixed a serious problem with &#38;</li> + <li>propagated the fix to FTP client</li> + <li>cleanup, bugfixes, etc ...</li> + <li>Added a page about <a href="encoding.html">libxml Internationalization + support</a></li> +</ul> + +<h3>1.8.9: July 9 2000</h3> +<ul> + <li>fixed the spec the RPMs should be better</li> + <li>fixed a serious bug in the FTP implementation, released 1.8.9 to solve + rpmfind users problem</li> +</ul> + +<h3>2.1.1: July 1 2000</h3> +<ul> + <li>fixes a couple of bugs in the 2.1.0 packaging</li> + <li>improvements on the HTML parser</li> +</ul> + +<h3>2.1.0 and 1.8.8: June 29 2000</h3> +<ul> + <li>1.8.8 is mostly a commodity package for upgrading to libxml2 according + to <a href="upgrade.html">new instructions</a>. It fixes a nasty problem + about &#38; charref parsing</li> + <li>2.1.0 also ease the upgrade from libxml v1 to the recent version. it + also contains numerous fixes and enhancements: + <ul> + <li>added xmlStopParser() to stop parsing</li> + <li>improved a lot parsing speed when there is large CDATA blocs</li> + <li>includes XPath patches provided by Picdar Technology</li> + <li>tried to fix as much as possible DTD validation and namespace + related problems</li> + <li>output to a given encoding has been added/tested</li> + <li>lot of various fixes</li> + </ul> + </li> +</ul> + +<h3>2.0.0: Apr 12 2000</h3> +<ul> + <li>First public release of libxml2. If you are using libxml, it's a good + idea to check the 1.x to 2.x upgrade instructions. NOTE: while initially + scheduled for Apr 3 the release occurred only on Apr 12 due to massive + workload.</li> + <li>The include are now located under $prefix/include/libxml (instead of + $prefix/include/gnome-xml), they also are referenced by + <pre>#include <libxml/xxx.h></pre> + <p>instead of</p> + <pre>#include "xxx.h"</pre> + </li> + <li>a new URI module for parsing URIs and following strictly RFC 2396</li> + <li>the memory allocation routines used by libxml can now be overloaded + dynamically by using xmlMemSetup()</li> + <li>The previously CVS only tool tester has been renamed + <strong>xmllint</strong> and is now installed as part of the libxml2 + package</li> + <li>The I/O interface has been revamped. There is now ways to plug in + specific I/O modules, either at the URI scheme detection level using + xmlRegisterInputCallbacks() or by passing I/O functions when creating a + parser context using xmlCreateIOParserCtxt()</li> + <li>there is a C preprocessor macro LIBXML_VERSION providing the version + number of the libxml module in use</li> + <li>a number of optional features of libxml can now be excluded at + configure time (FTP/HTTP/HTML/XPath/Debug)</li> +</ul> + +<h3>2.0.0beta: Mar 14 2000</h3> +<ul> + <li>This is a first Beta release of libxml version 2</li> + <li>It's available only from<a href="ftp://xmlsoft.org/">xmlsoft.org + FTP</a>, it's packaged as libxml2-2.0.0beta and available as tar and + RPMs</li> + <li>This version is now the head in the Gnome CVS base, the old one is + available under the tag LIB_XML_1_X</li> + <li>This includes a very large set of changes. From a programmatic point + of view applications should not have to be modified too much, check the + <a href="upgrade.html">upgrade page</a></li> + <li>Some interfaces may changes (especially a bit about encoding).</li> + <li>the updates includes: + <ul> + <li>fix I18N support. ISO-Latin-x/UTF-8/UTF-16 (nearly) seems correctly + handled now</li> + <li>Better handling of entities, especially well-formedness checking + and proper PEref extensions in external subsets</li> + <li>DTD conditional sections</li> + <li>Validation now correctly handle entities content</li> + <li><a href="http://rpmfind.net/tools/gdome/messages/0039.html">change + structures to accommodate DOM</a></li> + </ul> + </li> + <li>Serious progress were made toward compliance, <a + href="conf/result.html">here are the result of the test</a> against the + OASIS testsuite (except the Japanese tests since I don't support that + encoding yet). This URL is rebuilt every couple of hours using the CVS + head version.</li> +</ul> + +<h3>1.8.7: Mar 6 2000</h3> +<ul> + <li>This is a bug fix release:</li> + <li>It is possible to disable the ignorable blanks heuristic used by + libxml-1.x, a new function xmlKeepBlanksDefault(0) will allow this. Note + that for adherence to XML spec, this behaviour will be disabled by + default in 2.x . The same function will allow to keep compatibility for + old code.</li> + <li>Blanks in <a> </a> constructs are not ignored anymore, + avoiding heuristic is really the Right Way :-\</li> + <li>The unchecked use of snprintf which was breaking libxml-1.8.6 + compilation on some platforms has been fixed</li> + <li>nanoftp.c nanohttp.c: Fixed '#' and '?' stripping when processing + URIs</li> +</ul> + +<h3>1.8.6: Jan 31 2000</h3> +<ul> + <li>added a nanoFTP transport module, debugged until the new version of <a + href="http://rpmfind.net/linux/rpm2html/rpmfind.html">rpmfind</a> can use + it without troubles</li> +</ul> + +<h3>1.8.5: Jan 21 2000</h3> +<ul> + <li>adding APIs to parse a well balanced chunk of XML (production <a + href="http://www.w3.org/TR/REC-xml#NT-content">[43] content</a> of the + XML spec)</li> + <li>fixed a hideous bug in xmlGetProp pointed by Rune.Djurhuus@fast.no</li> + <li>Jody Goldberg <jgoldberg@home.com> provided another patch trying + to solve the zlib checks problems</li> + <li>The current state in gnome CVS base is expected to ship as 1.8.5 with + gnumeric soon</li> +</ul> + +<h3>1.8.4: Jan 13 2000</h3> +<ul> + <li>bug fixes, reintroduced xmlNewGlobalNs(), fixed xmlNewNs()</li> + <li>all exit() call should have been removed from libxml</li> + <li>fixed a problem with INCLUDE_WINSOCK on WIN32 platform</li> + <li>added newDocFragment()</li> +</ul> + +<h3>1.8.3: Jan 5 2000</h3> +<ul> + <li>a Push interface for the XML and HTML parsers</li> + <li>a shell-like interface to the document tree (try tester --shell :-)</li> + <li>lots of bug fixes and improvement added over XMas holidays</li> + <li>fixed the DTD parsing code to work with the xhtml DTD</li> + <li>added xmlRemoveProp(), xmlRemoveID() and xmlRemoveRef()</li> + <li>Fixed bugs in xmlNewNs()</li> + <li>External entity loading code has been revamped, now it uses + xmlLoadExternalEntity(), some fix on entities processing were added</li> + <li>cleaned up WIN32 includes of socket stuff</li> +</ul> + +<h3>1.8.2: Dec 21 1999</h3> +<ul> + <li>I got another problem with includes and C++, I hope this issue is fixed + for good this time</li> + <li>Added a few tree modification functions: xmlReplaceNode, + xmlAddPrevSibling, xmlAddNextSibling, xmlNodeSetName and + xmlDocSetRootElement</li> + <li>Tried to improve the HTML output with help from <a + href="mailto:clahey@umich.edu">Chris Lahey</a></li> +</ul> + +<h3>1.8.1: Dec 18 1999</h3> +<ul> + <li>various patches to avoid troubles when using libxml with C++ compilers + the "namespace" keyword and C escaping in include files</li> + <li>a problem in one of the core macros IS_CHAR was corrected</li> + <li>fixed a bug introduced in 1.8.0 breaking default namespace processing, + and more specifically the Dia application</li> + <li>fixed a posteriori validation (validation after parsing, or by using a + Dtd not specified in the original document)</li> + <li>fixed a bug in</li> +</ul> + +<h3>1.8.0: Dec 12 1999</h3> +<ul> + <li>cleanup, especially memory wise</li> + <li>the parser should be more reliable, especially the HTML one, it should + not crash, whatever the input !</li> + <li>Integrated various patches, especially a speedup improvement for large + dataset from <a href="mailto:cnygard@bellatlantic.net">Carl Nygard</a>, + configure with --with-buffers to enable them.</li> + <li>attribute normalization, oops should have been added long ago !</li> + <li>attributes defaulted from DTDs should be available, xmlSetProp() now + does entities escaping by default.</li> +</ul> + +<h3>1.7.4: Oct 25 1999</h3> +<ul> + <li>Lots of HTML improvement</li> + <li>Fixed some errors when saving both XML and HTML</li> + <li>More examples, the regression tests should now look clean</li> + <li>Fixed a bug with contiguous charref</li> +</ul> + +<h3>1.7.3: Sep 29 1999</h3> +<ul> + <li>portability problems fixed</li> + <li>snprintf was used unconditionally, leading to link problems on system + were it's not available, fixed</li> +</ul> + +<h3>1.7.1: Sep 24 1999</h3> +<ul> + <li>The basic type for strings manipulated by libxml has been renamed in + 1.7.1 from <strong>CHAR</strong> to <strong>xmlChar</strong>. The reason + is that CHAR was conflicting with a predefined type on Windows. However + on non WIN32 environment, compatibility is provided by the way of a + <strong>#define </strong>.</li> + <li>Changed another error : the use of a structure field called errno, and + leading to troubles on platforms where it's a macro</li> +</ul> + +<h3>1.7.0: Sep 23 1999</h3> +<ul> + <li>Added the ability to fetch remote DTD or parsed entities, see the <a + href="html/libxml-nanohttp.html">nanohttp</a> module.</li> + <li>Added an errno to report errors by another mean than a simple printf + like callback</li> + <li>Finished ID/IDREF support and checking when validation</li> + <li>Serious memory leaks fixed (there is now a <a + href="html/libxml-xmlmemory.html">memory wrapper</a> module)</li> + <li>Improvement of <a href="http://www.w3.org/TR/xpath">XPath</a> + implementation</li> + <li>Added an HTML parser front-end</li> +</ul> + +<h2><a name="XML">XML</a></h2> + +<p><a href="http://www.w3.org/TR/REC-xml">XML is a standard</a> for +markup-based structured documents. Here is <a name="example">an example XML +document</a>:</p> +<pre><?xml version="1.0"?> +<EXAMPLE prop1="gnome is great" prop2="&amp; linux too"> + <head> + <title>Welcome to Gnome</title> + </head> + <chapter> + <title>The Linux adventure</title> + <p>bla bla bla ...</p> + <image href="linus.gif"/> + <p>...</p> + </chapter> +</EXAMPLE></pre> + +<p>The first line specifies that it is an XML document and gives useful +information about its encoding. Then the rest of the document is a text +format whose structure is specified by tags between brackets. <strong>Each +tag opened has to be closed</strong>. XML is pedantic about this. However, if +a tag is empty (no content), a single tag can serve as both the opening and +closing tag if it ends with <code>/></code> rather than with +<code>></code>. Note that, for example, the image tag has no content (just +an attribute) and is closed by ending the tag with <code>/></code>.</p> + +<p>XML can be applied successfully to a wide range of tasks, ranging from +long term structured document maintenance (where it follows the steps of +SGML) to simple data encoding mechanisms like configuration file formatting +(glade), spreadsheets (gnumeric), or even shorter lived documents such as +WebDAV where it is used to encode remote calls between a client and a +server.</p> + +<h2><a name="XSLT">XSLT</a></h2> + +<p>Check <a href="http://xmlsoft.org/XSLT">the separate libxslt page</a></p> + +<p><a href="http://www.w3.org/TR/xslt">XSL Transformations</a>, is a +language for transforming XML documents into other XML documents (or +HTML/textual output).</p> + +<p>A separate library called libxslt is available implementing XSLT-1.0 for +libxml2. This module "libxslt" too can be found in the Gnome CVS base.</p> + +<p>You can check the <a +href="http://cvs.gnome.org/lxr/source/libxslt/FEATURES">features</a> +supported and the progresses on the <a +href="http://cvs.gnome.org/lxr/source/libxslt/ChangeLog" +name="Changelog">Changelog</a>.</p> + +<h2><a name="Python">Python and bindings</a></h2> + +<p>There are a number of language bindings and wrappers available for +libxml2, the list below is not exhaustive. Please contact the <a +href="http://mail.gnome.org/mailman/listinfo/xml-bindings">xml-bindings@gnome.org</a> +(<a href="http://mail.gnome.org/archives/xml-bindings/">archives</a>) in +order to get updates to this list or to discuss the specific topic of libxml2 +or libxslt wrappers or bindings:</p> +<ul> + <li><a href="http://libxmlplusplus.sourceforge.net/">Libxml++</a> seems the + most up-to-date C++ bindings for libxml2, check the <a + href="http://libxmlplusplus.sourceforge.net/reference/html/hierarchy.html">documentation</a> + and the <a + href="http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/libxmlplusplus/libxml%2b%2b/examples/">examples</a>.</li> + <li>There is another <a href="http://libgdome-cpp.berlios.de/">C++ wrapper + based on the gdome2 bindings</a> maintained by Tobias Peters.</li> + <li>and a third C++ wrapper by Peter Jones <pjones@pmade.org> + <p>Website: <a + href="http://pmade.org/pjones/software/xmlwrapp/">http://pmade.org/pjones/software/xmlwrapp/</a></p> + </li> + <li><a + href="http://mail.gnome.org/archives/xml/2001-March/msg00014.html">Matt + Sergeant</a> developed <a + href="http://axkit.org/download/">XML::LibXSLT</a>, a Perl wrapper for + libxml2/libxslt as part of the <a href="http://axkit.com/">AxKit XML + application server</a>.</li> + <li>If you're interested into scripting XML processing, have a look at <a + href="http://xsh.sourceforge.net/">XSH</a> an XML editing shell based on + Libxml2 Perl bindings.</li> + <li><a href="mailto:dkuhlman@cutter.rexx.com">Dave Kuhlman</a> provides an + earlier version of the libxml/libxslt <a + href="http://www.rexx.com/~dkuhlman">wrappers for Python</a>.</li> + <li>Gopal.V and Peter Minten develop <a + href="http://savannah.gnu.org/projects/libxmlsharp">libxml#</a>, a set of + C# libxml2 bindings.</li> + <li>Petr Kozelka provides <a + href="http://sourceforge.net/projects/libxml2-pas">Pascal units to glue + libxml2</a> with Kylix, Delphi and other Pascal compilers.</li> + <li>Uwe Fechner also provides <a + href="http://sourceforge.net/projects/idom2-pas/">idom2</a>, a DOM2 + implementation for Kylix2/D5/D6 from Borland.</li> + <li>Wai-Sun "Squidster" Chia provides <a + href="http://www.rubycolor.org/arc/redist/">bindings for Ruby</a> and + libxml2 bindings are also available in Ruby through the <a + href="http://libgdome-ruby.berlios.de/">libgdome-ruby</a> module + maintained by Tobias Peters.</li> + <li>Steve Ball and contributors maintains <a + href="http://tclxml.sourceforge.net/">libxml2 and libxslt bindings for + Tcl</a>.</li> + <li>There is support for libxml2 in the DOM module of PHP.</li> + <li><a href="http://savannah.gnu.org/projects/classpathx/">LibxmlJ</a> is + an effort to create a 100% JAXP-compatible Java wrapper for libxml2 and + libxslt as part of GNU ClasspathX project.</li> + <li>Patrick McPhee provides Rexx bindings fof libxml2 and libxslt, look for + <a href="http://www.interlog.com/~ptjm/software.html">RexxXML</a>.</li> +</ul> + +<p>The distribution includes a set of Python bindings, which are guaranteed +to be maintained as part of the library in the future, though the Python +interface have not yet reached the completeness of the C API.</p> + +<p><a href="mailto:stephane.bidoul@softwareag.com">Stéphane Bidoul</a> +maintains <a href="http://users.skynet.be/sbi/libxml-python/">a Windows port +of the Python bindings</a>.</p> + +<p>Note to people interested in building bindings, the API is formalized as +<a href="libxml2-api.xml">an XML API description file</a> which allows to +automate a large part of the Python bindings, this includes function +descriptions, enums, structures, typedefs, etc... The Python script used to +build the bindings is python/generator.py in the source distribution.</p> + +<p>To install the Python bindings there are 2 options:</p> +<ul> + <li>If you use an RPM based distribution, simply install the <a + href="http://rpmfind.net/linux/rpm2html/search.php?query=libxml2-python">libxml2-python + RPM</a> (and if needed the <a + href="http://rpmfind.net/linux/rpm2html/search.php?query=libxslt-python">libxslt-python + RPM</a>).</li> + <li>Otherwise use the <a href="ftp://xmlsoft.org/python/">libxml2-python + module distribution</a> corresponding to your installed version of + libxml2 and libxslt. Note that to install it you will need both libxml2 + and libxslt installed and run "python setup.py build install" in the + module tree.</li> +</ul> + +<p>The distribution includes a set of examples and regression tests for the +python bindings in the <code>python/tests</code> directory. Here are some +excerpts from those tests:</p> + +<h3>tst.py:</h3> + +<p>This is a basic test of the file interface and DOM navigation:</p> +<pre>import libxml2, sys + +doc = libxml2.parseFile("tst.xml") +if doc.name != "tst.xml": + print "doc.name failed" + sys.exit(1) +root = doc.children +if root.name != "doc": + print "root.name failed" + sys.exit(1) +child = root.children +if child.name != "foo": + print "child.name failed" + sys.exit(1) +doc.freeDoc()</pre> + +<p>The Python module is called libxml2; parseFile is the equivalent of +xmlParseFile (most of the bindings are automatically generated, and the xml +prefix is removed and the casing convention are kept). All node seen at the +binding level share the same subset of accessors:</p> +<ul> + <li><code>name</code> : returns the node name</li> + <li><code>type</code> : returns a string indicating the node type</li> + <li><code>content</code> : returns the content of the node, it is based on + xmlNodeGetContent() and hence is recursive.</li> + <li><code>parent</code> , <code>children</code>, <code>last</code>, + <code>next</code>, <code>prev</code>, <code>doc</code>, + <code>properties</code>: pointing to the associated element in the tree, + those may return None in case no such link exists.</li> +</ul> + +<p>Also note the need to explicitly deallocate documents with freeDoc() . +Reference counting for libxml2 trees would need quite a lot of work to +function properly, and rather than risk memory leaks if not implemented +correctly it sounds safer to have an explicit function to free a tree. The +wrapper python objects like doc, root or child are them automatically garbage +collected.</p> + +<h3>validate.py:</h3> + +<p>This test check the validation interfaces and redirection of error +messages:</p> +<pre>import libxml2 + +#deactivate error messages from the validation +def noerr(ctx, str): + pass + +libxml2.registerErrorHandler(noerr, None) + +ctxt = libxml2.createFileParserCtxt("invalid.xml") +ctxt.validate(1) +ctxt.parseDocument() +doc = ctxt.doc() +valid = ctxt.isValid() +doc.freeDoc() +if valid != 0: + print "validity check failed"</pre> + +<p>The first thing to notice is the call to registerErrorHandler(), it +defines a new error handler global to the library. It is used to avoid seeing +the error messages when trying to validate the invalid document.</p> + +<p>The main interest of that test is the creation of a parser context with +createFileParserCtxt() and how the behaviour can be changed before calling +parseDocument() . Similarly the informations resulting from the parsing phase +are also available using context methods.</p> + +<p>Contexts like nodes are defined as class and the libxml2 wrappers maps the +C function interfaces in terms of objects method as much as possible. The +best to get a complete view of what methods are supported is to look at the +libxml2.py module containing all the wrappers.</p> + +<h3>push.py:</h3> + +<p>This test show how to activate the push parser interface:</p> +<pre>import libxml2 + +ctxt = libxml2.createPushParser(None, "<foo", 4, "test.xml") +ctxt.parseChunk("/>", 2, 1) +doc = ctxt.doc() + +doc.freeDoc()</pre> + +<p>The context is created with a special call based on the +xmlCreatePushParser() from the C library. The first argument is an optional +SAX callback object, then the initial set of data, the length and the name of +the resource in case URI-References need to be computed by the parser.</p> + +<p>Then the data are pushed using the parseChunk() method, the last call +setting the third argument terminate to 1.</p> + +<h3>pushSAX.py:</h3> + +<p>this test show the use of the event based parsing interfaces. In this case +the parser does not build a document, but provides callback information as +the parser makes progresses analyzing the data being provided:</p> +<pre>import libxml2 +log = "" + +class callback: + def startDocument(self): + global log + log = log + "startDocument:" + + def endDocument(self): + global log + log = log + "endDocument:" + + def startElement(self, tag, attrs): + global log + log = log + "startElement %s %s:" % (tag, attrs) + + def endElement(self, tag): + global log + log = log + "endElement %s:" % (tag) + + def characters(self, data): + global log + log = log + "characters: %s:" % (data) + + def warning(self, msg): + global log + log = log + "warning: %s:" % (msg) + + def error(self, msg): + global log + log = log + "error: %s:" % (msg) + + def fatalError(self, msg): + global log + log = log + "fatalError: %s:" % (msg) + +handler = callback() + +ctxt = libxml2.createPushParser(handler, "<foo", 4, "test.xml") +chunk = " url='tst'>b" +ctxt.parseChunk(chunk, len(chunk), 0) +chunk = "ar</foo>" +ctxt.parseChunk(chunk, len(chunk), 1) + +reference = "startDocument:startElement foo {'url': 'tst'}:" + \ + "characters: bar:endElement foo:endDocument:" +if log != reference: + print "Error got: %s" % log + print "Expected: %s" % reference</pre> + +<p>The key object in that test is the handler, it provides a number of entry +points which can be called by the parser as it makes progresses to indicate +the information set obtained. The full set of callback is larger than what +the callback class in that specific example implements (see the SAX +definition for a complete list). The wrapper will only call those supplied by +the object when activated. The startElement receives the names of the element +and a dictionary containing the attributes carried by this element.</p> + +<p>Also note that the reference string generated from the callback shows a +single character call even though the string "bar" is passed to the parser +from 2 different call to parseChunk()</p> + +<h3>xpath.py:</h3> + +<p>This is a basic test of XPath wrappers support</p> +<pre>import libxml2 + +doc = libxml2.parseFile("tst.xml") +ctxt = doc.xpathNewContext() +res = ctxt.xpathEval("//*") +if len(res) != 2: + print "xpath query: wrong node set size" + sys.exit(1) +if res[0].name != "doc" or res[1].name != "foo": + print "xpath query: wrong node set value" + sys.exit(1) +doc.freeDoc() +ctxt.xpathFreeContext()</pre> + +<p>This test parses a file, then create an XPath context to evaluate XPath +expression on it. The xpathEval() method execute an XPath query and returns +the result mapped in a Python way. String and numbers are natively converted, +and node sets are returned as a tuple of libxml2 Python nodes wrappers. Like +the document, the XPath context need to be freed explicitly, also not that +the result of the XPath query may point back to the document tree and hence +the document must be freed after the result of the query is used.</p> + +<h3>xpathext.py:</h3> + +<p>This test shows how to extend the XPath engine with functions written in +python:</p> +<pre>import libxml2 + +def foo(ctx, x): + return x + 1 + +doc = libxml2.parseFile("tst.xml") +ctxt = doc.xpathNewContext() +libxml2.registerXPathFunction(ctxt._o, "foo", None, foo) +res = ctxt.xpathEval("foo(1)") +if res != 2: + print "xpath extension failure" +doc.freeDoc() +ctxt.xpathFreeContext()</pre> + +<p>Note how the extension function is registered with the context (but that +part is not yet finalized, this may change slightly in the future).</p> + +<h3>tstxpath.py:</h3> + +<p>This test is similar to the previous one but shows how the extension +function can access the XPath evaluation context:</p> +<pre>def foo(ctx, x): + global called + + # + # test that access to the XPath evaluation contexts + # + pctxt = libxml2.xpathParserContext(_obj=ctx) + ctxt = pctxt.context() + called = ctxt.function() + return x + 1</pre> + +<p>All the interfaces around the XPath parser(or rather evaluation) context +are not finalized, but it should be sufficient to do contextual work at the +evaluation point.</p> + +<h3>Memory debugging:</h3> + +<p>last but not least, all tests starts with the following prologue:</p> +<pre>#memory debug specific +libxml2.debugMemory(1)</pre> + +<p>and ends with the following epilogue:</p> +<pre>#memory debug specific +libxml2.cleanupParser() +if libxml2.debugMemory(1) == 0: + print "OK" +else: + print "Memory leak %d bytes" % (libxml2.debugMemory(1)) + libxml2.dumpMemory()</pre> + +<p>Those activate the memory debugging interface of libxml2 where all +allocated block in the library are tracked. The prologue then cleans up the +library state and checks that all allocated memory has been freed. If not it +calls dumpMemory() which saves that list in a <code>.memdump</code> file.</p> + +<h2><a name="architecture">libxml2 architecture</a></h2> + +<p>Libxml2 is made of multiple components; some of them are optional, and +most of the block interfaces are public. The main components are:</p> +<ul> + <li>an Input/Output layer</li> + <li>FTP and HTTP client layers (optional)</li> + <li>an Internationalization layer managing the encodings support</li> + <li>a URI module</li> + <li>the XML parser and its basic SAX interface</li> + <li>an HTML parser using the same SAX interface (optional)</li> + <li>a SAX tree module to build an in-memory DOM representation</li> + <li>a tree module to manipulate the DOM representation</li> + <li>a validation module using the DOM representation (optional)</li> + <li>an XPath module for global lookup in a DOM representation + (optional)</li> + <li>a debug module (optional)</li> +</ul> + +<p>Graphically this gives the following:</p> + +<p><img src="libxml.gif" alt="a graphical view of the various"></p> + +<p></p> + +<h2><a name="tree">The tree output</a></h2> + +<p>The parser returns a tree built during the document analysis. The value +returned is an <strong>xmlDocPtr</strong> (i.e., a pointer to an +<strong>xmlDoc</strong> structure). This structure contains information such +as the file name, the document type, and a <strong>children</strong> pointer +which is the root of the document (or more exactly the first child under the +root which is the document). The tree is made of <strong>xmlNode</strong>s, +chained in double-linked lists of siblings and with a children<->parent +relationship. An xmlNode can also carry properties (a chain of xmlAttr +structures). An attribute may have a value which is a list of TEXT or +ENTITY_REF nodes.</p> + +<p>Here is an example (erroneous with respect to the XML spec since there +should be only one ELEMENT under the root):</p> + +<p><img src="structure.gif" alt=" structure.gif "></p> + +<p>In the source package there is a small program (not installed by default) +called <strong>xmllint</strong> which parses XML files given as argument and +prints them back as parsed. This is useful for detecting errors both in XML +code and in the XML parser itself. It has an option <strong>--debug</strong> +which prints the actual in-memory structure of the document; here is the +result with the <a href="#example">example</a> given before:</p> +<pre>DOCUMENT +version=1.0 +standalone=true + ELEMENT EXAMPLE + ATTRIBUTE prop1 + TEXT + content=gnome is great + ATTRIBUTE prop2 + ENTITY_REF + TEXT + content= linux too + ELEMENT head + ELEMENT title + TEXT + content=Welcome to Gnome + ELEMENT chapter + ELEMENT title + TEXT + content=The Linux adventure + ELEMENT p + TEXT + content=bla bla bla ... + ELEMENT image + ATTRIBUTE href + TEXT + content=linus.gif + ELEMENT p + TEXT + content=...</pre> + +<p>This should be useful for learning the internal representation model.</p> + +<h2><a name="interface">The SAX interface</a></h2> + +<p>Sometimes the DOM tree output is just too large to fit reasonably into +memory. In that case (and if you don't expect to save back the XML document +loaded using libxml), it's better to use the SAX interface of libxml. SAX is +a <strong>callback-based interface</strong> to the parser. Before parsing, +the application layer registers a customized set of callbacks which are +called by the library as it progresses through the XML input.</p> + +<p>To get more detailed step-by-step guidance on using the SAX interface of +libxml, see the <a +href="http://www.daa.com.au/~james/gnome/xml-sax/xml-sax.html">nice +documentation</a>.written by <a href="mailto:james@daa.com.au">James +Henstridge</a>.</p> + +<p>You can debug the SAX behaviour by using the <strong>testSAX</strong> +program located in the gnome-xml module (it's usually not shipped in the +binary packages of libxml, but you can find it in the tar source +distribution). Here is the sequence of callbacks that would be reported by +testSAX when parsing the example XML document shown earlier:</p> +<pre>SAX.setDocumentLocator() +SAX.startDocument() +SAX.getEntity(amp) +SAX.startElement(EXAMPLE, prop1='gnome is great', prop2='&amp; linux too') +SAX.characters( , 3) +SAX.startElement(head) +SAX.characters( , 4) +SAX.startElement(title) +SAX.characters(Welcome to Gnome, 16) +SAX.endElement(title) +SAX.characters( , 3) +SAX.endElement(head) +SAX.characters( , 3) +SAX.startElement(chapter) +SAX.characters( , 4) +SAX.startElement(title) +SAX.characters(The Linux adventure, 19) +SAX.endElement(title) +SAX.characters( , 4) +SAX.startElement(p) +SAX.characters(bla bla bla ..., 15) +SAX.endElement(p) +SAX.characters( , 4) +SAX.startElement(image, href='linus.gif') +SAX.endElement(image) +SAX.characters( , 4) +SAX.startElement(p) +SAX.characters(..., 3) +SAX.endElement(p) +SAX.characters( , 3) +SAX.endElement(chapter) +SAX.characters( , 1) +SAX.endElement(EXAMPLE) +SAX.endDocument()</pre> + +<p>Most of the other interfaces of libxml2 are based on the DOM tree-building +facility, so nearly everything up to the end of this document presupposes the +use of the standard DOM tree build. Note that the DOM tree itself is built by +a set of registered default callbacks, without internal specific +interface.</p> + +<h2><a name="Validation">Validation & DTDs</a></h2> + +<p>Table of Content:</p> +<ol> + <li><a href="#General5">General overview</a></li> + <li><a href="#definition">The definition</a></li> + <li><a href="#Simple">Simple rules</a> + <ol> + <li><a href="#reference">How to reference a DTD from a document</a></li> + <li><a href="#Declaring">Declaring elements</a></li> + <li><a href="#Declaring1">Declaring attributes</a></li> + </ol> + </li> + <li><a href="#Some">Some examples</a></li> + <li><a href="#validate">How to validate</a></li> + <li><a href="#Other">Other resources</a></li> +</ol> + +<h3><a name="General5">General overview</a></h3> + +<p>Well what is validation and what is a DTD ?</p> + +<p>DTD is the acronym for Document Type Definition. This is a description of +the content for a family of XML files. This is part of the XML 1.0 +specification, and allows one to describe and verify that a given document +instance conforms to the set of rules detailing its structure and content.</p> + +<p>Validation is the process of checking a document against a DTD (more +generally against a set of construction rules).</p> + +<p>The validation process and building DTDs are the two most difficult parts +of the XML life cycle. Briefly a DTD defines all the possible elements to be +found within your document, what is the formal shape of your document tree +(by defining the allowed content of an element; either text, a regular +expression for the allowed list of children, or mixed content i.e. both text +and children). The DTD also defines the valid attributes for all elements and +the types of those attributes.</p> + +<h3><a name="definition1">The definition</a></h3> + +<p>The <a href="http://www.w3.org/TR/REC-xml">W3C XML Recommendation</a> (<a +href="http://www.xml.com/axml/axml.html">Tim Bray's annotated version of +Rev1</a>):</p> +<ul> + <li><a href="http://www.w3.org/TR/REC-xml#elemdecls">Declaring + elements</a></li> + <li><a href="http://www.w3.org/TR/REC-xml#attdecls">Declaring + attributes</a></li> +</ul> + +<p>(unfortunately) all this is inherited from the SGML world, the syntax is +ancient...</p> + +<h3><a name="Simple1">Simple rules</a></h3> + +<p>Writing DTDs can be done in many ways. The rules to build them if you need +something permanent or something which can evolve over time can be radically +different. Really complex DTDs like DocBook ones are flexible but quite +harder to design. I will just focus on DTDs for a formats with a fixed simple +structure. It is just a set of basic rules, and definitely not exhaustive nor +usable for complex DTD design.</p> + +<h4><a name="reference1">How to reference a DTD from a document</a>:</h4> + +<p>Assuming the top element of the document is <code>spec</code> and the dtd +is placed in the file <code>mydtd</code> in the subdirectory +<code>dtds</code> of the directory from where the document were loaded:</p> + +<p><code><!DOCTYPE spec SYSTEM "dtds/mydtd"></code></p> + +<p>Notes:</p> +<ul> + <li>The system string is actually an URI-Reference (as defined in <a + href="http://www.ietf.org/rfc/rfc2396.txt">RFC 2396</a>) so you can use a + full URL string indicating the location of your DTD on the Web. This is a + really good thing to do if you want others to validate your document.</li> + <li>It is also possible to associate a <code>PUBLIC</code> identifier (a + magic string) so that the DTD is looked up in catalogs on the client side + without having to locate it on the web.</li> + <li>A DTD contains a set of element and attribute declarations, but they + don't define what the root of the document should be. This is explicitly + told to the parser/validator as the first element of the + <code>DOCTYPE</code> declaration.</li> +</ul> + +<h4><a name="Declaring2">Declaring elements</a>:</h4> + +<p>The following declares an element <code>spec</code>:</p> + +<p><code><!ELEMENT spec (front, body, back?)></code></p> + +<p>It also expresses that the spec element contains one <code>front</code>, +one <code>body</code> and one optional <code>back</code> children elements in +this order. The declaration of one element of the structure and its content +are done in a single declaration. Similarly the following declares +<code>div1</code> elements:</p> + +<p><code><!ELEMENT div1 (head, (p | list | note)*, div2?)></code></p> + +<p>which means div1 contains one <code>head</code> then a series of optional +<code>p</code>, <code>list</code>s and <code>note</code>s and then an +optional <code>div2</code>. And last but not least an element can contain +text:</p> + +<p><code><!ELEMENT b (#PCDATA)></code></p> + +<p><code>b</code> contains text or being of mixed content (text and elements +in no particular order):</p> + +<p><code><!ELEMENT p (#PCDATA|a|ul|b|i|em)*></code></p> + +<p><code>p </code>can contain text or <code>a</code>, <code>ul</code>, +<code>b</code>, <code>i </code>or <code>em</code> elements in no particular +order.</p> + +<h4><a name="Declaring1">Declaring attributes</a>:</h4> + +<p>Again the attributes declaration includes their content definition:</p> + +<p><code><!ATTLIST termdef name CDATA #IMPLIED></code></p> + +<p>means that the element <code>termdef</code> can have a <code>name</code> +attribute containing text (<code>CDATA</code>) and which is optional +(<code>#IMPLIED</code>). The attribute value can also be defined within a +set:</p> + +<p><code><!ATTLIST list type (bullets|ordered|glossary) +"ordered"></code></p> + +<p>means <code>list</code> element have a <code>type</code> attribute with 3 +allowed values "bullets", "ordered" or "glossary" and which default to +"ordered" if the attribute is not explicitly specified.</p> + +<p>The content type of an attribute can be text (<code>CDATA</code>), +anchor/reference/references +(<code>ID</code>/<code>IDREF</code>/<code>IDREFS</code>), entity(ies) +(<code>ENTITY</code>/<code>ENTITIES</code>) or name(s) +(<code>NMTOKEN</code>/<code>NMTOKENS</code>). The following defines that a +<code>chapter</code> element can have an optional <code>id</code> attribute +of type <code>ID</code>, usable for reference from attribute of type +IDREF:</p> + +<p><code><!ATTLIST chapter id ID #IMPLIED></code></p> + +<p>The last value of an attribute definition can be <code>#REQUIRED +</code>meaning that the attribute has to be given, <code>#IMPLIED</code> +meaning that it is optional, or the default value (possibly prefixed by +<code>#FIXED</code> if it is the only allowed).</p> + +<p>Notes:</p> +<ul> + <li>Usually the attributes pertaining to a given element are declared in a + single expression, but it is just a convention adopted by a lot of DTD + writers: + <pre><!ATTLIST termdef + id ID #REQUIRED + name CDATA #IMPLIED></pre> + <p>The previous construct defines both <code>id</code> and + <code>name</code> attributes for the element <code>termdef</code>.</p> + </li> +</ul> + +<h3><a name="Some1">Some examples</a></h3> + +<p>The directory <code>test/valid/dtds/</code> in the libxml2 distribution +contains some complex DTD examples. The example in the file +<code>test/valid/dia.xml</code> shows an XML file where the simple DTD is +directly included within the document.</p> + +<h3><a name="validate1">How to validate</a></h3> + +<p>The simplest way is to use the xmllint program included with libxml. The +<code>--valid</code> option turns-on validation of the files given as input. +For example the following validates a copy of the first revision of the XML +1.0 specification:</p> + +<p><code>xmllint --valid --noout test/valid/REC-xml-19980210.xml</code></p> + +<p>the -- noout is used to disable output of the resulting tree.</p> + +<p>The <code>--dtdvalid dtd</code> allows validation of the document(s) +against a given DTD.</p> + +<p>Libxml2 exports an API to handle DTDs and validation, check the <a +href="http://xmlsoft.org/html/libxml-valid.html">associated +description</a>.</p> + +<h3><a name="Other1">Other resources</a></h3> + +<p>DTDs are as old as SGML. So there may be a number of examples on-line, I +will just list one for now, others pointers welcome:</p> +<ul> + <li><a href="http://www.xml101.com:8081/dtd/">XML-101 DTD</a></li> +</ul> + +<p>I suggest looking at the examples found under test/valid/dtd and any of +the large number of books available on XML. The dia example in test/valid +should be both simple and complete enough to allow you to build your own.</p> + +<p></p> + +<h2><a name="Memory">Memory Management</a></h2> + +<p>Table of Content:</p> +<ol> + <li><a href="#General3">General overview</a></li> + <li><a href="#setting">Setting libxml2 set of memory routines</a></li> + <li><a href="#cleanup">Cleaning up after parsing</a></li> + <li><a href="#Debugging">Debugging routines</a></li> + <li><a href="#General4">General memory requirements</a></li> +</ol> + +<h3><a name="General3">General overview</a></h3> + +<p>The module <code><a +href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlmemory.h</a></code> +provides the interfaces to the libxml2 memory system:</p> +<ul> + <li>libxml2 does not use the libc memory allocator directly but xmlFree(), + xmlMalloc() and xmlRealloc()</li> + <li>those routines can be reallocated to a specific set of routine, by + default the libc ones i.e. free(), malloc() and realloc()</li> + <li>the xmlmemory.c module includes a set of debugging routine</li> +</ul> + +<h3><a name="setting">Setting libxml2 set of memory routines</a></h3> + +<p>It is sometimes useful to not use the default memory allocator, either for +debugging, analysis or to implement a specific behaviour on memory management +(like on embedded systems). Two function calls are available to do so:</p> +<ul> + <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemGet + ()</a> which return the current set of functions in use by the parser</li> + <li><a + href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemSetup()</a> + which allow to set up a new set of memory allocation functions</li> +</ul> + +<p>Of course a call to xmlMemSetup() should probably be done before calling +any other libxml2 routines (unless you are sure your allocations routines are +compatibles).</p> + +<h3><a name="cleanup">Cleaning up after parsing</a></h3> + +<p>Libxml2 is not stateless, there is a few set of memory structures needing +allocation before the parser is fully functional (some encoding structures +for example). This also mean that once parsing is finished there is a tiny +amount of memory (a few hundred bytes) which can be recollected if you don't +reuse the parser immediately:</p> +<ul> + <li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlCleanupParser + ()</a> is a centralized routine to free the parsing states. Note that it + won't deallocate any produced tree if any (use the xmlFreeDoc() and + related routines for this).</li> + <li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlInitParser + ()</a> is the dual routine allowing to preallocate the parsing state + which can be useful for example to avoid initialization reentrancy + problems when using libxml2 in multithreaded applications</li> +</ul> + +<p>Generally xmlCleanupParser() is safe, if needed the state will be rebuild +at the next invocation of parser routines, but be careful of the consequences +in multithreaded applications.</p> + +<h3><a name="Debugging">Debugging routines</a></h3> + +<p>When configured using --with-mem-debug flag (off by default), libxml2 uses +a set of memory allocation debugging routines keeping track of all allocated +blocks and the location in the code where the routine was called. A couple of +other debugging routines allow to dump the memory allocated infos to a file +or call a specific routine when a given block number is allocated:</p> +<ul> + <li><a + href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMallocLoc()</a> + <a + href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlReallocLoc()</a> + and <a + href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemStrdupLoc()</a> + are the memory debugging replacement allocation routines</li> + <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemoryDump + ()</a> dumps all the informations about the allocated memory block lefts + in the <code>.memdump</code> file</li> +</ul> + +<p>When developing libxml2 memory debug is enabled, the tests programs call +xmlMemoryDump () and the "make test" regression tests will check for any +memory leak during the full regression test sequence, this helps a lot +ensuring that libxml2 does not leak memory and bullet proof memory +allocations use (some libc implementations are known to be far too permissive +resulting in major portability problems!).</p> + +<p>If the .memdump reports a leak, it displays the allocation function and +also tries to give some informations about the content and structure of the +allocated blocks left. This is sufficient in most cases to find the culprit, +but not always. Assuming the allocation problem is reproducible, it is +possible to find more easily:</p> +<ol> + <li>write down the block number xxxx not allocated</li> + <li>export the environment variable XML_MEM_BREAKPOINT=xxxx , the easiest + when using GDB is to simply give the command + <p><code>set environment XML_MEM_BREAKPOINT xxxx</code></p> + <p>before running the program.</p> + </li> + <li>run the program under a debugger and set a breakpoint on + xmlMallocBreakpoint() a specific function called when this precise block + is allocated</li> + <li>when the breakpoint is reached you can then do a fine analysis of the + allocation an step to see the condition resulting in the missing + deallocation.</li> +</ol> + +<p>I used to use a commercial tool to debug libxml2 memory problems but after +noticing that it was not detecting memory leaks that simple mechanism was +used and proved extremely efficient until now. Lately I have also used <a +href="http://developer.kde.org/~sewardj/">valgrind</a> with quite some +success, it is tied to the i386 architecture since it works by emulating the +processor and instruction set, it is slow but extremely efficient, i.e. it +spot memory usage errors in a very precise way.</p> + +<h3><a name="General4">General memory requirements</a></h3> + +<p>How much libxml2 memory require ? It's hard to tell in average it depends +of a number of things:</p> +<ul> + <li>the parser itself should work in a fixed amount of memory, except for + information maintained about the stacks of names and entities locations. + The I/O and encoding handlers will probably account for a few KBytes. + This is true for both the XML and HTML parser (though the HTML parser + need more state).</li> + <li>If you are generating the DOM tree then memory requirements will grow + nearly linear with the size of the data. In general for a balanced + textual document the internal memory requirement is about 4 times the + size of the UTF8 serialization of this document (example the XML-1.0 + recommendation is a bit more of 150KBytes and takes 650KBytes of main + memory when parsed). Validation will add a amount of memory required for + maintaining the external Dtd state which should be linear with the + complexity of the content model defined by the Dtd</li> + <li>If you need to work with fixed memory requirements or don't need the + full DOM tree then using the <a href="xmlreader.html">xmlReader + interface</a> is probably the best way to proceed, it still allows to + validate or operate on subset of the tree if needed.</li> + <li>If you don't care about the advanced features of libxml2 like + validation, DOM, XPath or XPointer, don't use entities, need to work with + fixed memory requirements, and try to get the fastest parsing possible + then the SAX interface should be used, but it has known restrictions.</li> +</ul> + +<p></p> + +<h2><a name="Encodings">Encodings support</a></h2> + +<p>Table of Content:</p> +<ol> + <li><a href="encoding.html#What">What does internationalization support + mean ?</a></li> + <li><a href="encoding.html#internal">The internal encoding, how and + why</a></li> + <li><a href="encoding.html#implemente">How is it implemented ?</a></li> + <li><a href="encoding.html#Default">Default supported encodings</a></li> + <li><a href="encoding.html#extend">How to extend the existing + support</a></li> +</ol> + +<h3><a name="What">What does internationalization support mean ?</a></h3> + +<p>If you are not really familiar with Internationalization (usual shortcut +is I18N) , Unicode, characters and glyphs, I suggest you read a <a +href="http://www.tbray.org/ongoing/When/200x/2003/04/06/Unicode">presentation</a> +by Tim Bray on Unicode and why you should care about it.</p> + +<p>XML was designed from the start to allow the support of any character set +by using Unicode. Any conformant XML parser has to support the UTF-8 and +UTF-16 default encodings which can both express the full unicode ranges. UTF8 +is a variable length encoding whose greatest points are to reuse the same +encoding for ASCII and to save space for Western encodings, but it is a bit +more complex to handle in practice. UTF-16 use 2 bytes per character (and +sometimes combines two pairs), it makes implementation easier, but looks a +bit overkill for Western languages encoding. Moreover the XML specification +allows the document to be encoded in other encodings at the condition that +they are clearly labeled as such. For example the following is a wellformed +XML document encoded in ISO-8859-1 and using accentuated letters that we +French like for both markup and content:</p> +<pre><?xml version="1.0" encoding="ISO-8859-1"?> +<très>là</très></pre> + +<p>Having internationalization support in libxml2 means the following:</p> +<ul> + <li>the document is properly parsed</li> + <li>informations about it's encoding are saved</li> + <li>it can be modified</li> + <li>it can be saved in its original encoding</li> + <li>it can also be saved in another encoding supported by libxml2 (for + example straight UTF8 or even an ASCII form)</li> +</ul> + +<p>Another very important point is that the whole libxml2 API, with the +exception of a few routines to read with a specific encoding or save to a +specific encoding, is completely agnostic about the original encoding of the +document.</p> + +<p>It should be noted too that the HTML parser embedded in libxml2 now obey +the same rules too, the following document will be (as of 2.2.2) handled in +an internationalized fashion by libxml2 too:</p> +<pre><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" + "http://www.w3.org/TR/REC-html40/loose.dtd"> +<html lang="fr"> +<head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> +</head> +<body> +<p>W3C crée des standards pour le Web.</body> +</html></pre> + +<h3><a name="internal">The internal encoding, how and why</a></h3> + +<p>One of the core decisions was to force all documents to be converted to a +default internal encoding, and that encoding to be UTF-8, here are the +rationales for those choices:</p> +<ul> + <li>keeping the native encoding in the internal form would force the libxml + users (or the code associated) to be fully aware of the encoding of the + original document, for examples when adding a text node to a document, + the content would have to be provided in the document encoding, i.e. the + client code would have to check it before hand, make sure it's conformant + to the encoding, etc ... Very hard in practice, though in some specific + cases this may make sense.</li> + <li>the second decision was which encoding. From the XML spec only UTF8 and + UTF16 really makes sense as being the two only encodings for which there + is mandatory support. UCS-4 (32 bits fixed size encoding) could be + considered an intelligent choice too since it's a direct Unicode mapping + support. I selected UTF-8 on the basis of efficiency and compatibility + with surrounding software: + <ul> + <li>UTF-8 while a bit more complex to convert from/to (i.e. slightly + more costly to import and export CPU wise) is also far more compact + than UTF-16 (and UCS-4) for a majority of the documents I see it used + for right now (RPM RDF catalogs, advogato data, various configuration + file formats, etc.) and the key point for today's computer + architecture is efficient uses of caches. If one nearly double the + memory requirement to store the same amount of data, this will trash + caches (main memory/external caches/internal caches) and my take is + that this harms the system far more than the CPU requirements needed + for the conversion to UTF-8</li> + <li>Most of libxml2 version 1 users were using it with straight ASCII + most of the time, doing the conversion with an internal encoding + requiring all their code to be rewritten was a serious show-stopper + for using UTF-16 or UCS-4.</li> + <li>UTF-8 is being used as the de-facto internal encoding standard for + related code like the <a href="http://www.pango.org/">pango</a> + upcoming Gnome text widget, and a lot of Unix code (yet another place + where Unix programmer base takes a different approach from Microsoft + - they are using UTF-16)</li> + </ul> + </li> +</ul> + +<p>What does this mean in practice for the libxml2 user:</p> +<ul> + <li>xmlChar, the libxml2 data type is a byte, those bytes must be assembled + as UTF-8 valid strings. The proper way to terminate an xmlChar * string + is simply to append 0 byte, as usual.</li> + <li>One just need to make sure that when using chars outside the ASCII set, + the values has been properly converted to UTF-8</li> +</ul> + +<h3><a name="implemente">How is it implemented ?</a></h3> + +<p>Let's describe how all this works within libxml, basically the I18N +(internationalization) support get triggered only during I/O operation, i.e. +when reading a document or saving one. Let's look first at the reading +sequence:</p> +<ol> + <li>when a document is processed, we usually don't know the encoding, a + simple heuristic allows to detect UTF-16 and UCS-4 from encodings where + the ASCII range (0-0x7F) maps with ASCII</li> + <li>the xml declaration if available is parsed, including the encoding + declaration. At that point, if the autodetected encoding is different + from the one declared a call to xmlSwitchEncoding() is issued.</li> + <li>If there is no encoding declaration, then the input has to be in either + UTF-8 or UTF-16, if it is not then at some point when processing the + input, the converter/checker of UTF-8 form will raise an encoding error. + You may end-up with a garbled document, or no document at all ! Example: + <pre>~/XML -> ./xmllint err.xml +err.xml:1: error: Input is not proper UTF-8, indicate encoding ! +<très>là</très> + ^ +err.xml:1: error: Bytes: 0xE8 0x73 0x3E 0x6C +<très>là</très> + ^</pre> + </li> + <li>xmlSwitchEncoding() does an encoding name lookup, canonicalize it, and + then search the default registered encoding converters for that encoding. + If it's not within the default set and iconv() support has been compiled + it, it will ask iconv for such an encoder. If this fails then the parser + will report an error and stops processing: + <pre>~/XML -> ./xmllint err2.xml +err2.xml:1: error: Unsupported encoding UnsupportedEnc +<?xml version="1.0" encoding="UnsupportedEnc"?> + ^</pre> + </li> + <li>From that point the encoder processes progressively the input (it is + plugged as a front-end to the I/O module) for that entity. It captures + and converts on-the-fly the document to be parsed to UTF-8. The parser + itself just does UTF-8 checking of this input and process it + transparently. The only difference is that the encoding information has + been added to the parsing context (more precisely to the input + corresponding to this entity).</li> + <li>The result (when using DOM) is an internal form completely in UTF-8 + with just an encoding information on the document node.</li> +</ol> + +<p>Ok then what happens when saving the document (assuming you +collected/built an xmlDoc DOM like structure) ? It depends on the function +called, xmlSaveFile() will just try to save in the original encoding, while +xmlSaveFileTo() and xmlSaveFileEnc() can optionally save to a given +encoding:</p> +<ol> + <li>if no encoding is given, libxml2 will look for an encoding value + associated to the document and if it exists will try to save to that + encoding, + <p>otherwise everything is written in the internal form, i.e. UTF-8</p> + </li> + <li>so if an encoding was specified, either at the API level or on the + document, libxml2 will again canonicalize the encoding name, lookup for a + converter in the registered set or through iconv. If not found the + function will return an error code</li> + <li>the converter is placed before the I/O buffer layer, as another kind of + buffer, then libxml2 will simply push the UTF-8 serialization to through + that buffer, which will then progressively be converted and pushed onto + the I/O layer.</li> + <li>It is possible that the converter code fails on some input, for example + trying to push an UTF-8 encoded Chinese character through the UTF-8 to + ISO-8859-1 converter won't work. Since the encoders are progressive they + will just report the error and the number of bytes converted, at that + point libxml2 will decode the offending character, remove it from the + buffer and replace it with the associated charRef encoding &#123; and + resume the conversion. This guarantees that any document will be saved + without losses (except for markup names where this is not legal, this is + a problem in the current version, in practice avoid using non-ascii + characters for tag or attribute names). A special "ascii" encoding name + is used to save documents to a pure ascii form can be used when + portability is really crucial</li> +</ol> + +<p>Here are a few examples based on the same test document:</p> +<pre>~/XML -> ./xmllint isolat1 +<?xml version="1.0" encoding="ISO-8859-1"?> +<très>là</très> +~/XML -> ./xmllint --encode UTF-8 isolat1 +<?xml version="1.0" encoding="UTF-8"?> +<très>là </très> +~/XML -> </pre> + +<p>The same processing is applied (and reuse most of the code) for HTML I18N +processing. Looking up and modifying the content encoding is a bit more +difficult since it is located in a <meta> tag under the <head>, +so a couple of functions htmlGetMetaEncoding() and htmlSetMetaEncoding() have +been provided. The parser also attempts to switch encoding on the fly when +detecting such a tag on input. Except for that the processing is the same +(and again reuses the same code).</p> + +<h3><a name="Default">Default supported encodings</a></h3> + +<p>libxml2 has a set of default converters for the following encodings +(located in encoding.c):</p> +<ol> + <li>UTF-8 is supported by default (null handlers)</li> + <li>UTF-16, both little and big endian</li> + <li>ISO-Latin-1 (ISO-8859-1) covering most western languages</li> + <li>ASCII, useful mostly for saving</li> + <li>HTML, a specific handler for the conversion of UTF-8 to ASCII with HTML + predefined entities like &copy; for the Copyright sign.</li> +</ol> + +<p>More over when compiled on an Unix platform with iconv support the full +set of encodings supported by iconv can be instantly be used by libxml. On a +linux machine with glibc-2.1 the list of supported encodings and aliases fill +3 full pages, and include UCS-4, the full set of ISO-Latin encodings, and the +various Japanese ones.</p> + +<h4>Encoding aliases</h4> + +<p>From 2.2.3, libxml2 has support to register encoding names aliases. The +goal is to be able to parse document whose encoding is supported but where +the name differs (for example from the default set of names accepted by +iconv). The following functions allow to register and handle new aliases for +existing encodings. Once registered libxml2 will automatically lookup the +aliases when handling a document:</p> +<ul> + <li>int xmlAddEncodingAlias(const char *name, const char *alias);</li> + <li>int xmlDelEncodingAlias(const char *alias);</li> + <li>const char * xmlGetEncodingAlias(const char *alias);</li> + <li>void xmlCleanupEncodingAliases(void);</li> +</ul> + +<h3><a name="extend">How to extend the existing support</a></h3> + +<p>Well adding support for new encoding, or overriding one of the encoders +(assuming it is buggy) should not be hard, just write input and output +conversion routines to/from UTF-8, and register them using +xmlNewCharEncodingHandler(name, xxxToUTF8, UTF8Toxxx), and they will be +called automatically if the parser(s) encounter such an encoding name +(register it uppercase, this will help). The description of the encoders, +their arguments and expected return values are described in the encoding.h +header.</p> + +<p>A quick note on the topic of subverting the parser to use a different +internal encoding than UTF-8, in some case people will absolutely want to +keep the internal encoding different, I think it's still possible (but the +encoding must be compliant with ASCII on the same subrange) though I didn't +tried it. The key is to override the default conversion routines (by +registering null encoders/decoders for your charsets), and bypass the UTF-8 +checking of the parser by setting the parser context charset +(ctxt->charset) to something different than XML_CHAR_ENCODING_UTF8, but +there is no guarantee that this will work. You may also have some troubles +saving back.</p> + +<p>Basically proper I18N support is important, this requires at least +libxml-2.0.0, but a lot of features and corrections are really available only +starting 2.2.</p> + +<h2><a name="IO">I/O Interfaces</a></h2> + +<p>Table of Content:</p> +<ol> + <li><a href="#General1">General overview</a></li> + <li><a href="#basic">The basic buffer type</a></li> + <li><a href="#Input">Input I/O handlers</a></li> + <li><a href="#Output">Output I/O handlers</a></li> + <li><a href="#entities">The entities loader</a></li> + <li><a href="#Example2">Example of customized I/O</a></li> +</ol> + +<h3><a name="General1">General overview</a></h3> + +<p>The module <code><a +href="http://xmlsoft.org/html/libxml-xmlio.html">xmlIO.h</a></code> provides +the interfaces to the libxml2 I/O system. This consists of 4 main parts:</p> +<ul> + <li>Entities loader, this is a routine which tries to fetch the entities + (files) based on their PUBLIC and SYSTEM identifiers. The default loader + don't look at the public identifier since libxml2 do not maintain a + catalog. You can redefine you own entity loader by using + <code>xmlGetExternalEntityLoader()</code> and + <code>xmlSetExternalEntityLoader()</code>. <a href="#entities">Check the + example</a>.</li> + <li>Input I/O buffers which are a commodity structure used by the parser(s) + input layer to handle fetching the informations to feed the parser. This + provides buffering and is also a placeholder where the encoding + converters to UTF8 are piggy-backed.</li> + <li>Output I/O buffers are similar to the Input ones and fulfill similar + task but when generating a serialization from a tree.</li> + <li>A mechanism to register sets of I/O callbacks and associate them with + specific naming schemes like the protocol part of the URIs. + <p>This affect the default I/O operations and allows to use specific I/O + handlers for certain names.</p> + </li> +</ul> + +<p>The general mechanism used when loading http://rpmfind.net/xml.html for +example in the HTML parser is the following:</p> +<ol> + <li>The default entity loader calls <code>xmlNewInputFromFile()</code> with + the parsing context and the URI string.</li> + <li>the URI string is checked against the existing registered handlers + using their match() callback function, if the HTTP module was compiled + in, it is registered and its match() function will succeeds</li> + <li>the open() function of the handler is called and if successful will + return an I/O Input buffer</li> + <li>the parser will the start reading from this buffer and progressively + fetch information from the resource, calling the read() function of the + handler until the resource is exhausted</li> + <li>if an encoding change is detected it will be installed on the input + buffer, providing buffering and efficient use of the conversion + routines</li> + <li>once the parser has finished, the close() function of the handler is + called once and the Input buffer and associated resources are + deallocated.</li> +</ol> + +<p>The user defined callbacks are checked first to allow overriding of the +default libxml2 I/O routines.</p> + +<h3><a name="basic">The basic buffer type</a></h3> + +<p>All the buffer manipulation handling is done using the +<code>xmlBuffer</code> type define in <code><a +href="http://xmlsoft.org/html/libxml-tree.html">tree.h</a> </code>which is a +resizable memory buffer. The buffer allocation strategy can be selected to be +either best-fit or use an exponential doubling one (CPU vs. memory use +trade-off). The values are <code>XML_BUFFER_ALLOC_EXACT</code> and +<code>XML_BUFFER_ALLOC_DOUBLEIT</code>, and can be set individually or on a +system wide basis using <code>xmlBufferSetAllocationScheme()</code>. A number +of functions allows to manipulate buffers with names starting with the +<code>xmlBuffer...</code> prefix.</p> + +<h3><a name="Input">Input I/O handlers</a></h3> + +<p>An Input I/O handler is a simple structure +<code>xmlParserInputBuffer</code> containing a context associated to the +resource (file descriptor, or pointer to a protocol handler), the read() and +close() callbacks to use and an xmlBuffer. And extra xmlBuffer and a charset +encoding handler are also present to support charset conversion when +needed.</p> + +<h3><a name="Output">Output I/O handlers</a></h3> + +<p>An Output handler <code>xmlOutputBuffer</code> is completely similar to an +Input one except the callbacks are write() and close().</p> + +<h3><a name="entities">The entities loader</a></h3> + +<p>The entity loader resolves requests for new entities and create inputs for +the parser. Creating an input from a filename or an URI string is done +through the xmlNewInputFromFile() routine. The default entity loader do not +handle the PUBLIC identifier associated with an entity (if any). So it just +calls xmlNewInputFromFile() with the SYSTEM identifier (which is mandatory in +XML).</p> + +<p>If you want to hook up a catalog mechanism then you simply need to +override the default entity loader, here is an example:</p> +<pre>#include <libxml/xmlIO.h> + +xmlExternalEntityLoader defaultLoader = NULL; + +xmlParserInputPtr +xmlMyExternalEntityLoader(const char *URL, const char *ID, + xmlParserCtxtPtr ctxt) { + xmlParserInputPtr ret; + const char *fileID = NULL; + /* lookup for the fileID depending on ID */ + + ret = xmlNewInputFromFile(ctxt, fileID); + if (ret != NULL) + return(ret); + if (defaultLoader != NULL) + ret = defaultLoader(URL, ID, ctxt); + return(ret); +} + +int main(..) { + ... + + /* + * Install our own entity loader + */ + defaultLoader = xmlGetExternalEntityLoader(); + xmlSetExternalEntityLoader(xmlMyExternalEntityLoader); + + ... +}</pre> + +<h3><a name="Example2">Example of customized I/O</a></h3> + +<p>This example come from <a href="http://xmlsoft.org/messages/0708.html">a +real use case</a>, xmlDocDump() closes the FILE * passed by the application +and this was a problem. The <a +href="http://xmlsoft.org/messages/0711.html">solution</a> was to redefine a +new output handler with the closing call deactivated:</p> +<ol> + <li>First define a new I/O output allocator where the output don't close + the file: + <pre>xmlOutputBufferPtr +xmlOutputBufferCreateOwn(FILE *file, xmlCharEncodingHandlerPtr encoder) { + xmlOutputBufferPtr ret; + + if (xmlOutputCallbackInitialized == 0) + xmlRegisterDefaultOutputCallbacks(); + + if (file == NULL) return(NULL); + ret = xmlAllocOutputBuffer(encoder); + if (ret != NULL) { + ret->context = file; + ret->writecallback = xmlFileWrite; + ret->closecallback = NULL; /* No close callback */ + } + return(ret); +} </pre> + </li> + <li>And then use it to save the document: + <pre>FILE *f; +xmlOutputBufferPtr output; +xmlDocPtr doc; +int res; + +f = ... +doc = .... + +output = xmlOutputBufferCreateOwn(f, NULL); +res = xmlSaveFileTo(output, doc, NULL); + </pre> + </li> +</ol> + +<h2><a name="Catalog">Catalog support</a></h2> + +<p>Table of Content:</p> +<ol> + <li><a href="General2">General overview</a></li> + <li><a href="#definition">The definition</a></li> + <li><a href="#Simple">Using catalogs</a></li> + <li><a href="#Some">Some examples</a></li> + <li><a href="#reference">How to tune catalog usage</a></li> + <li><a href="#validate">How to debug catalog processing</a></li> + <li><a href="#Declaring">How to create and maintain catalogs</a></li> + <li><a href="#implemento">The implementor corner quick review of the + API</a></li> + <li><a href="#Other">Other resources</a></li> +</ol> + +<h3><a name="General2">General overview</a></h3> + +<p>What is a catalog? Basically it's a lookup mechanism used when an entity +(a file or a remote resource) references another entity. The catalog lookup +is inserted between the moment the reference is recognized by the software +(XML parser, stylesheet processing, or even images referenced for inclusion +in a rendering) and the time where loading that resource is actually +started.</p> + +<p>It is basically used for 3 things:</p> +<ul> + <li>mapping from "logical" names, the public identifiers and a more + concrete name usable for download (and URI). For example it can associate + the logical name + <p>"-//OASIS//DTD DocBook XML V4.1.2//EN"</p> + <p>of the DocBook 4.1.2 XML DTD with the actual URL where it can be + downloaded</p> + <p>http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd</p> + </li> + <li>remapping from a given URL to another one, like an HTTP indirection + saying that + <p>"http://www.oasis-open.org/committes/tr.xsl"</p> + <p>should really be looked at</p> + <p>"http://www.oasis-open.org/committes/entity/stylesheets/base/tr.xsl"</p> + </li> + <li>providing a local cache mechanism allowing to load the entities + associated to public identifiers or remote resources, this is a really + important feature for any significant deployment of XML or SGML since it + allows to avoid the aleas and delays associated to fetching remote + resources.</li> +</ul> + +<h3><a name="definition">The definitions</a></h3> + +<p>Libxml, as of 2.4.3 implements 2 kind of catalogs:</p> +<ul> + <li>the older SGML catalogs, the official spec is SGML Open Technical + Resolution TR9401:1997, but is better understood by reading <a + href="http://www.jclark.com/sp/catalog.htm">the SP Catalog page</a> from + James Clark. This is relatively old and not the preferred mode of + operation of libxml.</li> + <li><a href="http://www.oasis-open.org/committees/entity/spec.html">XML + Catalogs</a> is far more flexible, more recent, uses an XML syntax and + should scale quite better. This is the default option of libxml.</li> +</ul> + +<p></p> + +<h3><a name="Simple">Using catalog</a></h3> + +<p>In a normal environment libxml2 will by default check the presence of a +catalog in /etc/xml/catalog, and assuming it has been correctly populated, +the processing is completely transparent to the document user. To take a +concrete example, suppose you are authoring a DocBook document, this one +starts with the following DOCTYPE definition:</p> +<pre><?xml version='1.0'?> +<!DOCTYPE book PUBLIC "-//Norman Walsh//DTD DocBk XML V3.1.4//EN" + "http://nwalsh.com/docbook/xml/3.1.4/db3xml.dtd"></pre> + +<p>When validating the document with libxml, the catalog will be +automatically consulted to lookup the public identifier "-//Norman Walsh//DTD +DocBk XML V3.1.4//EN" and the system identifier +"http://nwalsh.com/docbook/xml/3.1.4/db3xml.dtd", and if these entities have +been installed on your system and the catalogs actually point to them, libxml +will fetch them from the local disk.</p> + +<p style="font-size: 10pt"><strong>Note</strong>: Really don't use this +DOCTYPE example it's a really old version, but is fine as an example.</p> + +<p>Libxml2 will check the catalog each time that it is requested to load an +entity, this includes DTD, external parsed entities, stylesheets, etc ... If +your system is correctly configured all the authoring phase and processing +should use only local files, even if your document stays portable because it +uses the canonical public and system ID, referencing the remote document.</p> + +<h3><a name="Some">Some examples:</a></h3> + +<p>Here is a couple of fragments from XML Catalogs used in libxml2 early +regression tests in <code>test/catalogs</code> :</p> +<pre><?xml version="1.0"?> +<!DOCTYPE catalog PUBLIC + "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" + "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd"> +<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> + <public publicId="-//OASIS//DTD DocBook XML V4.1.2//EN" + uri="http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"/> +...</pre> + +<p>This is the beginning of a catalog for DocBook 4.1.2, XML Catalogs are +written in XML, there is a specific namespace for catalog elements +"urn:oasis:names:tc:entity:xmlns:xml:catalog". The first entry in this +catalog is a <code>public</code> mapping it allows to associate a Public +Identifier with an URI.</p> +<pre>... + <rewriteSystem systemIdStartString="http://www.oasis-open.org/docbook/" + rewritePrefix="file:///usr/share/xml/docbook/"/> +...</pre> + +<p>A <code>rewriteSystem</code> is a very powerful instruction, it says that +any URI starting with a given prefix should be looked at another URI +constructed by replacing the prefix with an new one. In effect this acts like +a cache system for a full area of the Web. In practice it is extremely useful +with a file prefix if you have installed a copy of those resources on your +local system.</p> +<pre>... +<delegatePublic publicIdStartString="-//OASIS//DTD XML Catalog //" + catalog="file:///usr/share/xml/docbook.xml"/> +<delegatePublic publicIdStartString="-//OASIS//ENTITIES DocBook XML" + catalog="file:///usr/share/xml/docbook.xml"/> +<delegatePublic publicIdStartString="-//OASIS//DTD DocBook XML" + catalog="file:///usr/share/xml/docbook.xml"/> +<delegateSystem systemIdStartString="http://www.oasis-open.org/docbook/" + catalog="file:///usr/share/xml/docbook.xml"/> +<delegateURI uriStartString="http://www.oasis-open.org/docbook/" + catalog="file:///usr/share/xml/docbook.xml"/> +...</pre> + +<p>Delegation is the core features which allows to build a tree of catalogs, +easier to maintain than a single catalog, based on Public Identifier, System +Identifier or URI prefixes it instructs the catalog software to look up +entries in another resource. This feature allow to build hierarchies of +catalogs, the set of entries presented should be sufficient to redirect the +resolution of all DocBook references to the specific catalog in +<code>/usr/share/xml/docbook.xml</code> this one in turn could delegate all +references for DocBook 4.2.1 to a specific catalog installed at the same time +as the DocBook resources on the local machine.</p> + +<h3><a name="reference">How to tune catalog usage:</a></h3> + +<p>The user can change the default catalog behaviour by redirecting queries +to its own set of catalogs, this can be done by setting the +<code>XML_CATALOG_FILES</code> environment variable to a list of catalogs, an +empty one should deactivate loading the default <code>/etc/xml/catalog</code> +default catalog</p> + +<h3><a name="validate">How to debug catalog processing:</a></h3> + +<p>Setting up the <code>XML_DEBUG_CATALOG</code> environment variable will +make libxml2 output debugging informations for each catalog operations, for +example:</p> +<pre>orchis:~/XML -> xmllint --memory --noout test/ent2 +warning: failed to load external entity "title.xml" +orchis:~/XML -> export XML_DEBUG_CATALOG= +orchis:~/XML -> xmllint --memory --noout test/ent2 +Failed to parse catalog /etc/xml/catalog +Failed to parse catalog /etc/xml/catalog +warning: failed to load external entity "title.xml" +Catalogs cleanup +orchis:~/XML -> </pre> + +<p>The test/ent2 references an entity, running the parser from memory makes +the base URI unavailable and the the "title.xml" entity cannot be loaded. +Setting up the debug environment variable allows to detect that an attempt is +made to load the <code>/etc/xml/catalog</code> but since it's not present the +resolution fails.</p> + +<p>But the most advanced way to debug XML catalog processing is to use the +<strong>xmlcatalog</strong> command shipped with libxml2, it allows to load +catalogs and make resolution queries to see what is going on. This is also +used for the regression tests:</p> +<pre>orchis:~/XML -> ./xmlcatalog test/catalogs/docbook.xml \ + "-//OASIS//DTD DocBook XML V4.1.2//EN" +http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd +orchis:~/XML -> </pre> + +<p>For debugging what is going on, adding one -v flags increase the verbosity +level to indicate the processing done (adding a second flag also indicate +what elements are recognized at parsing):</p> +<pre>orchis:~/XML -> ./xmlcatalog -v test/catalogs/docbook.xml \ + "-//OASIS//DTD DocBook XML V4.1.2//EN" +Parsing catalog test/catalogs/docbook.xml's content +Found public match -//OASIS//DTD DocBook XML V4.1.2//EN +http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd +Catalogs cleanup +orchis:~/XML -> </pre> + +<p>A shell interface is also available to debug and process multiple queries +(and for regression tests):</p> +<pre>orchis:~/XML -> ./xmlcatalog -shell test/catalogs/docbook.xml \ + "-//OASIS//DTD DocBook XML V4.1.2//EN" +> help +Commands available: +public PublicID: make a PUBLIC identifier lookup +system SystemID: make a SYSTEM identifier lookup +resolve PublicID SystemID: do a full resolver lookup +add 'type' 'orig' 'replace' : add an entry +del 'values' : remove values +dump: print the current catalog state +debug: increase the verbosity level +quiet: decrease the verbosity level +exit: quit the shell +> public "-//OASIS//DTD DocBook XML V4.1.2//EN" +http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd +> quit +orchis:~/XML -> </pre> + +<p>This should be sufficient for most debugging purpose, this was actually +used heavily to debug the XML Catalog implementation itself.</p> + +<h3><a name="Declaring">How to create and maintain</a> catalogs:</h3> + +<p>Basically XML Catalogs are XML files, you can either use XML tools to +manage them or use <strong>xmlcatalog</strong> for this. The basic step is +to create a catalog the -create option provide this facility:</p> +<pre>orchis:~/XML -> ./xmlcatalog --create tst.xml +<?xml version="1.0"?> +<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" + "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd"> +<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"/> +orchis:~/XML -> </pre> + +<p>By default xmlcatalog does not overwrite the original catalog and save the +result on the standard output, this can be overridden using the -noout +option. The <code>-add</code> command allows to add entries in the +catalog:</p> +<pre>orchis:~/XML -> ./xmlcatalog --noout --create --add "public" \ + "-//OASIS//DTD DocBook XML V4.1.2//EN" \ + http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd tst.xml +orchis:~/XML -> cat tst.xml +<?xml version="1.0"?> +<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" \ + "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd"> +<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> +<public publicId="-//OASIS//DTD DocBook XML V4.1.2//EN" + uri="http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"/> +</catalog> +orchis:~/XML -> </pre> + +<p>The <code>-add</code> option will always take 3 parameters even if some of +the XML Catalog constructs (like nextCatalog) will have only a single +argument, just pass a third empty string, it will be ignored.</p> + +<p>Similarly the <code>-del</code> option remove matching entries from the +catalog:</p> +<pre>orchis:~/XML -> ./xmlcatalog --del \ + "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" tst.xml +<?xml version="1.0"?> +<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" + "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd"> +<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"/> +orchis:~/XML -> </pre> + +<p>The catalog is now empty. Note that the matching of <code>-del</code> is +exact and would have worked in a similar fashion with the Public ID +string.</p> + +<p>This is rudimentary but should be sufficient to manage a not too complex +catalog tree of resources.</p> + +<h3><a name="implemento">The implementor corner quick review of the +API:</a></h3> + +<p>First, and like for every other module of libxml, there is an +automatically generated <a href="html/libxml-catalog.html">API page for +catalog support</a>.</p> + +<p>The header for the catalog interfaces should be included as:</p> +<pre>#include <libxml/catalog.h></pre> + +<p>The API is voluntarily kept very simple. First it is not obvious that +applications really need access to it since it is the default behaviour of +libxml2 (Note: it is possible to completely override libxml2 default catalog +by using <a href="html/libxml-parser.html">xmlSetExternalEntityLoader</a> to +plug an application specific resolver).</p> + +<p>Basically libxml2 support 2 catalog lists:</p> +<ul> + <li>the default one, global shared by all the application</li> + <li>a per-document catalog, this one is built if the document uses the + <code>oasis-xml-catalog</code> PIs to specify its own catalog list, it is + associated to the parser context and destroyed when the parsing context + is destroyed.</li> +</ul> + +<p>the document one will be used first if it exists.</p> + +<h4>Initialization routines:</h4> + +<p>xmlInitializeCatalog(), xmlLoadCatalog() and xmlLoadCatalogs() should be +used at startup to initialize the catalog, if the catalog should be +initialized with specific values xmlLoadCatalog() or xmlLoadCatalogs() +should be called before xmlInitializeCatalog() which would otherwise do a +default initialization first.</p> + +<p>The xmlCatalogAddLocal() call is used by the parser to grow the document +own catalog list if needed.</p> + +<h4>Preferences setup:</h4> + +<p>The XML Catalog spec requires the possibility to select default +preferences between public and system delegation, +xmlCatalogSetDefaultPrefer() allows this, xmlCatalogSetDefaults() and +xmlCatalogGetDefaults() allow to control if XML Catalogs resolution should +be forbidden, allowed for global catalog, for document catalog or both, the +default is to allow both.</p> + +<p>And of course xmlCatalogSetDebug() allows to generate debug messages +(through the xmlGenericError() mechanism).</p> + +<h4>Querying routines:</h4> + +<p>xmlCatalogResolve(), xmlCatalogResolveSystem(), xmlCatalogResolvePublic() +and xmlCatalogResolveURI() are relatively explicit if you read the XML +Catalog specification they correspond to section 7 algorithms, they should +also work if you have loaded an SGML catalog with a simplified semantic.</p> + +<p>xmlCatalogLocalResolve() and xmlCatalogLocalResolveURI() are the same but +operate on the document catalog list</p> + +<h4>Cleanup and Miscellaneous:</h4> + +<p>xmlCatalogCleanup() free-up the global catalog, xmlCatalogFreeLocal() is +the per-document equivalent.</p> + +<p>xmlCatalogAdd() and xmlCatalogRemove() are used to dynamically modify the +first catalog in the global list, and xmlCatalogDump() allows to dump a +catalog state, those routines are primarily designed for xmlcatalog, I'm not +sure that exposing more complex interfaces (like navigation ones) would be +really useful.</p> + +<p>The xmlParseCatalogFile() is a function used to load XML Catalog files, +it's similar as xmlParseFile() except it bypass all catalog lookups, it's +provided because this functionality may be useful for client tools.</p> + +<h4>threaded environments:</h4> + +<p>Since the catalog tree is built progressively, some care has been taken to +try to avoid troubles in multithreaded environments. The code is now thread +safe assuming that the libxml2 library has been compiled with threads +support.</p> + +<p></p> + +<h3><a name="Other">Other resources</a></h3> + +<p>The XML Catalog specification is relatively recent so there isn't much +literature to point at:</p> +<ul> + <li>You can find a good rant from Norm Walsh about <a + href="http://www.arbortext.com/Think_Tank/XML_Resources/Issue_Three/issue_three.html">the + need for catalogs</a>, it provides a lot of context informations even if + I don't agree with everything presented. Norm also wrote a more recent + article <a + href="http://wwws.sun.com/software/xml/developers/resolver/article/">XML + entities and URI resolvers</a> describing them.</li> + <li>An <a href="http://home.ccil.org/~cowan/XML/XCatalog.html">old XML + catalog proposal</a> from John Cowan</li> + <li>The <a href="http://www.rddl.org/">Resource Directory Description + Language</a> (RDDL) another catalog system but more oriented toward + providing metadata for XML namespaces.</li> + <li>the page from the OASIS Technical <a + href="http://www.oasis-open.org/committees/entity/">Committee on Entity + Resolution</a> who maintains XML Catalog, you will find pointers to the + specification update, some background and pointers to others tools + providing XML Catalog support</li> + <li>There is a <a href="buildDocBookCatalog">shell script</a> to generate + XML Catalogs for DocBook 4.1.2 . If it can write to the /etc/xml/ + directory, it will set-up /etc/xml/catalog and /etc/xml/docbook based on + the resources found on the system. Otherwise it will just create + ~/xmlcatalog and ~/dbkxmlcatalog and doing: + <p><code>export XML_CATALOG_FILES=$HOME/xmlcatalog</code></p> + <p>should allow to process DocBook documentations without requiring + network accesses for the DTD or stylesheets</p> + </li> + <li>I have uploaded <a href="ftp://xmlsoft.org/test/dbk412catalog.tar.gz">a + small tarball</a> containing XML Catalogs for DocBook 4.1.2 which seems + to work fine for me too</li> + <li>The <a href="http://www.xmlsoft.org/xmlcatalog_man.html">xmlcatalog + manual page</a></li> +</ul> + +<p>If you have suggestions for corrections or additions, simply contact +me:</p> + +<h2><a name="library">The parser interfaces</a></h2> + +<p>This section is directly intended to help programmers getting bootstrapped +using the XML tollkit from the C language. It is not intended to be +extensive. I hope the automatically generated documents will provide the +completeness required, but as a separate set of documents. The interfaces of +the XML parser are by principle low level, Those interested in a higher level +API should <a href="#DOM">look at DOM</a>.</p> + +<p>The <a href="html/libxml-parser.html">parser interfaces for XML</a> are +separated from the <a href="html/libxml-htmlparser.html">HTML parser +interfaces</a>. Let's have a look at how the XML parser can be called:</p> + +<h3><a name="Invoking">Invoking the parser : the pull method</a></h3> + +<p>Usually, the first thing to do is to read an XML input. The parser accepts +documents either from in-memory strings or from files. The functions are +defined in "parser.h":</p> +<dl> + <dt><code>xmlDocPtr xmlParseMemory(char *buffer, int size);</code></dt> + <dd><p>Parse a null-terminated string containing the document.</p> + </dd> +</dl> +<dl> + <dt><code>xmlDocPtr xmlParseFile(const char *filename);</code></dt> + <dd><p>Parse an XML document contained in a (possibly compressed) + file.</p> + </dd> +</dl> + +<p>The parser returns a pointer to the document structure (or NULL in case of +failure).</p> + +<h3 id="Invoking1">Invoking the parser: the push method</h3> + +<p>In order for the application to keep the control when the document is +being fetched (which is common for GUI based programs) libxml2 provides a +push interface, too, as of version 1.8.3. Here are the interface +functions:</p> +<pre>xmlParserCtxtPtr xmlCreatePushParserCtxt(xmlSAXHandlerPtr sax, + void *user_data, + const char *chunk, + int size, + const char *filename); +int xmlParseChunk (xmlParserCtxtPtr ctxt, + const char *chunk, + int size, + int terminate);</pre> + +<p>and here is a simple example showing how to use the interface:</p> +<pre> FILE *f; + + f = fopen(filename, "r"); + if (f != NULL) { + int res, size = 1024; + char chars[1024]; + xmlParserCtxtPtr ctxt; + + res = fread(chars, 1, 4, f); + if (res > 0) { + ctxt = xmlCreatePushParserCtxt(NULL, NULL, + chars, res, filename); + while ((res = fread(chars, 1, size, f)) > 0) { + xmlParseChunk(ctxt, chars, res, 0); + } + xmlParseChunk(ctxt, chars, 0, 1); + doc = ctxt->myDoc; + xmlFreeParserCtxt(ctxt); + } + }</pre> + +<p>The HTML parser embedded into libxml2 also has a push interface; the +functions are just prefixed by "html" rather than "xml".</p> + +<h3 id="Invoking2">Invoking the parser: the SAX interface</h3> + +<p>The tree-building interface makes the parser memory-hungry, first loading +the document in memory and then building the tree itself. Reading a document +without building the tree is possible using the SAX interfaces (see SAX.h and +<a href="http://www.daa.com.au/~james/gnome/xml-sax/xml-sax.html">James +Henstridge's documentation</a>). Note also that the push interface can be +limited to SAX: just use the two first arguments of +<code>xmlCreatePushParserCtxt()</code>.</p> + +<h3><a name="Building">Building a tree from scratch</a></h3> + +<p>The other way to get an XML tree in memory is by building it. Basically +there is a set of functions dedicated to building new elements. (These are +also described in <libxml/tree.h>.) For example, here is a piece of +code that produces the XML document used in the previous examples:</p> +<pre> #include <libxml/tree.h> + xmlDocPtr doc; + xmlNodePtr tree, subtree; + + doc = xmlNewDoc("1.0"); + doc->children = xmlNewDocNode(doc, NULL, "EXAMPLE", NULL); + xmlSetProp(doc->children, "prop1", "gnome is great"); + xmlSetProp(doc->children, "prop2", "& linux too"); + tree = xmlNewChild(doc->children, NULL, "head", NULL); + subtree = xmlNewChild(tree, NULL, "title", "Welcome to Gnome"); + tree = xmlNewChild(doc->children, NULL, "chapter", NULL); + subtree = xmlNewChild(tree, NULL, "title", "The Linux adventure"); + subtree = xmlNewChild(tree, NULL, "p", "bla bla bla ..."); + subtree = xmlNewChild(tree, NULL, "image", NULL); + xmlSetProp(subtree, "href", "linus.gif");</pre> + +<p>Not really rocket science ...</p> + +<h3><a name="Traversing">Traversing the tree</a></h3> + +<p>Basically by <a href="html/libxml-tree.html">including "tree.h"</a> your +code has access to the internal structure of all the elements of the tree. +The names should be somewhat simple like <strong>parent</strong>, +<strong>children</strong>, <strong>next</strong>, <strong>prev</strong>, +<strong>properties</strong>, etc... For example, still with the previous +example:</p> +<pre><code>doc->children->children->children</code></pre> + +<p>points to the title element,</p> +<pre>doc->children->children->next->children->children</pre> + +<p>points to the text node containing the chapter title "The Linux +adventure".</p> + +<p><strong>NOTE</strong>: XML allows <em>PI</em>s and <em>comments</em> to be +present before the document root, so <code>doc->children</code> may point +to an element which is not the document Root Element; a function +<code>xmlDocGetRootElement()</code> was added for this purpose.</p> + +<h3><a name="Modifying">Modifying the tree</a></h3> + +<p>Functions are provided for reading and writing the document content. Here +is an excerpt from the <a href="html/libxml-tree.html">tree API</a>:</p> +<dl> + <dt><code>xmlAttrPtr xmlSetProp(xmlNodePtr node, const xmlChar *name, const + xmlChar *value);</code></dt> + <dd><p>This sets (or changes) an attribute carried by an ELEMENT node. + The value can be NULL.</p> + </dd> +</dl> +<dl> + <dt><code>const xmlChar *xmlGetProp(xmlNodePtr node, const xmlChar + *name);</code></dt> + <dd><p>This function returns a pointer to new copy of the property + content. Note that the user must deallocate the result.</p> + </dd> +</dl> + +<p>Two functions are provided for reading and writing the text associated +with elements:</p> +<dl> + <dt><code>xmlNodePtr xmlStringGetNodeList(xmlDocPtr doc, const xmlChar + *value);</code></dt> + <dd><p>This function takes an "external" string and converts it to one + text node or possibly to a list of entity and text nodes. All + non-predefined entity references like &Gnome; will be stored + internally as entity nodes, hence the result of the function may not be + a single node.</p> + </dd> +</dl> +<dl> + <dt><code>xmlChar *xmlNodeListGetString(xmlDocPtr doc, xmlNodePtr list, int + inLine);</code></dt> + <dd><p>This function is the inverse of + <code>xmlStringGetNodeList()</code>. It generates a new string + containing the content of the text and entity nodes. Note the extra + argument inLine. If this argument is set to 1, the function will expand + entity references. For example, instead of returning the &Gnome; + XML encoding in the string, it will substitute it with its value (say, + "GNU Network Object Model Environment").</p> + </dd> +</dl> + +<h3><a name="Saving">Saving a tree</a></h3> + +<p>Basically 3 options are possible:</p> +<dl> + <dt><code>void xmlDocDumpMemory(xmlDocPtr cur, xmlChar**mem, int + *size);</code></dt> + <dd><p>Returns a buffer into which the document has been saved.</p> + </dd> +</dl> +<dl> + <dt><code>extern void xmlDocDump(FILE *f, xmlDocPtr doc);</code></dt> + <dd><p>Dumps a document to an open file descriptor.</p> + </dd> +</dl> +<dl> + <dt><code>int xmlSaveFile(const char *filename, xmlDocPtr cur);</code></dt> + <dd><p>Saves the document to a file. In this case, the compression + interface is triggered if it has been turned on.</p> + </dd> +</dl> + +<h3><a name="Compressio">Compression</a></h3> + +<p>The library transparently handles compression when doing file-based +accesses. The level of compression on saves can be turned on either globally +or individually for one file:</p> +<dl> + <dt><code>int xmlGetDocCompressMode (xmlDocPtr doc);</code></dt> + <dd><p>Gets the document compression ratio (0-9).</p> + </dd> +</dl> +<dl> + <dt><code>void xmlSetDocCompressMode (xmlDocPtr doc, int mode);</code></dt> + <dd><p>Sets the document compression ratio.</p> + </dd> +</dl> +<dl> + <dt><code>int xmlGetCompressMode(void);</code></dt> + <dd><p>Gets the default compression ratio.</p> + </dd> +</dl> +<dl> + <dt><code>void xmlSetCompressMode(int mode);</code></dt> + <dd><p>Sets the default compression ratio.</p> + </dd> +</dl> + +<h2><a name="Entities">Entities or no entities</a></h2> + +<p>Entities in principle are similar to simple C macros. An entity defines an +abbreviation for a given string that you can reuse many times throughout the +content of your document. Entities are especially useful when a given string +may occur frequently within a document, or to confine the change needed to a +document to a restricted area in the internal subset of the document (at the +beginning). Example:</p> +<pre>1 <?xml version="1.0"?> +2 <!DOCTYPE EXAMPLE SYSTEM "example.dtd" [ +3 <!ENTITY xml "Extensible Markup Language"> +4 ]> +5 <EXAMPLE> +6 &xml; +7 </EXAMPLE></pre> + +<p>Line 3 declares the xml entity. Line 6 uses the xml entity, by prefixing +its name with '&' and following it by ';' without any spaces added. There +are 5 predefined entities in libxml2 allowing you to escape characters with +predefined meaning in some parts of the xml document content: +<strong>&lt;</strong> for the character '<', <strong>&gt;</strong> +for the character '>', <strong>&apos;</strong> for the character ''', +<strong>&quot;</strong> for the character '"', and +<strong>&amp;</strong> for the character '&'.</p> + +<p>One of the problems related to entities is that you may want the parser to +substitute an entity's content so that you can see the replacement text in +your application. Or you may prefer to keep entity references as such in the +content to be able to save the document back without losing this usually +precious information (if the user went through the pain of explicitly +defining entities, he may have a a rather negative attitude if you blindly +substitute them as saving time). The <a +href="html/libxml-parser.html#xmlSubstituteEntitiesDefault">xmlSubstituteEntitiesDefault()</a> +function allows you to check and change the behaviour, which is to not +substitute entities by default.</p> + +<p>Here is the DOM tree built by libxml2 for the previous document in the +default case:</p> +<pre>/gnome/src/gnome-xml -> ./xmllint --debug test/ent1 +DOCUMENT +version=1.0 + ELEMENT EXAMPLE + TEXT + content= + ENTITY_REF + INTERNAL_GENERAL_ENTITY xml + content=Extensible Markup Language + TEXT + content=</pre> + +<p>And here is the result when substituting entities:</p> +<pre>/gnome/src/gnome-xml -> ./tester --debug --noent test/ent1 +DOCUMENT +version=1.0 + ELEMENT EXAMPLE + TEXT + content= Extensible Markup Language</pre> + +<p>So, entities or no entities? Basically, it depends on your use case. I +suggest that you keep the non-substituting default behaviour and avoid using +entities in your XML document or data if you are not willing to handle the +entity references elements in the DOM tree.</p> + +<p>Note that at save time libxml2 enforces the conversion of the predefined +entities where necessary to prevent well-formedness problems, and will also +transparently replace those with chars (i.e. it will not generate entity +reference elements in the DOM tree or call the reference() SAX callback when +finding them in the input).</p> + +<p><span style="background-color: #FF0000">WARNING</span>: handling entities +on top of the libxml2 SAX interface is difficult!!! If you plan to use +non-predefined entities in your documents, then the learning curve to handle +then using the SAX API may be long. If you plan to use complex documents, I +strongly suggest you consider using the DOM interface instead and let libxml +deal with the complexity rather than trying to do it yourself.</p> + +<h2><a name="Namespaces">Namespaces</a></h2> + +<p>The libxml2 library implements <a +href="http://www.w3.org/TR/REC-xml-names/">XML namespaces</a> support by +recognizing namespace constructs in the input, and does namespace lookup +automatically when building the DOM tree. A namespace declaration is +associated with an in-memory structure and all elements or attributes within +that namespace point to it. Hence testing the namespace is a simple and fast +equality operation at the user level.</p> + +<p>I suggest that people using libxml2 use a namespace, and declare it in the +root element of their document as the default namespace. Then they don't need +to use the prefix in the content but we will have a basis for future semantic +refinement and merging of data from different sources. This doesn't increase +the size of the XML output significantly, but significantly increases its +value in the long-term. Example:</p> +<pre><mydoc xmlns="http://mydoc.example.org/schemas/"> + <elem1>...</elem1> + <elem2>...</elem2> +</mydoc></pre> + +<p>The namespace value has to be an absolute URL, but the URL doesn't have to +point to any existing resource on the Web. It will bind all the element and +attributes with that URL. I suggest to use an URL within a domain you +control, and that the URL should contain some kind of version information if +possible. For example, <code>"http://www.gnome.org/gnumeric/1.0/"</code> is a +good namespace scheme.</p> + +<p>Then when you load a file, make sure that a namespace carrying the +version-independent prefix is installed on the root element of your document, +and if the version information don't match something you know, warn the user +and be liberal in what you accept as the input. Also do *not* try to base +namespace checking on the prefix value. <foo:text> may be exactly the +same as <bar:text> in another document. What really matters is the URI +associated with the element or the attribute, not the prefix string (which is +just a shortcut for the full URI). In libxml, element and attributes have an +<code>ns</code> field pointing to an xmlNs structure detailing the namespace +prefix and its URI.</p> + +<p>@@Interfaces@@</p> +<pre>xmlNodePtr node; +if(!strncmp(node->name,"mytag",5) + && node->ns + && !strcmp(node->ns->href,"http://www.mysite.com/myns/1.0")) { + ... +}</pre> + +<p>Usually people object to using namespaces together with validity checking. +I will try to make sure that using namespaces won't break validity checking, +so even if you plan to use or currently are using validation I strongly +suggest adding namespaces to your document. A default namespace scheme +<code>xmlns="http://...."</code> should not break validity even on less +flexible parsers. Using namespaces to mix and differentiate content coming +from multiple DTDs will certainly break current validation schemes. To check +such documents one needs to use schema-validation, which is supported in +libxml2 as well. See <a href="http://www.relaxng.org/">relagx-ng</a> and <a +href="http://www.w3c.org/XML/Schema">w3c-schema</a>.</p> + +<h2><a name="Upgrading">Upgrading 1.x code</a></h2> + +<p>Incompatible changes:</p> + +<p>Version 2 of libxml2 is the first version introducing serious backward +incompatible changes. The main goals were:</p> +<ul> + <li>a general cleanup. A number of mistakes inherited from the very early + versions couldn't be changed due to compatibility constraints. Example + the "childs" element in the nodes.</li> + <li>Uniformization of the various nodes, at least for their header and link + parts (doc, parent, children, prev, next), the goal is a simpler + programming model and simplifying the task of the DOM implementors.</li> + <li>better conformances to the XML specification, for example version 1.x + had an heuristic to try to detect ignorable white spaces. As a result the + SAX event generated were ignorableWhitespace() while the spec requires + character() in that case. This also mean that a number of DOM node + containing blank text may populate the DOM tree which were not present + before.</li> +</ul> + +<h3>How to fix libxml-1.x code:</h3> + +<p>So client code of libxml designed to run with version 1.x may have to be +changed to compile against version 2.x of libxml. Here is a list of changes +that I have collected, they may not be sufficient, so in case you find other +change which are required, <a href="mailto:Daniel.Veillard@w3.org">drop me a +mail</a>:</p> +<ol> + <li>The package name have changed from libxml to libxml2, the library name + is now -lxml2 . There is a new xml2-config script which should be used to + select the right parameters libxml2</li> + <li>Node <strong>childs</strong> field has been renamed + <strong>children</strong> so s/childs/children/g should be applied + (probability of having "childs" anywhere else is close to 0+</li> + <li>The document don't have anymore a <strong>root</strong> element it has + been replaced by <strong>children</strong> and usually you will get a + list of element here. For example a Dtd element for the internal subset + and it's declaration may be found in that list, as well as processing + instructions or comments found before or after the document root element. + Use <strong>xmlDocGetRootElement(doc)</strong> to get the root element of + a document. Alternatively if you are sure to not reference DTDs nor have + PIs or comments before or after the root element + s/->root/->children/g will probably do it.</li> + <li>The white space issue, this one is more complex, unless special case of + validating parsing, the line breaks and spaces usually used for indenting + and formatting the document content becomes significant. So they are + reported by SAX and if your using the DOM tree, corresponding nodes are + generated. Too approach can be taken: + <ol> + <li>lazy one, use the compatibility call + <strong>xmlKeepBlanksDefault(0)</strong> but be aware that you are + relying on a special (and possibly broken) set of heuristics of + libxml to detect ignorable blanks. Don't complain if it breaks or + make your application not 100% clean w.r.t. to it's input.</li> + <li>the Right Way: change you code to accept possibly insignificant + blanks characters, or have your tree populated with weird blank text + nodes. You can spot them using the commodity function + <strong>xmlIsBlankNode(node)</strong> returning 1 for such blank + nodes.</li> + </ol> + <p>Note also that with the new default the output functions don't add any + extra indentation when saving a tree in order to be able to round trip + (read and save) without inflating the document with extra formatting + chars.</p> + </li> + <li>The include path has changed to $prefix/libxml/ and the includes + themselves uses this new prefix in includes instructions... If you are + using (as expected) the + <pre>xml2-config --cflags</pre> + <p>output to generate you compile commands this will probably work out of + the box</p> + </li> + <li>xmlDetectCharEncoding takes an extra argument indicating the length in + byte of the head of the document available for character detection.</li> +</ol> + +<h3>Ensuring both libxml-1.x and libxml-2.x compatibility</h3> + +<p>Two new version of libxml (1.8.11) and libxml2 (2.3.4) have been released +to allow smooth upgrade of existing libxml v1code while retaining +compatibility. They offers the following:</p> +<ol> + <li>similar include naming, one should use + <strong>#include<libxml/...></strong> in both cases.</li> + <li>similar identifiers defined via macros for the child and root fields: + respectively <strong>xmlChildrenNode</strong> and + <strong>xmlRootNode</strong></li> + <li>a new macro <strong>LIBXML_TEST_VERSION</strong> which should be + inserted once in the client code</li> +</ol> + +<p>So the roadmap to upgrade your existing libxml applications is the +following:</p> +<ol> + <li>install the libxml-1.8.8 (and libxml-devel-1.8.8) packages</li> + <li>find all occurrences where the xmlDoc <strong>root</strong> field is + used and change it to <strong>xmlRootNode</strong></li> + <li>similarly find all occurrences where the xmlNode + <strong>childs</strong> field is used and change it to + <strong>xmlChildrenNode</strong></li> + <li>add a <strong>LIBXML_TEST_VERSION</strong> macro somewhere in your + <strong>main()</strong> or in the library init entry point</li> + <li>Recompile, check compatibility, it should still work</li> + <li>Change your configure script to look first for xml2-config and fall + back using xml-config . Use the --cflags and --libs output of the command + as the Include and Linking parameters needed to use libxml.</li> + <li>install libxml2-2.3.x and libxml2-devel-2.3.x (libxml-1.8.y and + libxml-devel-1.8.y can be kept simultaneously)</li> + <li>remove your config.cache, relaunch your configuration mechanism, and + recompile, if steps 2 and 3 were done right it should compile as-is</li> + <li>Test that your application is still running correctly, if not this may + be due to extra empty nodes due to formating spaces being kept in libxml2 + contrary to libxml1, in that case insert xmlKeepBlanksDefault(1) in your + code before calling the parser (next to + <strong>LIBXML_TEST_VERSION</strong> is a fine place).</li> +</ol> + +<p>Following those steps should work. It worked for some of my own code.</p> + +<p>Let me put some emphasis on the fact that there is far more changes from +libxml 1.x to 2.x than the ones you may have to patch for. The overall code +has been considerably cleaned up and the conformance to the XML specification +has been drastically improved too. Don't take those changes as an excuse to +not upgrade, it may cost a lot on the long term ...</p> + +<h2><a name="Thread">Thread safety</a></h2> + +<p>Starting with 2.4.7, libxml2 makes provisions to ensure that concurrent +threads can safely work in parallel parsing different documents. There is +however a couple of things to do to ensure it:</p> +<ul> + <li>configure the library accordingly using the --with-threads options</li> + <li>call xmlInitParser() in the "main" thread before using any of the + libxml2 API (except possibly selecting a different memory allocator)</li> +</ul> + +<p>Note that the thread safety cannot be ensured for multiple threads sharing +the same document, the locking must be done at the application level, libxml +exports a basic mutex and reentrant mutexes API in <libxml/threads.h>. +The parts of the library checked for thread safety are:</p> +<ul> + <li>concurrent loading</li> + <li>file access resolution</li> + <li>catalog access</li> + <li>catalog building</li> + <li>entities lookup/accesses</li> + <li>validation</li> + <li>global variables per-thread override</li> + <li>memory handling</li> +</ul> + +<p>XPath is supposed to be thread safe now, but this wasn't tested +seriously.</p> + +<h2><a name="DOM"></a><a name="Principles">DOM Principles</a></h2> + +<p><a href="http://www.w3.org/DOM/">DOM</a> stands for the <em>Document +Object Model</em>; this is an API for accessing XML or HTML structured +documents. Native support for DOM in Gnome is on the way (module gnome-dom), +and will be based on gnome-xml. This will be a far cleaner interface to +manipulate XML files within Gnome since it won't expose the internal +structure.</p> + +<p>The current DOM implementation on top of libxml2 is the <a +href="http://cvs.gnome.org/lxr/source/gdome2/">gdome2 Gnome module</a>, this +is a full DOM interface, thanks to Paolo Casarini, check the <a +href="http://www.cs.unibo.it/~casarini/gdome2/">Gdome2 homepage</a> for more +informations.</p> + +<h2><a name="Example"></a><a name="real">A real example</a></h2> + +<p>Here is a real size example, where the actual content of the application +data is not kept in the DOM tree but uses internal structures. It is based on +a proposal to keep a database of jobs related to Gnome, with an XML based +storage structure. Here is an <a href="gjobs.xml">XML encoded jobs +base</a>:</p> +<pre><?xml version="1.0"?> +<gjob:Helping xmlns:gjob="http://www.gnome.org/some-location"> + <gjob:Jobs> + + <gjob:Job> + <gjob:Project ID="3"/> + <gjob:Application>GBackup</gjob:Application> + <gjob:Category>Development</gjob:Category> + + <gjob:Update> + <gjob:Status>Open</gjob:Status> + <gjob:Modified>Mon, 07 Jun 1999 20:27:45 -0400 MET DST</gjob:Modified> + <gjob:Salary>USD 0.00</gjob:Salary> + </gjob:Update> + + <gjob:Developers> + <gjob:Developer> + </gjob:Developer> + </gjob:Developers> + + <gjob:Contact> + <gjob:Person>Nathan Clemons</gjob:Person> + <gjob:Email>nathan@windsofstorm.net</gjob:Email> + <gjob:Company> + </gjob:Company> + <gjob:Organisation> + </gjob:Organisation> + <gjob:Webpage> + </gjob:Webpage> + <gjob:Snailmail> + </gjob:Snailmail> + <gjob:Phone> + </gjob:Phone> + </gjob:Contact> + + <gjob:Requirements> + The program should be released as free software, under the GPL. + </gjob:Requirements> + + <gjob:Skills> + </gjob:Skills> + + <gjob:Details> + A GNOME based system that will allow a superuser to configure + compressed and uncompressed files and/or file systems to be backed + up with a supported media in the system. This should be able to + perform via find commands generating a list of files that are passed + to tar, dd, cpio, cp, gzip, etc., to be directed to the tape machine + or via operations performed on the filesystem itself. Email + notification and GUI status display very important. + </gjob:Details> + + </gjob:Job> + + </gjob:Jobs> +</gjob:Helping></pre> + +<p>While loading the XML file into an internal DOM tree is a matter of +calling only a couple of functions, browsing the tree to gather the data and +generate the internal structures is harder, and more error prone.</p> + +<p>The suggested principle is to be tolerant with respect to the input +structure. For example, the ordering of the attributes is not significant, +the XML specification is clear about it. It's also usually a good idea not to +depend on the order of the children of a given node, unless it really makes +things harder. Here is some code to parse the information for a person:</p> +<pre>/* + * A person record + */ +typedef struct person { + char *name; + char *email; + char *company; + char *organisation; + char *smail; + char *webPage; + char *phone; +} person, *personPtr; + +/* + * And the code needed to parse it + */ +personPtr parsePerson(xmlDocPtr doc, xmlNsPtr ns, xmlNodePtr cur) { + personPtr ret = NULL; + +DEBUG("parsePerson\n"); + /* + * allocate the struct + */ + ret = (personPtr) malloc(sizeof(person)); + if (ret == NULL) { + fprintf(stderr,"out of memory\n"); + return(NULL); + } + memset(ret, 0, sizeof(person)); + + /* We don't care what the top level element name is */ + cur = cur->xmlChildrenNode; + while (cur != NULL) { + if ((!strcmp(cur->name, "Person")) && (cur->ns == ns)) + ret->name = xmlNodeListGetString(doc, cur->xmlChildrenNode, 1); + if ((!strcmp(cur->name, "Email")) && (cur->ns == ns)) + ret->email = xmlNodeListGetString(doc, cur->xmlChildrenNode, 1); + cur = cur->next; + } + + return(ret); +}</pre> + +<p>Here are a couple of things to notice:</p> +<ul> + <li>Usually a recursive parsing style is the more convenient one: XML data + is by nature subject to repetitive constructs and usually exhibits highly + structured patterns.</li> + <li>The two arguments of type <em>xmlDocPtr</em> and <em>xmlNsPtr</em>, + i.e. the pointer to the global XML document and the namespace reserved to + the application. Document wide information are needed for example to + decode entities and it's a good coding practice to define a namespace for + your application set of data and test that the element and attributes + you're analyzing actually pertains to your application space. This is + done by a simple equality test (cur->ns == ns).</li> + <li>To retrieve text and attributes value, you can use the function + <em>xmlNodeListGetString</em> to gather all the text and entity reference + nodes generated by the DOM output and produce an single text string.</li> +</ul> + +<p>Here is another piece of code used to parse another level of the +structure:</p> +<pre>#include <libxml/tree.h> +/* + * a Description for a Job + */ +typedef struct job { + char *projectID; + char *application; + char *category; + personPtr contact; + int nbDevelopers; + personPtr developers[100]; /* using dynamic alloc is left as an exercise */ +} job, *jobPtr; + +/* + * And the code needed to parse it + */ +jobPtr parseJob(xmlDocPtr doc, xmlNsPtr ns, xmlNodePtr cur) { + jobPtr ret = NULL; + +DEBUG("parseJob\n"); + /* + * allocate the struct + */ + ret = (jobPtr) malloc(sizeof(job)); + if (ret == NULL) { + fprintf(stderr,"out of memory\n"); + return(NULL); + } + memset(ret, 0, sizeof(job)); + + /* We don't care what the top level element name is */ + cur = cur->xmlChildrenNode; + while (cur != NULL) { + + if ((!strcmp(cur->name, "Project")) && (cur->ns == ns)) { + ret->projectID = xmlGetProp(cur, "ID"); + if (ret->projectID == NULL) { + fprintf(stderr, "Project has no ID\n"); + } + } + if ((!strcmp(cur->name, "Application")) && (cur->ns == ns)) + ret->application = xmlNodeListGetString(doc, cur->xmlChildrenNode, 1); + if ((!strcmp(cur->name, "Category")) && (cur->ns == ns)) + ret->category = xmlNodeListGetString(doc, cur->xmlChildrenNode, 1); + if ((!strcmp(cur->name, "Contact")) && (cur->ns == ns)) + ret->contact = parsePerson(doc, ns, cur); + cur = cur->next; + } + + return(ret); +}</pre> + +<p>Once you are used to it, writing this kind of code is quite simple, but +boring. Ultimately, it could be possible to write stubbers taking either C +data structure definitions, a set of XML examples or an XML DTD and produce +the code needed to import and export the content between C data and XML +storage. This is left as an exercise to the reader :-)</p> + +<p>Feel free to use <a href="example/gjobread.c">the code for the full C +parsing example</a> as a template, it is also available with Makefile in the +Gnome CVS base under gnome-xml/example</p> + +<h2><a name="Contributi">Contributions</a></h2> +<ul> + <li>Bjorn Reese, William Brack and Thomas Broyer have provided a number of + patches, Gary Pennington worked on the validation API, threading support + and Solaris port.</li> + <li>John Fleck helps maintaining the documentation and man pages.</li> + <li><a href="mailto:igor@zlatkovic.com">Igor Zlatkovic</a> is now the + maintainer of the Windows port, <a + href="http://www.zlatkovic.com/projects/libxml/index.html">he provides + binaries</a></li> + <li><a href="mailto:Gary.Pennington@sun.com">Gary Pennington</a> provides + <a href="http://garypennington.net/libxml2/">Solaris binaries</a></li> + <li><a + href="http://mail.gnome.org/archives/xml/2001-March/msg00014.html">Matt + Sergeant</a> developed <a + href="http://axkit.org/download/">XML::LibXSLT</a>, a Perl wrapper for + libxml2/libxslt as part of the <a href="http://axkit.com/">AxKit XML + application server</a></li> + <li><a href="mailto:fnatter@gmx.net">Felix Natter</a> and <a + href="mailto:geertk@ai.rug.nl">Geert Kloosterman</a> provide <a + href="libxml-doc.el">an emacs module</a> to lookup libxml(2) functions + documentation</li> + <li><a href="mailto:sherwin@nlm.nih.gov">Ziying Sherwin</a> provided <a + href="http://xmlsoft.org/messages/0488.html">man pages</a></li> + <li>there is a module for <a + href="http://acs-misc.sourceforge.net/nsxml.html">libxml/libxslt support + in OpenNSD/AOLServer</a></li> + <li><a href="mailto:dkuhlman@cutter.rexx.com">Dave Kuhlman</a> provided the + first version of libxml/libxslt <a + href="http://www.rexx.com/~dkuhlman">wrappers for Python</a></li> + <li>Petr Kozelka provides <a + href="http://sourceforge.net/projects/libxml2-pas">Pascal units to glue + libxml2</a> with Kylix and Delphi and other Pascal compilers</li> + <li><a href="mailto:aleksey@aleksey.com">Aleksey Sanin</a> implemented the + <a href="http://www.w3.org/Signature/">XML Canonicalization and XML + Digital Signature</a> <a + href="http://www.aleksey.com/xmlsec/">implementations for libxml2</a></li> + <li><a href="mailto:Steve.Ball@zveno.com">Steve Ball</a>, <a + href="http://www.zveno.com/">Zveno</a> and contributors maintain <a + href="http://tclxml.sourceforge.net/">tcl bindings for libxml2 and + libxslt</a>, as well as <a + href="http://tclxml.sf.net/tkxmllint.html">tkxmllint</a> a GUI for + xmllint and <a href="http://tclxml.sf.net/tkxsltproc.html">tkxsltproc</a> + a GUI for xsltproc.</li> +</ul> + +<p></p> +</body> +</html> |