1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
|
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" /><link rel="SHORTCUT ICON" href="/favicon.ico" /><style type="text/css">
TD {font-family: Verdana,Arial,Helvetica}
BODY {font-family: Verdana,Arial,Helvetica; margin-top: 2em; margin-left: 0em; margin-right: 0em}
H1 {font-family: Verdana,Arial,Helvetica}
H2 {font-family: Verdana,Arial,Helvetica}
H3 {font-family: Verdana,Arial,Helvetica}
A:link, A:visited, A:active { text-decoration: underline }
</style><title>Memory Management</title></head><body bgcolor="#8b7765" text="#000000" link="#a06060" vlink="#000000"><table border="0" width="100%" cellpadding="5" cellspacing="0" align="center"><tr><td width="120"><a href="http://swpat.ffii.org/"><img src="epatents.png" alt="Action against software patents" /></a></td><td width="180"><a href="http://www.gnome.org/"><img src="gnome2.png" alt="Gnome2 Logo" /></a><a href="http://www.w3.org/Status"><img src="w3c.png" alt="W3C Logo" /></a><a href="http://www.redhat.com/"><img src="redhat.gif" alt="Red Hat Logo" /></a><div align="left"><a href="http://xmlsoft.org/"><img src="Libxml2-Logo-180x168.gif" alt="Made with Libxml2 Logo" /></a></div></td><td><table border="0" width="90%" cellpadding="2" cellspacing="0" align="center" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3" bgcolor="#fffacd"><tr><td align="center"><h1>The XML C parser and toolkit of Gnome</h1><h2>Memory Management</h2></td></tr></table></td></tr></table></td></tr></table><table border="0" cellpadding="4" cellspacing="0" width="100%" align="center"><tr><td bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="2" width="100%"><tr><td valign="top" width="200" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Developer Menu</b></center></td></tr><tr><td bgcolor="#fffacd"><form action="search.php" enctype="application/x-www-form-urlencoded" method="get"><input name="query" type="text" size="20" value="" /><input name="submit" type="submit" value="Search ..." /></form><ul><li><a href="index.html" style="font-weight:bold">Main Menu</a></li><li><a href="html/index.html" style="font-weight:bold">Reference Manual</a></li><li><a href="examples/index.html" style="font-weight:bold">Code Examples</a></li><li><a href="guidelines.html">XML Guidelines</a></li><li><a href="tutorial/index.html">Tutorial</a></li><li><a href="xmlreader.html">The Reader Interface</a></li><li><a href="ChangeLog.html">ChangeLog</a></li><li><a href="XSLT.html">XSLT</a></li><li><a href="python.html">Python and bindings</a></li><li><a href="architecture.html">libxml2 architecture</a></li><li><a href="tree.html">The tree output</a></li><li><a href="interface.html">The SAX interface</a></li><li><a href="xmlmem.html">Memory Management</a></li><li><a href="xmlio.html">I/O Interfaces</a></li><li><a href="library.html">The parser interfaces</a></li><li><a href="entities.html">Entities or no entities</a></li><li><a href="namespaces.html">Namespaces</a></li><li><a href="upgrade.html">Upgrading 1.x code</a></li><li><a href="threads.html">Thread safety</a></li><li><a href="DOM.html">DOM Principles</a></li><li><a href="example.html">A real example</a></li><li><a href="xml.html">flat page</a>, <a href="site.xsl">stylesheet</a></li></ul></td></tr></table><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>API Indexes</b></center></td></tr><tr><td bgcolor="#fffacd"><ul><li><a href="APIchunk0.html">Alphabetic</a></li><li><a href="APIconstructors.html">Constructors</a></li><li><a href="APIfunctions.html">Functions/Types</a></li><li><a href="APIfiles.html">Modules</a></li><li><a href="APIsymbols.html">Symbols</a></li></ul></td></tr></table><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Related links</b></center></td></tr><tr><td bgcolor="#fffacd"><ul><li><a href="http://mail.gnome.org/archives/xml/">Mail archive</a></li><li><a href="http://xmlsoft.org/XSLT/">XSLT libxslt</a></li><li><a href="http://phd.cs.unibo.it/gdome2/">DOM gdome2</a></li><li><a href="http://www.aleksey.com/xmlsec/">XML-DSig xmlsec</a></li><li><a href="ftp://xmlsoft.org/">FTP</a></li><li><a href="http://www.zlatkovic.com/projects/libxml/">Windows binaries</a></li><li><a href="http://www.blastwave.org/packages.php/libxml2">Solaris binaries</a></li><li><a href="http://www.explain.com.au/oss/libxml2xslt.html">MacOsX binaries</a></li><li><a href="http://libxmlplusplus.sourceforge.net/">C++ bindings</a></li><li><a href="http://www.zend.com/php5/articles/php5-xmlphp.php#Heading4">PHP bindings</a></li><li><a href="http://sourceforge.net/projects/libxml2-pas/">Pascal bindings</a></li><li><a href="http://libxml.rubyforge.org/">Ruby bindings</a></li><li><a href="http://tclxml.sourceforge.net/">Tcl bindings</a></li><li><a href="http://bugzilla.gnome.org/buglist.cgi?product=libxml2">Bug Tracker</a></li></ul></td></tr></table></td></tr></table></td><td valign="top" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%"><tr><td><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table border="0" cellpadding="3" cellspacing="1" width="100%"><tr><td bgcolor="#fffacd"><p>Table of Content:</p><ol><li><a href="#General3">General overview</a></li>
<li><a href="#setting">Setting libxml2 set of memory routines</a></li>
<li><a href="#cleanup">Cleaning up after parsing</a></li>
<li><a href="#Debugging">Debugging routines</a></li>
<li><a href="#General4">General memory requirements</a></li>
</ol><h3><a name="General3" id="General3">General overview</a></h3><p>The module <code><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlmemory.h</a></code>providesthe
interfaces to the libxml2 memory system:</p><ul><li>libxml2 does not use the libc memory allocator directly
butxmlFree(),xmlMalloc() and xmlRealloc()</li>
<li>those routines can be reallocated to a specific set of
routine,bydefault the libc ones i.e. free(), malloc() and realloc()</li>
<li>the xmlmemory.c module includes a set of debugging routine</li>
</ul><h3><a name="setting" id="setting">Setting libxml2 set of memory routines</a></h3><p>It is sometimes useful to not use the default memory allocator,
eitherfordebugging, analysis or to implement a specific behaviour on
memorymanagement(like on embedded systems). Two function calls are available
to doso:</p><ul><li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemGet()</a>whichreturn
the current set of functions in use by the parser</li>
<li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemSetup()</a>whichallow
to set up a new set of memory allocation functions</li>
</ul><p>Of course a call to xmlMemSetup() should probably be done beforecallingany
other libxml2 routines (unless you are sure your allocationsroutines
arecompatibles).</p><h3><a name="cleanup" id="cleanup">Cleaning up after parsing</a></h3><p>Libxml2 is not stateless, there is a few set of memory
structuresneedingallocation before the parser is fully functional (some
encodingstructuresfor example). This also mean that once parsing is finished
there isa tinyamount of memory (a few hundred bytes) which can be recollected
if youdon'treuse the parser immediately:</p><ul><li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlCleanupParser()</a>isa
centralized routine to free the parsing states. Note that
itwon'tdeallocate any produced tree if any (use the xmlFreeDoc()
andrelatedroutines for this).</li>
<li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlInitParser()</a>isthe
dual routine allowing to preallocate the parsing statewhich can beuseful
for example to avoid initialization reentrancyproblems when usinglibxml2
in multithreaded applications</li>
</ul><p>Generally xmlCleanupParser() is safe, if needed the state will berebuildat
the next invocation of parser routines, but be careful of theconsequencesin
multithreaded applications.</p><h3><a name="Debugging" id="Debugging">Debugging routines</a></h3><p>When configured using --with-mem-debug flag (off by default), libxml2usesa
set of memory allocation debugging routines keeping track of
allallocatedblocks and the location in the code where the routine was called.
Acouple ofother debugging routines allow to dump the memory allocated infos
toa fileor call a specific routine when a given block number is allocated:</p><ul><li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMallocLoc()</a><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlReallocLoc()</a>and<a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemStrdupLoc()</a>arethe
memory debugging replacement allocation routines</li>
<li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemoryDump()</a>dumpsall
the informations about the allocated memory block leftsin
the<code>.memdump</code>file</li>
</ul><p>When developing libxml2 memory debug is enabled, the tests
programscallxmlMemoryDump () and the "make test" regression tests will check
foranymemory leak during the full regression test sequence, this helps
alotensuring that libxml2 does not leak memory and bullet
proofmemoryallocations use (some libc implementations are known to be far
toopermissiveresulting in major portability problems!).</p><p>If the .memdump reports a leak, it displays the allocation functionandalso
tries to give some informations about the content and structure
oftheallocated blocks left. This is sufficient in most cases to find
theculprit,but not always. Assuming the allocation problem is reproducible,
itispossible to find more easily:</p><ol><li>write down the block number xxxx not allocated</li>
<li>export the environment variable XML_MEM_BREAKPOINT=xxxx ,
theeasiestwhen using GDB is to simply give the command
<p><code>set environment XML_MEM_BREAKPOINT xxxx</code></p>
<p>before running the program.</p>
</li>
<li>run the program under a debugger and set a
breakpointonxmlMallocBreakpoint() a specific function called when this
preciseblockis allocated</li>
<li>when the breakpoint is reached you can then do a fine analysis
oftheallocation an step to see the condition resulting in
themissingdeallocation.</li>
</ol><p>I used to use a commercial tool to debug libxml2 memory problems
butafternoticing that it was not detecting memory leaks that simple
mechanismwasused and proved extremely efficient until now. Lately I have also
used <a href="http://developer.kde.org/~sewardj/">valgrind</a>with quite
somesuccess,it is tied to the i386 architecture since it works by emulating
theprocessorand instruction set, it is slow but extremely efficient, i.e.
itspot memoryusage errors in a very precise way.</p><h3><a name="General4" id="General4">General memory requirements</a></h3><p>How much libxml2 memory require ? It's hard to tell in average itdependsof
a number of things:</p><ul><li>the parser itself should work in a fixed amount of memory,
exceptforinformation maintained about the stacks of names and
entitieslocations.The I/O and encoding handlers will probably account for
a fewKBytes.This is true for both the XML and HTML parser (though the
HTMLparserneed more state).</li>
<li>If you are generating the DOM tree then memory requirements
willgrownearly linear with the size of the data. In general for
abalancedtextual document the internal memory requirement is about 4
timesthesize of the UTF8 serialization of this document (example
theXML-1.0recommendation is a bit more of 150KBytes and takes 650KBytes
ofmainmemory when parsed). Validation will add a amount of memory
requiredformaintaining the external Dtd state which should be linear
withthecomplexity of the content model defined by the Dtd</li>
<li>If you need to work with fixed memory requirements or don't needthefull
DOM tree then using the <a href="xmlreader.html">xmlReaderinterface</a>is
probably the best way toproceed, it still allows tovalidate or operate on
subset of the tree ifneeded.</li>
<li>If you don't care about the advanced features of libxml2likevalidation,
DOM, XPath or XPointer, don't use entities, need to workwithfixed memory
requirements, and try to get the fastest parsingpossiblethen the SAX
interface should be used, but it has knownrestrictions.</li>
</ul><p></p><p><a href="bugs.html">Daniel Veillard</a></p></td></tr></table></td></tr></table></td></tr></table></td></tr></table></td></tr></table></body></html>
|
