diff options
Diffstat (limited to 'doc/xmlmem.html')
| -rw-r--r-- | doc/xmlmem.html | 166 |
1 files changed, 83 insertions, 83 deletions
diff --git a/doc/xmlmem.html b/doc/xmlmem.html index 40dc4cb..58c2987 100644 --- a/doc/xmlmem.html +++ b/doc/xmlmem.html @@ -12,91 +12,91 @@ A:link, A:visited, A:active { text-decoration: underline } <li><a href="#cleanup">Cleaning up after parsing</a></li> <li><a href="#Debugging">Debugging routines</a></li> <li><a href="#General4">General memory requirements</a></li> -</ol><h3><a name="General3" id="General3">General overview</a></h3><p>The module <code><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlmemory.h</a></code>providesthe -interfaces to the libxml2 memory system:</p><ul><li>libxml2 does not use the libc memory allocator directly - butxmlFree(),xmlMalloc() and xmlRealloc()</li> - <li>those routines can be reallocated to a specific set of - routine,bydefault the libc ones i.e. free(), malloc() and realloc()</li> +</ol><h3><a name="General3" id="General3">General overview</a></h3><p>The module <code><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlmemory.h</a></code> +provides the interfaces to the libxml2 memory system:</p><ul><li>libxml2 does not use the libc memory allocator directly but xmlFree(), + xmlMalloc() and xmlRealloc()</li> + <li>those routines can be reallocated to a specific set of routine, by + default the libc ones i.e. free(), malloc() and realloc()</li> <li>the xmlmemory.c module includes a set of debugging routine</li> -</ul><h3><a name="setting" id="setting">Setting libxml2 set of memory routines</a></h3><p>It is sometimes useful to not use the default memory allocator, -eitherfordebugging, analysis or to implement a specific behaviour on -memorymanagement(like on embedded systems). Two function calls are available -to doso:</p><ul><li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemGet()</a>whichreturn - the current set of functions in use by the parser</li> - <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemSetup()</a>whichallow - to set up a new set of memory allocation functions</li> -</ul><p>Of course a call to xmlMemSetup() should probably be done beforecallingany -other libxml2 routines (unless you are sure your allocationsroutines -arecompatibles).</p><h3><a name="cleanup" id="cleanup">Cleaning up after parsing</a></h3><p>Libxml2 is not stateless, there is a few set of memory -structuresneedingallocation before the parser is fully functional (some -encodingstructuresfor example). This also mean that once parsing is finished -there isa tinyamount of memory (a few hundred bytes) which can be recollected -if youdon'treuse the parser immediately:</p><ul><li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlCleanupParser()</a>isa - centralized routine to free the parsing states. Note that - itwon'tdeallocate any produced tree if any (use the xmlFreeDoc() - andrelatedroutines for this).</li> - <li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlInitParser()</a>isthe - dual routine allowing to preallocate the parsing statewhich can beuseful - for example to avoid initialization reentrancyproblems when usinglibxml2 - in multithreaded applications</li> -</ul><p>Generally xmlCleanupParser() is safe, if needed the state will berebuildat -the next invocation of parser routines, but be careful of theconsequencesin -multithreaded applications.</p><h3><a name="Debugging" id="Debugging">Debugging routines</a></h3><p>When configured using --with-mem-debug flag (off by default), libxml2usesa -set of memory allocation debugging routines keeping track of -allallocatedblocks and the location in the code where the routine was called. -Acouple ofother debugging routines allow to dump the memory allocated infos -toa fileor call a specific routine when a given block number is allocated:</p><ul><li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMallocLoc()</a><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlReallocLoc()</a>and<a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemStrdupLoc()</a>arethe - memory debugging replacement allocation routines</li> - <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemoryDump()</a>dumpsall - the informations about the allocated memory block leftsin - the<code>.memdump</code>file</li> -</ul><p>When developing libxml2 memory debug is enabled, the tests -programscallxmlMemoryDump () and the "make test" regression tests will check -foranymemory leak during the full regression test sequence, this helps -alotensuring that libxml2 does not leak memory and bullet -proofmemoryallocations use (some libc implementations are known to be far -toopermissiveresulting in major portability problems!).</p><p>If the .memdump reports a leak, it displays the allocation functionandalso -tries to give some informations about the content and structure -oftheallocated blocks left. This is sufficient in most cases to find -theculprit,but not always. Assuming the allocation problem is reproducible, -itispossible to find more easily:</p><ol><li>write down the block number xxxx not allocated</li> - <li>export the environment variable XML_MEM_BREAKPOINT=xxxx , - theeasiestwhen using GDB is to simply give the command +</ul><h3><a name="setting" id="setting">Setting libxml2 set of memory routines</a></h3><p>It is sometimes useful to not use the default memory allocator, either for +debugging, analysis or to implement a specific behaviour on memory management +(like on embedded systems). Two function calls are available to do so:</p><ul><li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemGet + ()</a> which return the current set of functions in use by the parser</li> + <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemSetup()</a> + which allow to set up a new set of memory allocation functions</li> +</ul><p>Of course a call to xmlMemSetup() should probably be done before calling +any other libxml2 routines (unless you are sure your allocations routines are +compatibles).</p><h3><a name="cleanup" id="cleanup">Cleaning up after parsing</a></h3><p>Libxml2 is not stateless, there is a few set of memory structures needing +allocation before the parser is fully functional (some encoding structures +for example). This also mean that once parsing is finished there is a tiny +amount of memory (a few hundred bytes) which can be recollected if you don't +reuse the parser immediately:</p><ul><li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlCleanupParser + ()</a> is a centralized routine to free the parsing states. Note that it + won't deallocate any produced tree if any (use the xmlFreeDoc() and + related routines for this).</li> + <li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlInitParser + ()</a> is the dual routine allowing to preallocate the parsing state + which can be useful for example to avoid initialization reentrancy + problems when using libxml2 in multithreaded applications</li> +</ul><p>Generally xmlCleanupParser() is safe, if needed the state will be rebuild +at the next invocation of parser routines, but be careful of the consequences +in multithreaded applications.</p><h3><a name="Debugging" id="Debugging">Debugging routines</a></h3><p>When configured using --with-mem-debug flag (off by default), libxml2 uses +a set of memory allocation debugging routines keeping track of all allocated +blocks and the location in the code where the routine was called. A couple of +other debugging routines allow to dump the memory allocated infos to a file +or call a specific routine when a given block number is allocated:</p><ul><li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMallocLoc()</a> + <a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlReallocLoc()</a> + and <a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemStrdupLoc()</a> + are the memory debugging replacement allocation routines</li> + <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemoryDump + ()</a> dumps all the informations about the allocated memory block lefts + in the <code>.memdump</code> file</li> +</ul><p>When developing libxml2 memory debug is enabled, the tests programs call +xmlMemoryDump () and the "make test" regression tests will check for any +memory leak during the full regression test sequence, this helps a lot +ensuring that libxml2 does not leak memory and bullet proof memory +allocations use (some libc implementations are known to be far too permissive +resulting in major portability problems!).</p><p>If the .memdump reports a leak, it displays the allocation function and +also tries to give some informations about the content and structure of the +allocated blocks left. This is sufficient in most cases to find the culprit, +but not always. Assuming the allocation problem is reproducible, it is +possible to find more easily:</p><ol><li>write down the block number xxxx not allocated</li> + <li>export the environment variable XML_MEM_BREAKPOINT=xxxx , the easiest + when using GDB is to simply give the command <p><code>set environment XML_MEM_BREAKPOINT xxxx</code></p> <p>before running the program.</p> </li> - <li>run the program under a debugger and set a - breakpointonxmlMallocBreakpoint() a specific function called when this - preciseblockis allocated</li> - <li>when the breakpoint is reached you can then do a fine analysis - oftheallocation an step to see the condition resulting in - themissingdeallocation.</li> -</ol><p>I used to use a commercial tool to debug libxml2 memory problems -butafternoticing that it was not detecting memory leaks that simple -mechanismwasused and proved extremely efficient until now. Lately I have also -used <a href="http://developer.kde.org/~sewardj/">valgrind</a>with quite -somesuccess,it is tied to the i386 architecture since it works by emulating -theprocessorand instruction set, it is slow but extremely efficient, i.e. -itspot memoryusage errors in a very precise way.</p><h3><a name="General4" id="General4">General memory requirements</a></h3><p>How much libxml2 memory require ? It's hard to tell in average itdependsof -a number of things:</p><ul><li>the parser itself should work in a fixed amount of memory, - exceptforinformation maintained about the stacks of names and - entitieslocations.The I/O and encoding handlers will probably account for - a fewKBytes.This is true for both the XML and HTML parser (though the - HTMLparserneed more state).</li> - <li>If you are generating the DOM tree then memory requirements - willgrownearly linear with the size of the data. In general for - abalancedtextual document the internal memory requirement is about 4 - timesthesize of the UTF8 serialization of this document (example - theXML-1.0recommendation is a bit more of 150KBytes and takes 650KBytes - ofmainmemory when parsed). Validation will add a amount of memory - requiredformaintaining the external Dtd state which should be linear - withthecomplexity of the content model defined by the Dtd</li> - <li>If you need to work with fixed memory requirements or don't needthefull - DOM tree then using the <a href="xmlreader.html">xmlReaderinterface</a>is - probably the best way toproceed, it still allows tovalidate or operate on - subset of the tree ifneeded.</li> - <li>If you don't care about the advanced features of libxml2likevalidation, - DOM, XPath or XPointer, don't use entities, need to workwithfixed memory - requirements, and try to get the fastest parsingpossiblethen the SAX - interface should be used, but it has knownrestrictions.</li> + <li>run the program under a debugger and set a breakpoint on + xmlMallocBreakpoint() a specific function called when this precise block + is allocated</li> + <li>when the breakpoint is reached you can then do a fine analysis of the + allocation an step to see the condition resulting in the missing + deallocation.</li> +</ol><p>I used to use a commercial tool to debug libxml2 memory problems but after +noticing that it was not detecting memory leaks that simple mechanism was +used and proved extremely efficient until now. Lately I have also used <a href="http://developer.kde.org/~sewardj/">valgrind</a> with quite some +success, it is tied to the i386 architecture since it works by emulating the +processor and instruction set, it is slow but extremely efficient, i.e. it +spot memory usage errors in a very precise way.</p><h3><a name="General4" id="General4">General memory requirements</a></h3><p>How much libxml2 memory require ? It's hard to tell in average it depends +of a number of things:</p><ul><li>the parser itself should work in a fixed amount of memory, except for + information maintained about the stacks of names and entities locations. + The I/O and encoding handlers will probably account for a few KBytes. + This is true for both the XML and HTML parser (though the HTML parser + need more state).</li> + <li>If you are generating the DOM tree then memory requirements will grow + nearly linear with the size of the data. In general for a balanced + textual document the internal memory requirement is about 4 times the + size of the UTF8 serialization of this document (example the XML-1.0 + recommendation is a bit more of 150KBytes and takes 650KBytes of main + memory when parsed). Validation will add a amount of memory required for + maintaining the external Dtd state which should be linear with the + complexity of the content model defined by the Dtd</li> + <li>If you need to work with fixed memory requirements or don't need the + full DOM tree then using the <a href="xmlreader.html">xmlReader + interface</a> is probably the best way to proceed, it still allows to + validate or operate on subset of the tree if needed.</li> + <li>If you don't care about the advanced features of libxml2 like + validation, DOM, XPath or XPointer, don't use entities, need to work with + fixed memory requirements, and try to get the fastest parsing possible + then the SAX interface should be used, but it has known restrictions.</li> </ul><p></p><p><a href="bugs.html">Daniel Veillard</a></p></td></tr></table></td></tr></table></td></tr></table></td></tr></table></td></tr></table></body></html> |