/xmlstarlet-patch-huge

Patch xmlstarlet 1.6.1 to use 'huge' nodes and 'big lines'

GNU General Public License v3.0GPL-3.0

Patch xmlstarlet 1.6.1 to use 'huge' nodes and 'big lines'

This patch adds 2 global options to xmlstarlet, available in the subcommands elements, select, edit, format, canonic, validate, and transform:

  • --huge
    Load XML files with libxml2's XML_PARSE_HUGE parser option. Without this option the parser will fail with a xmlSAX2Characters: huge text node: out of memory error when loading a text node larger than 10 MB. (xmlstarlet's pyx subcommand uses the SAX API which has no such limitation.) In libxml2 the XML_PARSE_HUGE option is disabled by default to prevent denial-of-service attacks.

  • --big-lines
    Load XML files with libxml2's XML_PARSE_BIG_LINES parser option. This allows line numbers larger than 65535 to be reported correctly in error messages and (for select and transform) in output from the saxon:line-number extension. There is currently an open issue on this option suggesting it's limited to text nodes, however, it appears to have been resolved by now as line numbers are output as expected for all node types except the root node (/) which is fixed at line -1.

The patch requires xmlstarlet to be rebuilt from source code:

  1. Install the packages libxml2-dev (most recent version for the updated XML_PARSE_BIG_LINES) and libxslt1-dev (e.g. sudo apt-get install libxml2-dev libxslt1-dev).

  2. cd to an empty directory, download (sf.net or fossies.org) and unpack (e.g. tar xaf xmlstarlet-1.6.1.tar.gz) the xmlstarlet source code.

  3. Download xmlstarlet-huge.diff and apply the patch by running patch -u -p1 -d xmlstarlet-1.6.1 < xmlstarlet-huge.diff to modify source files.

  4. Follow the instructions in xmlstarlet-1.6.1/INSTALL to build, test, and install xmlstarlet.

    On a recent Debian system it was done with the following commands -- resulting in 3 compiler warnings (maybe-uninitialized, incompatible-pointer-types, unused-result) and 2 tests (bigxml-dtd, ed-namespace) failing as expected. Note that this builds a dynamically linked executable but ./configure --enable-static-libs ... is there if you can brave the .as.

$ cd xmlstarlet-1.6.1
$ ./configure CFLAGS="-O3 $(xml2-config --cflags)" LIBS="$(xml2-config --libs)"
…
$ make
…
$ make check
…
$ sudo make install-strip
…
$ sudo mv /usr/local/bin/xml /usr/local/bin/xmlstarlet

Global options are listed in the general usage reminder (xmlstarlet --help).