nux.xom.sandbox
Class XQueryBenchmark

java.lang.Object
  nux.xom.sandbox.XQueryBenchmark

public final class XQueryBenchmark
extends Object

Simple benchmark measuring XQuery and/or XPath performance of a given set of queries.

For XPath, the default queries and test data are taken from Ryan Cox's and Shirasu Hiroyuki's XPath comparison. (The results are drastically different than the one's they reported.) The queries can be used to measure rough overall performance of basic XPath expression building blocks.

For XQuery, the default queries and test data are taken from the XMark XQuery benchmark. XMark consists of 20 carefully chosen queries, each stressing key performance aspects of exact match, ordered access, regular XPath expressions, following references, construction of complex results, join on values, search for missing elements, and so on. See their tech report for details of the underlying rationale.

The default queries and test data are included in the download, along with other samples.

Disclaimer: Note that any given queries may or may not reflect your specific application usage profile. As always, your mileage may vary, and your applications may exercise significantly different query operations. Hence, make sure to cross-check with your own application benchmarks.

Example usage:

 export CLASSPATH=lib/nux.jar:lib/saxon8.jar:lib/xom.jar
 # XPATH benchmark
 java -server nux.xom.sandbox.XQueryBenchmark 1000 3 cache samples/data/romeo.xml samples/xpath/queries1.xml 
 java -server nux.xom.sandbox.XQueryBenchmark 1000 3 nocache samples/data/romeo.xml samples/xpath/queries1.xml 
 java -server nux.xom.sandbox.XQueryBenchmark 1000 3 xom samples/data/romeo.xml samples/xpath/queries1.xml 
 
 # XQUERY benchmark
 java -server nux.xom.sandbox.XQueryBenchmark 1000 3 cache samples/xmark/auction-0.01.xml samples/xmark/*.xq 
 

The first three examples run each XPath query found in the file queries1.xml 1000 times against the romeo.xml data file (230KB), repeating all of it for 3 repetition blocks. Results are given in milliseconds and, perhaps more interestingly, queries/sec.

A parameter says that the benchmark should be separately run for Nux with the precompiled query cache enabled ("cache") or disabled ("nocache"), and then with the Jaxen based XPath implementation of xom-1.1 CVS ("xom").

The last example runs all XQueries (*.xq files) in the xmark/ directory 1000 times against the auction-0.01.xml file (1 MB), repeating all of it for 3 repetition blocks.

Note that the time of the first 2 repetition blocks should not be considered for timing comparisons, because the JVM hotspot compiler introduces strong perturbations on warmup. Ignore those and scroll down to the last repetition block.

Here are example Nux XPath outputs for a 230 KB and 230 MB romeo.xml file with JDK 1.5 server VM on a dual Pentium4 @ 2.8Ghz, 2 GB, Redhat 9, a typical commodity cluster node configuration. Note that the 230 MB file requires some 1.0 GB for the XOM main memory tree. Overall, both scenarios demonstrate excellent performance, with execution time growing at most linearly with the problem size even under harsh conditions.

Here are example Nux XQuery XMark outputs for a 1 MB and 10 MB auction.xml file with JDK 1.5 server VM on a dual Pentium4 @ 2.8Ghz, 2 GB, Redhat 9, a typical commodity cluster node configuration. Again, both scenarios demonstrate excellent overall performance, with the caveat that JOIN performance inside saxonb-8.6 currently does not (yet) scale well with data size (q08.xq - q12.xq).

Author:

whoschek.AT.lbl.DOT.gov, $Author: hoschek3 $

Field Summary

static int	dummy
static boolean	IS_BENCHMARK

Method Summary

static void	generateTestData(String[] args) Reads an XML file and multiplies its size by concatenating it N times.
static void	main(String[] args) Runs the benchmark

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

IS_BENCHMARK

public static boolean IS_BENCHMARK

dummy

public static int dummy

Method Detail

generateTestData

public static void generateTestData(String[] args)
                             throws Exception

Reads an XML file and multiplies its size by concatenating it N times. This can be used to produce benchmark data files of a given desired size. For example, this will produce an output file that is 100 times larger than the input file, containing the same data 100 times:

 java nux.xom.tests.XQueryBenchmark data/romeo.xml 100 data/romeo100.xml
 

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Runs the benchmark

Throws:

Exception