Benchmarking Qizx
Since one of Qizx's distinctive characteristics is its querying speed, we give concrete execution measurements on real-size examples.
- Qizx is compared with two open-source products on the XMark benchmark.
- An example of multi-criteria search is given on a large database of small documents.
Please remember that like for any benchmark, figures given here are highly dependent on test conditions (details below), hence subject to variations from one platform to another, and are given for information purpose only.
Test platform
All following tests have been performed on the same commodity PC:
XMark
XMark is a XML database benchmark designed by the CWI Amsterdam and other organizations. It consist of 20 queries on a single document (whose reference size is 112 Megabytes, but can be scaled at will).
Note: another comparison of several products on XMark is available on the site of Monet DB, but it does not include Qizx and is performed on a different platform.
XMark favors massive scans and joins, and puts little emphasis on value-based queries (unlike example in the section above).
The following test compares Qizx with:
- eXist, a widely used open-source Java native XML database.
- Qizx/open, which is not a database engine: it parses documents in memory and works without indexes.
- Monet DB XQuery 5.22, the XML Query database also developed by CWI. Monet DB is written in C and has been in the past years deemed the fastest XQuery database engine available. [Timings not yet available]
- Berkeley DB XML 2.5.16, an XML Query database now owned by Oracle. [Timings not yet available]
These 3 first products share the same characteristic that queries can be executed "out of the box" without need to define indexes manually.
For legal reasons, we cannot not include tests performed on commercial products (some vendors explicitly prohibit it), but that does not make a real difference.
Test conditions:
Comparison of Qizx 4.1 with other open-source databases on XMark 112 Mb| Queries | Qizx/open 4.1 | Qizx4.1 | eXist 1.4 | MonetDB | Berkeley DB XML 2.5.16 |
|---|
Q01 | 8 | 0.5 | 85 | | |
Q02 | 14 | 153 | 295 | | |
Q03 | 37 | 79 | 850 | | |
Q04 | 36 | 88 | AL | | |
Q05 | 12 | 91 | 88 | | |
Q06 | 155 | 4 | 11 | | |
Q07 | 288 | 17 | 48 | | |
Q08 | 54 | 145 | DNF | | |
Q09 | 95 | 256 | DNF | | |
Q10 | 474 | 690 | DNF | | |
Q11 | 327 | 440 | DNF | | |
Q12 | 285 | 367 | DNF | | |
Q13 | 10 | 54 | 63 | | |
Q14 | 190 | 170 | 320 | | |
Q15 | 12 | 55 | 58 | | |
Q16 | 17 | 108 | 320 | | |
Q17 | 25 | 77 | 275 | | |
Q18 | 21 | 166 | 145 | | |
Q19 | 109 | 395 | 615 | | |
Q20 | 58 | 92 | 390 | | |
| Geometric average [2] | 52 | 84 | > 1000 | | |
| Import time (seconds) | 8 | 49 (without full-text) | 150 (with full-text) | | |
Remarks:
- The geometric average is more meaningful when aggregating short queries and long queries.