Version 4.4p1 (Feb 22, 2012)

Maintenance release.

Minor enhancements:

Accept collations like "fr" without leading slash. A leading slash was required so far.
Beware however that if a base-URI is defined through options or API, a relative collation URI can be resolved against this base URI.
Accept Document object as argument of xlib:document(). This returns the document itself, for compatibility.
Two issues resulting into excessive memory use have been removed.

Bug fixes:

xlib:document() was returning an empty sequence in some cases.
The REST API client method RestAPIConnection.putDocument could garble non-Ascii XML characters, depending on the platform's default encoding. The underlying Apache http client was missing an explicit encoding (UTF8) for the transport.
spurious message "page should have been saved", signalling a bug in the cache of Collections. This can happen when specific conditions are met: 1) a super-collection contains at least 200,000 documents (at any level) , 2) a document is added or removed within this super-collection, 3) several thousands other collections are then added.

Version 4.4 (Dec 23, 2011)

Functional enhancements:

Incremental backup: works in the same way as full backup, but only copies differences.
Interfaces in: Java API, XQuery extension functions, Server (manual and scheduled), user interface in QizxAdmin.
Qizx Studio/Admin: navigation in large collections through Next/Previous items (displayed in blue).

Optimizations:

Modifying a very large number of collections (in tens of thousands or more), by adding or replacing documents, could use a lot of memory. This has been improved, but some progress can still be made.

Bug fixes:

Concurrent updating: "document NNN not found" errors were appearing when several threads modified the same document without locking (which is BTW not a good practice).
Another internal issue fixed at the same time.
Query could return no results when executed simultaneously by a large number of threads (threading issue in element name lookup).
Check on a database check created by several threads: some collections were reported as not found, some queries could fail due to an index segment ordering issue.

Version 4.3 (Oct 31, 2011)

Enhancements:

Administration facilities for Qizx Server and for embedded XML databases:
- Configuration display and editing,
- Display of runtime statistics,
- Maintenance tasks (backup, optimization),
- Management of running queries (display and cancelling).
These facilities are implemented at several levels:
- Native Java API
- Server REST API
- XQuery extension functions:
  functions prefixed by server: are callable through the REST API and can handle several XML Libraries.
  Functions prefixed by admin: are callable in any context and handle the current Library.
Administrator tools:
- QizxAdmin, a variant of QizxStudio providing a GUI for administrations task, Admin facilities have been removed from QizxStudio, which is now meant for loading databases and developing queries.
- A web application using
Qizx Server: Web Applications (aka XQuery Server Pages):
A mechanism for writing web applications in XQuery:
Basically a page template is a stored XQuery generating (X)HTML elements. HTTP Request parameters are bound automatically to XQuery variables (with possible type conversion), then the result of evaluation is directly serialized as the HTTP response.
This mechanism comes with a library of XQuery extension functions to handle http parameters, headers, sessions, cookies etc
Qizx Server:
- caching of Result Sequence in eval request. This allows iterating efficiently on a long sequence of results, without having to reevaluate an XQuery expression on each request.
  Cache size can be defined in Server configuration.
- support of several counting modes in eval request: exact (default), estimated and none.
Miscellaneous:
- Rename of Collection and Document now accepts an (existing) Collection as destination. The short name of the Library member is kept. For example if 'coll2' is a collection then xlib:rename-member('/coll1/doc.xml', '/coll2') would end up with a document '/coll2/doc.xml'.
- xlib:rename-member() and xlib:copy-member() now accept Document and Collection object types as first argument, like in xlib:delete-member().

Optimizations:

The query compiler now recognizes a "range predicate" in the form [@x >= 1 and @x <= 10] and optimizes it as if the x:in-range function was used (i.e [x:in-range(@x, 1, 10)] ).
Notice that this optimization is not possible in general on child elements, because for example the meaning of [date >= 1999 and date <= 2011] and [ x:in-range(date, 1999, 2011) ] can be different if there are several child elements date. Therefore it is recommended to use x:in-range() when appropriate.
The predicate [ x:in-range(date, 1999, 2011) ] is equivalent to: [some $adate in date satisfies $adate >= 1999 and $adate <= 2011], but the latter form is not yet recognized by the optimizer.
The query compiler now optimizes an expression such as collection(...)[ predicate ] : a predicate (e.g full-text) directly on a collection, returning document nodes.

Bug fixes:

Transaction concurrency: it was possible to get a duplicate document if it was modified by 2 different transactions without locking. The fix checks that situation and silently deletes the older version.
Notice that this fix still does not provide a full support of "Optimistic Concurrency", because it does not guarantee Consistency: locks must be used for that purpose.
There was a risk of Indexes damaged on shutdown, if compaction still running. A running compaction is now aborted on shutdown after a short, configurable delay.
Reindexing Progress indication was broken.
XQuery User Function call stack was broken.
DTD lost in XQuery Update operations. Minor regression due to the introduction of configuration property "doc_node_stats".
fn:put() was not considered updating.
random IllegalStateException when shutting down Qizx Server running inside JBoss.
Path predicates like //foo[ *[2]/bar ] could be compiled in an invalid way.
Some path predicates using self:: , like //*[not(self::foo)] could produce an NPE.
A query on a document just imported in an XML Library, without commit, could fail because the query compiler was attempting to use indexes while in fact the document is not yet indexed.
Reminder: in any case, it is better to do queries after a commit, because otherwise indexes for newly loaded documents are not yet available, hence slower query execution.

Version 4.2 (June 6, 2011)

Enhancements:

Move to Java 6:

A JVM version 6+ is now required to run Qizx.

The API is mainly unchanged, except a few additions listed hereafter.

Pluggable Content Importers

A Content Importer is a parser for 'semi-structured' content format, converting it to XML.

Qizx 4.2 contains importers for JSON, HTML through TagSoup (jar included), HTML5 (available as a separate package).

Content can be imported as XML with the extension function x:content-parse(data, format, options).

Other importers are planned in future versions: Mime mail, CSV

Support of JSON

JSON data is converted to XML (proprietary schema) through a particular Content Importer, hence parsing is achieved with x:content-parse(data, "json").

Serializing JSON-as-XML back to JSON format is achieved through function x:serialize-json().

Support of EXPath:

From 4.2, Qizx supports extensions specified by the EXPath initiative.

Documentation for these modules can be found in the Developer Guide, as well as on the EXPath site.

"Packaging System": EXPath Packaging provides a framework for describing library or application packages, independent of the XQuery platform.
Packages can contain not only XQuery modules, but also XSLT2 and XProc modules. Modules are referenced only through their public URIs.
EXPath includes tools for creating packages.
In addition, the Qizx implementation of EXPath Packaging System supports Qizx-specific modules, such as Content Importers (see above) and the "zip" package (see below).
"zip": a set of EXPath extension functions for handling zip files (available as a separate package).

We will keep expanding this support with packages such as:

http-client a set of extension functions for communicating with HTTP servers,
webapp, a mechanism for executing server-side applications written in XQuery, XSLT2, XProc. This will be the implementation of Web Apps in Qizx Server.

Configuration

The Java API now supports extended configuration methods on LibraryManager and XQuerySessionManager, by which a number of properties can be specified, in particular runtime parameters and implementations of add-ons.

A new class Configuration replaces LibraryManagerFactory as bootstrap mechanism.

This facility is supported in tools (qizx and QizxStudio) through option -config , and in Qizx Server.

Optimizations:

Qizx 4.2 includes significant optimizations for large databases of small documents. These improvements were required by a customer use-case involving databases of more than 100 millions documents (2Kb each), with demanding performance goals in bulk-load, queries and updates.

Removed a size-scalability issue (algorithm complexity) in Index segment compaction (visible after a large number of transactions).
Much improved speed of updates on very large databases (100 Millions docs+).
Reduced memory use in XQuery: operations like group by and order by now require much less memory (at least 10 times less). The problem was appearing when trying to group or sort nodes belonging to tens of thousands of documents in a database. It is now feasible to sort from 1 million documents, while still requiring a sizable amount of memory (about 600 Mb).
XML parsers and Catalog resolvers are now reused in bulk-load.

Note that fine-tuning the configuration (see above) can help optimize bulk-load: please ask us for advice.

Bug fixes:

Some documents could become unreadable after bulk loading several millions of small (less than 2K) documents.
Bug in Document store manifested as occasional "OVERLAP" messages. This could lead to damaged databases.
Overflow issue: maximal number of documents was in fact limited to around 60 millions due to a design bug, while the limit is 1 billion in principle.
The actual limit is now 250 millions. Future Qizx versions are expected to remove any practical limit.
Index: overflow issue caused by very large posting counts (over 32 bits).
Index: minor issue on index keys with 0 postings (caused by document deletions).
Logging: timezone was ignored in daily rolling log files.
Logging: on shutdown in J2EE environment (e.g JBoss), logs were truncated prematurely by the Shutdown Hook used in Java standard logging (java.util.logging).
Java API: Library.setAutoCommit(false) was not effective.

Licenses and Prices (March 1, 2011)

These conditions apply to any product sold or distributed starting from March 1, 2011.

Terms of licenses have been rewritten for more accuracy.
See present licenses for applicable terms.
Price-list and renewal conditions have been changed.
Our policy is to adapt prices in accordance with the development of functionalities. Please note that prices had not been changed since the introduction of Qizx in 2008.

Version 4.1p1 (December 6, 2010)

Maintenance release.

Optimizations & Enhancements:

The let $var := is now fully lazy (unless an explicit type checking is specified). This is in general more efficient, but can occasionally lead to multiple computations of the same expression. This case will be optimized further in a later version.
(XQuery 1.1) Missing functions fn:head(), fn:tail(), fn:function-name(), fn:function-arity() have been added.
Qizx Server: documented troubleshooting with security on some Servlet servers (namely Tomcat on Ubuntu).

Bug fixes:

occasional database corruptions happening on massive XQuery Update jobs (EFL). Cause: bug in an internal cache implementation.
XQuery 1.1: higher-order functions implementation revised. Several issues fixed after trying the XQuery 1.1 test suite.
Database query compilation: Collection objects were not accepted as roots of Path Expressions (unlike Document objects).
Database query compilation: remaining issue in attribute in predicate e.g //item[relation/@category = "category123"]
Qizx Server: "too many open files" occasionally happening on large bulk uploads.
Database bulk store: in rare cases, the occurrence count per document -stored in indexes- could be 0, leading to slightly incorrect values returned by count(), for example count(//SomeElement) on the whole database. This would happen only if documents were indexed by batches of 4096 or more.
If it is the case for your databases, it is necessary to re-index completely to get correct counts.
A few failing conformance tests in XQTS 1.0.3 have been fixed, but there are still a number to correct. Most of it is of petty importance.

Version 4.1 (October 11, 2010)

Enhancements:

XQuery 1.1. This version adds the following XQuery 1.1 features:
- "inline functions" (aka lambda functions) and high-order functions as specified in XQuery 1.1 Draft Specifications.
- Standard try/catch.
- Switch instruction (simple, unoptimized implementation)
- Trigonometric functions, plus extension pow(x, y).
Implemented XQuery 1.1 Features: functions as items, group by, for window, try catch, switch, output declarations.
Yet to implement: count clause in FLWOR, computed namespace constructors, formatting and miscellaneous functions.
Support of non-XML documents: Qizx can now store documents containing any binary data. They are similar to "Blobs" of relational databases, and can be used for example to store images or plain text. Access is sequential, compression can be used as an option.
Scalability and efficiency: Qizx has been validated on very large databases and documents. Improvements have been made to reduce memory consumption and bottlenecks.
A new "Extra Large" (XL) edition of Qizx is available through scripts qizx-xl and qizxstudio-xl (using qizxxl.jar). It can handle very large documents (up to 1 Terabyte, instead of 2 Gigabytes for the standard version).
[See details in this page]
- Qizx has been validated with a 100 million-documents database, representing about 330 Gigabytes of FpML data.
- Qizx Full-text capabilities have been tested on the English Wikipedia, containing some 9 millions pages and 38 Gb of XML. Realistic full-text queries with scoring were measured under 1 second (on a i7 860 processor).
- A 44 Gigabyte document has been stored and indexed, using the specific "XL" edition (jar qizx64.jar). This edition has a theoretical limit of 1 Terabyte approximately for a single document, instead of 2 Gb in the standard version).
- Improved recovery and cleanup after crashes (notably OutOfMemory in QizxStudio or qizx with local databases).
Server installation wizard: Qizx Server bundled with a Servlet container is installable very easily using a wizard.
The manual installation has been made simpler; the (buggy) former script mkserver has been discarded.
Minor improvements:
- Logging of XML Library operations has been improved and exposed in the Java API. Applications can now tune loggers and add outputs.
- The Message window in QizxStudio has been restored and displays warnings and errors logged by the XML Library engine.

Bug fixes:

Cancel method on XQuery Expression did not always stop execution.
Query compilation issue on attribute in predicate
Group-by producing long sequences (over 65536 items), first items were lost.
Compilation error in some numeric predicates: eg (1 to 5)[let $x := position() return $x] said " variable x not declared".
QizxStudio did not close local databases on exit, which could cause minor corruptions after updates.
Wrapper scripts qizxstudio and qizx did not work on Mac OS X.
Support of parameters passed in a File Part (POST multipart) in XQuery Services.
Missing or improper conversions in Java API (ItemFactory.createItem). Added conversions for xs:hexBinary and xs:base64Binary (from String and byte[]), and fixed xs:time.

Version 4.0 (June 8, 2010)

Enhancements:

Qizx Server:
Qizx Server makes the embeddable XML database engine accessible to remote clients through a set of interfaces:
- API service exposes the main capabilities of the Qizx engine to clients implemented on any platform that supports HTTP client requests: front-end servers in PHP, JSP. .NET languages etc, heavy clients running on any platform.
  The architecture implied by this service is similar to many applications using a SQL back-end server.
- XQuery Services is a mechanism for calling XQuery scripts stored on the server, passing parameters.
  It can be used to implement lightweight Web Services-like applications, or to implement business logic middle tiers.
- Other services are in development, namely for directly implementing web applications in XQuery.
- All services are based on a HTTP REST-style protocol.
  They are extensible and their source code is available.
- Qizx Server features an access control mechanism based on ACL.
- See documentation for more details.
Tools
- Qizx Studio and the command-line tool qizx have been enhanced to work with the API service, thus providing remote administration tools.
- Qizx Studio editor has been improved (undo, redo, save, save as). The query history is suppressed.
- In Qizx/open, both tools are able to connect to a server (browsing and remote execution).
XQuery language and extensions:
- Preliminary support of XQuery Scripting. No support yet for validity checks and Update semantics.
Java API enhancements and changes:
- Interface LibraryManager has been enhanced. A LibraryManager is no more tied to a 'Library Group' on disk. The group descriptor files group.qlg are deprecated. See the documentation for details.
- Slightly incompatible changes in LibraryManager: createLibrary() and openLibrary(); most users should be unaffected.
- LibraryMember.setProperty now accepts values of type Item. Supported Item types are xs:string, xs:boolean, xs:integer, xs:double, node(), xs:dateTime. Other types are cast to these basic types.
Moved to Java 5. A JVM version 5+ is now required to run Qizx (Java 6 recommended because of XML parser issues and better Unicode support).
The API is unchanged, except minor enhancements listed here.

Bug fixes:

Operators except, intersect, union on XML Libraries could return incorrect results.
Resolution of XQuery module URIs.
ft:highlight spurious empty elements.
Overload of user functions now works (upgrade to current XQuery specifications).

Version 3.1 (Feb 1, 2010)

Enhancements

See version 3.1beta for main changes.
Upgraded to latest XQuery Update Candidate Recommendation (09 June 2009). Qizx 3.1 passes 100 % of tests of XQUpdate Test Suite 1.0.0.
Automatic DOCTYPE generation when serializing a document. Can be disabled by new option auto-dtd=false.

Bug fixes

Serialization: improper generation of default namespace from attributes.
Query compilation issue on: $node/*[. ftcontains "..."]

Version 3.1 beta (Nov 9, 2009)

Enhancements

Added simple full-text query syntax and several full-text convenience functions for scoring, highlighting, extracting document snippets, looking up indexes: see chapter full-text extension functions in documentation for more details.
Improved full-text scoring speed. Document ranking (or weighting) through a meta-property ft-weight is now deprecated because too slow (this will be replaced by a more efficient mechanism in a later version).
Added a chapter in the manual introducing the main utility classes offered by the Java API.

Bug fixes

Miscellaneous full-text bugs detected by passing the XQFT Test Suite (results to be published by W3C).
Opening an XML Library Group twice was throwing an unclear exception on Windows. This exception now explicitly tells that only one instance of LibraryManager can exist for a Library Group at a time.
XQuery Update: updates performed within an (updating) function were often lost.
Calling Library.createCollection(path), where path has no leading slash, provoked an infinite loop.
Several inefficiencies and memory leaks (yet unreported).

Version 3.0 (May 20, 2009)

Bug fixes

A noxious bug that appeared under very heavy load, with many threads performing transactions concurrently, and that could corrupt an XML Library (reported by Lóránt Pintér).
This new version has been tested with over 200 threads.
Introduced a limit to the number of index segments created when a large number of threads perform transactions simultaneously. The multiplication of index segments increases memory consumption and could provoke OutOfMemoryException's.
Also fixed 2 memory leaks contributing to these OOME.

Enhancements

Improved crash recovery and database sanity checks.

Version 3.0 beta (March 9, 2009)

Enhancements

Support of the standard XQuery Full-Text functionalities. This new W3C Recommendation provides a comprehensive specification for full-text search, well integrated with the XQuery language.
The new full-text functionalities in Qizx are a nearly complete implementation of the standard, with an execution speed slightly improved with respect to the former implementation. Documentation can be found in the User's Manual section of the documentation. A new tutorial is available here if you wish to get acquainted with this language extension.
Attention: the former support of full-text in Qizx 2.x, based on extension functions, has been completely withdrawn. To help migrating your applications, we have written a dedicated chapter in the documentation: "Migration Guide from former Full-Text implementation". Version 3.1 might restore the former full-text functions as a layer on top of new full-text, but only if we feel a demand from our users.
Due to the new full-text, Indexing Specifications have slightly changed: "word sieves" are deprecated. The Java API contains a new package com.qizx.api.fulltext that provides means of customizing full-text features (tokenization, stemming, thesaurus, scoring).
Miscellaneous optimizations. As a result, Qizx 3.0 is about 40% faster on the XMark benchmark. More details can be found in the chapter "Writing efficient queries" of the Developer's Manual.

Bug fixes

A speed regression was introduced in 2.2 in function count() when access-control is not used.
Data conversions in Indexing Specifications were not used when the query is not indexable, leading to inconsistent execution errors.
Note: this data conversion mechanism is actually an extension of XQuery and goes beyond XML Schema. It is not yet available in Qizx/open.
For example assuming that Indexing Specifications are able to recognize and convert dates written like "12/31/99" in element and attributes values, a non-indexable query like //*[date = xs:date("1999-12-31")] did generate an error, whereas the indexable query //event[date = xs:date("1999-12-31")] was working as expected.
Issue when loading a module in the current directory when no location is provided.

Version 2.2 (November 24, 2008)

New Licenses and new pricing

Qizx 2.2 has a new licensing scheme and a new price list.

Enhancements

Support of XQuery 1.1 features: the very useful and long awaited group by in FLWOR, and the for window clause which allows "chunking" a sequence based on conditions.
The documentation "Getting Started" now includes a section about XQuery Update.

Bug fixes

XQuery join optimization: a join using the operator "<" could miss a result item due to improper bound test.
XQuery: built-in functions handling the type xs:anyUri (for example fn:resolve-uri) had type inconsistencies that could create surprising run-time type errors.
XQuery Update: Updating Expressions within most operators (except FLOWR, if, typeswitch, comma) are illegal but errors were not raised. As a consequence, some (invalid) expressions silently failed to execute. For example an Updating Expression within a node constructor, which is not allowed by the language, could not be executed.
Java API: inconsistencies in ItemSequence counts and positions when Access Control is used.
countItems() was counting nodes barred by Access Control, skip() was not consistent with moveToNextItem(). Now all methods take AccessControl into account, possibly with some speed penalty.
Java API: using a dangling Node of an actually deleted document was generating obscure exceptions. A DataModelException with message "deleted document" should now be raised. Notice it might happen that in-cache Nodes still be accessible even though their document is deleted.
XML Library engine: an error "bad block mark" was sometimes appearing when performing commit operations at high speed (for example: doing a massive number of document imports each followed immediately by a commit, which is definitely not a recommended practice).
XML Library engine: Library.deleteMember had caching issues before a commit was done.
for example deleteMember("/a") followed by renameMember("/b", "/a") generated an error claiming that "/a" was still existing.
XML Library engine: Library.renameMember had caching issues before a commit was done. If a document was first accessed, then renamed (or if an enclosing collection was renamed), then the getMember or getDocument methods could return stale handles still having the old path. Similar issue for collections.
Serialization bug in Indexing specifications, built programmatically and using names with namespaces: could prevent re-opening an XML Library.

Version 2.1 (May 21, 2008)

XQuery Update: Qizx 2.1 fully supports the XQuery Update Facility draft specifications.
This implementation passes 100% of the current XQUF Test Suite.
Qizx Studio improvements:
- New editing dialog allows easier definition of most frequent customizations of Indexing Specifications.
- Progress bars for database operations like re-indexing and index compaction. These operations are now synchronous (see below).
- Miscellaneous: more graceful namespace display in XML views, new icons, directories and paths remembered, double-click in tree views performs a full expand/collapse.
API extensions:
- Methods optimize and reIndex of Library become synchronous (or blocking). The LibraryProgressObserver interface is extended to include these operations (slight incompatibility).
- New utility class PushNodeBuilder to build native in-memory Nodes, as an alternative to using DOM or XML parsing for binding Node values to XQuery global variables.
- DOM types are now recognized in Java Binding (arguments and return value of methods) and converted (by copy) to internal Qizx representation.
- The XQuerySession interface has a new method enableJavaBinding for making Java classes eligible for use with the Java Binding mechanism. Attention: Java Binding is no more enabled by default when using the API. It is possible, though not recommended, to allow all Java classes at once.
Miscellaneous:
- Some names of built-in properties of Documents have been changed. All default built-in properties are now documented.

Version 2.0 (January 14, 2008)

First release.