Chapter 5. Installing and Using Qizx Server

Chapter 5. Installing and Using Qizx Server
Prev	Part II. User's Guide	Next

1. Architecture

Qizx Server is a modular system that provides several interfaces (called services) which can be used in different types of applications:

Figure 5.1. Qizx Server Architecture

Qizx API service
This service offers nearly the same services as the embeddable Qizx engine, but can be accessed by remote clients. It is both an application and administration interface:
- A client can be a web application running in a different server, using classical environments such as PHP, ASP, JSP etc.
  It relates to Qizx Server by sending XQuery scripts and receiving XML fragments that can be included in their own responses. This is similar to the way many simple web applications use a Relational DBMS by sending SQL queries.
- A client can be a heavy client implemented for example in Java or on top of the .NET platform.
- As a particular case, the command-line tool qizx and the graphic interface QizxStudio coming with Qizx are able to use this API service. They can be used for administration tasks.
XQuery services
This interface provides a functionality similar to Web Services, but much simpler:
- Clients call named services, passing parameters, and retrieving results (generally as XML).
- Services are implemented directly as XQuery scripts, stored on the server.
- Services are self-described: a list of possible calls and a description of each service call can be requested.
  This allows generic binding on the client side, like in classical web services.
- Such services can also seamlessly respond to XForms submissions.
◾ Alternately, this service can be regarded as a way of implementing middle-tier business logic on top of a XML database.
Whatever the use, this approach offers the means of encapsulating the core logic of an application, by publishing an API for the application and hiding the internals of the database. This seems a great advantage in terms of elegance and maintainability over the other solution, using the API service and passing scripts.
XQuery Server Pages
Not available in 4.0, planned in a future version of Qizx.
This service allows implementing Web applications using XQuery as a dynamic page template language.
XQuery Server Pages is a service similar to many web application environments such as JSP, ASP, PHP etc. It will also support XForms.
Delayed indexing service
Not available in 4.0, planned in a future version of Qizx.
This service can be used when a feed of information provides a continuous flow of XML documents, and it is not critical that incoming XML data be immediately visible to applications. Example: logging of transactions, mails etc.
The service indexes incoming documents at regular intervals (for example one minute) and ensures data safety through journaling.
The purpose of this service is to help developments by a greater simplicity, safety and efficacy of such an operation.

1.1. Protocol

All these services are based on HTTP, with simple REST-style interfaces using only GET and POST.

There are good reasons for not using a proprietary protocol:

Applications can be implemented in any language or platform that supports client HTTP requests: Java, .NET, PHP etc.
Using only GET and POST is required because many HTTP client libraries have limited HTTP support.
In many companies, security constraints (firewalls and proxies) make it difficult to use anything but HTTP and GET/POST.
HTTP protocols are well known, simple to understand and can be tested through web browsers.

1.2. Server-side Implementation

All these services are implemented as Java Servlets. Thus a server can be hosted by any web application server that supports servlets.

Services can coexist inside a Servlet Container or a Web Application. Any combination of services is possible through simple configuration.

1.3. Client-side Implementation

It is important to note that no client-side libraries are provided in the first version (4.0) of Qizx Server.

The main reason for that is the large variety of potential clients: it is simply no possible to provide a client layer for all existing environments. We also do not want to impose particular third-party software implementing such client layers.

On the other hand, the protocols used in Qizx Server are simple enough to be easily implemented with the generic client libraries available in many platforms like PHP, .NET.

A client library will be provided in a next version for at least:

The Java platform
The .NET platform
Possibly also PHP

Automatic binding to XQuery Services on these platforms is also a planned feature.

2. Installation

In the current version, Qizx Server is a Web Application hosted by any J2EE Servlet Container. The installation is manual but uses standard Java Servlet techniques. In future versions we plan to bundle Qizx Server with its own servlet container and provide an installation wizard.

By following the following procedure step by step, a member of your IT staff should be able to easily deploy Qizx Server on a servlet container. This procedure involves:

Creating the storage directory of the server: this is kept separate from the Servlet container to avoid accidentally destroying precious data.
This directory contains the configuration, the XML database(s) (called Library) and optionally XQuery modules and stored XQuery scripts.
Completing and deploying the Qizx Server WebApp on your Servlet Container,
Specifying how user authentication is to be performed, and making sure at least one user has administrator privilege.

2.1. Requirements

Java runtime environment version 5+ (version 6 recommended).
A Servlet Container that supports at least Servlets version 2.4, for example Apache Tomcat 5.5.
Note: so far the server has been tested only with Tomcat 5.5 and 6, and Caucho Resin 3.1.

2.2. Installation Procedure

This procedure is achieved through commands in a terminal:

Choose the server storage directory: decide the place where you want your server data to reside. This directory, referred to as Qizx_Server_Storage all over this documentation, contains configuration files and data files.
Make sure you have enough free disk space (e.g. several gigabytes) to handle the amount of data that will be stored there. A Qizx Library with all indexes is roughly the size of the source XML it contains, but indexing or reindexing uses temporary files that can reach up to 2 or 3 times this size.
This directory will actually be created and filled by a script, as explained below.
Choose the name of the Web App in which the server will run.
The name qizx is used in this documentation, but this can be changed at will. Attention: examples coming with Qizx Server will not work if this name is not qizx.
This name is important for determining the address (URL) that applications will use to connect to the server.
For example, if you choose qizx, and if your host name is myhost and the port 8080, the URL of the Qizx server will be http://myhost:8080/qizx/api .
Technical note: in this address, qizx is the name of the Web App, and api is a mapping of the API service of Qizx Server. This mapping can also be changed by editing the web.xml configuration file of the Web App. We will get back at this later.
Prepare to create the Web App:
1. Stop the Servlet Container.
  This is not mandatory but recommended, as the new Web App will perhaps require some adjustments before running.
2. open a terminal and change current directory to the directory in your Servlet Container installation that contains web applications, generally called webapps.
  For example if you have Tomcat installed in /opt/tomcat:
```
> cd /opt/tomcat/webapps
```
  Or if you have Caucho Resin installed in c:\works\resin4.0.6 :
```
C:\> cd works\resin4.0.6\webapps
```
Actual creation: this uses a script mkserver located in QIZX_ROOT/bin (where QIZX_ROOT is the directory where Qizx has been installed). If you are on a Unix machine, use mkserver, on a Windows machine, use mkserver.bat (even if using Cygwin).
This script requires 2 arguments: first the server storage directory designated by Qizx_Server_Storage, then the name of the webapp.
For example, if you have decided that the server storage directory is /srv/qizxserver and the web app name is qizx:
```
Unix> QIZX_ROOT/bin/mkserver  /srv/qizxserver  qizx
```
```
Windows>  QIZX_ROOT\bin\mkserver.bat  c:\srv\qizxserver  qizx
```
Attention: it is recommended to use an absolute path for Qizx_Server_Storage.
In the server storage directory Qizx_Server_Storage, you should now have the following contents:
- qizx-server.conf : contains the configuration of the server. It is self-documented and allows modifying parameters like maximum memory sizes, XML catalogs, administrator credentials.
  It is advisable to review this file. In particular it contains the name of administrator users or role. This is explained in section "user authentication" below.
  If you modify this configuration while the Qizx engine in running, you have to restart the engine by using the "-server reload" command in qizx tool (see its documentation).
- xlibraries: a directory that contains one or several XML databases (called Libraries). For the moment there is none.
- modules: a place where XQuery modules can be stored.
- xqs: a place where stored XQuery scripts can be stored, to implement XQuery Services.
In the web app directory (here named qizx), you should find the following contents:
- index.html : points to the documentation of the API service.
- apidemo: a directory containing HTML files that are both a documentation and a demonstration of the API requests. Once the server will be online, you can use this demonstration to discover and understand the API.
- WEB-INF/web.xml, the webapp configuration file. It contains the definition of servlets that implement the services, and the mapping of these servlets to URLs. The servlets have only one initialization parameter which is a pointer to Qizx_Server_Storage:
```
<servlet>
  <description>This servlet implements the REST-style API Service.</description>
  <servlet-name>qizx-api</servlet-name>
  <servlet-class>com.qizx.server.api.RESTAPIServlet</servlet-class>
  <init-param>
      <description>Location of the Qizx Server root.</description>
      <param-name>qizx-server-root</param-name>
      <param-value>/srv/qizx/db1</param-value>
  </init-param>
```
  We might have to modify web.xml for the sake of user authentication (see below).
User authentication: this step depends on your Servlet Container and on the desired type of authentication. We give examples for two containers, Tomcat and Caucho Resin.
1. Concepts:
  1. Qizx Server has a notion of administrator privilege: operations of administration type can only be achieved by privileged user. This can be granted on user names or through a role defined in the servlet container.
  2. A privileged user name can be specified explicitly in the configuration file qizx-server.conf, in the property admin_users. By default, the name 'qizx-admin' is defined there.
  3. A privileged role can be defined through property admin_role.
    Any user having this role has administrator privilege in Qizx Server. By default the value is 'manager'.
  4. If both admin_role and admin_users are empty or undefined, then there is no restriction on privileged operations: for security this is not recommended.
2. Examples using BASIC authentication:
  In this example, both qizx-admin and john will be able to perform privileged operations on Qizx Server:
  1. With Tomcat:
    Edit the file conf/tomcat-users.xml in your Tomcat installation and add:
    <tomcat-users> <role rolename="manager"/> <user username="qizx-admin" password="changeit!" roles="manager"/> <user username="john" password="changeit!" roles="manager"/> ...
  2. With Caucho Resin:
    Add a file resin-web.xml into directory WEB-INF of the web app, and edit it to define user qizx-admin and add role manager to user john:
    <resin:XmlAuthenticator password-digest="none"> <resin:user name="qizx-admin" password="changeit!" roles="manager"/> <resin:user name="john" password="changeit!" roles="manager"/> ...
  Note: these examples assume that you keep the user and role defined by default in Qizx_Server_Storage/qizx-server.conf. You can also change these properties and keep your own users already defined in your servlet container.
3. Other types of authentication depend much on the actual servlet container. This is left to advanced users.
  Please note that the default WEB-INF/web.xml in Qizx web app uses BASIC authentication in the <login-config> item.
The servlet container can now be restarted.
Before restarting:
- Check that the account running the servlet container has rights for reading and writing on the Qizx_Server_Storage.
- Check that firewalls, if any, are allowing HTTP connections on the desired port.
Test that Qizx Server is working by using a web browser and entering the address
http://myhost:8080/qizx
(assuming that the server host is myhost and the server is listening on port 8080, and you chose qizx as web app name).
You should see a page pointing the documentation/demonstrations of the server's APIs.
Before that, you will be asked a login and password by your browser: enter the name and password of an administrator user.
Figure 5.2. API demonstration and test pages
You can also use Qizx Studio or the command-line tool qizx for creating a first XML Library (database).
In that case, you have to use a slightly different address:
http://myhost:8080/qizx/api
For example:
```
qizxstudio -g http://myhost:8080/qizx/api
```
Or using the menu Tools | Connect to Server in Qizx Studio, enter this same address.
Note that the http: prefix is required for distinguishing a remote server from a local Library group.
You will likely be asked a login and password (unless you have deactivated user authentication in the web app configuration). You can also specify credentials on the command line (not very secure):
```
qizxstudio -g http://myhost:8080/qizx/api -login me:mypassword
```
or in a file (see documentation of tools):
```
qizxstudio -g http://myhost:8080/qizx/api -auth credentials
```

2.3. What to do next

After starting Qizx Server, you might want to:

Create a first XML Library (database):
- With QizxStudio, right-click on the server icon in the Library view and choose Create a Library.
- With the demo in your web browser, click on 'mklib: create an XML Library' on the left, and enter the name of your Library, then submit request.
- With command-line tool qizx, type a command similar to the following one:
```
qizx -g http://myhost:8080/qizx/api -auth credentials -library mylib -create
```
Populate the XML Library with documents:
This is similar to using a local XML Library group, and explained in chapter Chapter 4, Getting started.
Define Access Control rules: this is useful if your applications have several classes of users, and you want to restrict the access of some users to certain parts of your database. By default all users can read, query and update all documents and collections.
See dedicated chapter Section 3, “Access Control”.

3. Access Control

Access Control is the mechanism that controls whether a User (already authenticated) may read, query or modify a Document or a Collection inside an XML Library.

By default all users can read, query and update all documents and collections.
Restricting access is useful if an application of Qizx Server involves diverse kinds of users, some less trusted that others, and you want to prevent some users of doing some operations.
If all users are trusted, or if there is only one generic user, this feature is perhaps not useful.

In Qizx, Access Control is clearly separated from user authentication and user management (which are not part of Qizx core, and delegated to the servlet container in Qizx Server).

User names and Role names are defined externally: in Qizx Server, they are defined by the Servlet container configuration.

Access Control in Qizx Server is by default based on ACL (Access Control Lists). This allows a powerful control with negligible performance impact.

It is possible to change the AccessControl implementation by plugging a different class, but that is advanced customization, unlikely to be necessary.

3.1. How ACL work in Qizx Server

Basically, Access Control Lists are lists of elementary grant and deny entries.
Each Access Control Entry (ACE) defines
- access rights (permissions),
- for a set of users,
- on an object (in Qizx: a Collection or a Document).
For example (in informal syntax): "grant user john permission read,write on collection /users/john"
or "deny all users permission write on collection /system".
ACL are inherited. This means that access rights defined on a Collection are applicable on all documents and sub-collections contained within the collection (unless they have their own rules).
This is a powerful mechanism, since a few rules (ACE) are sufficient for controlling access to an entire hierarchy of documents, without need to define rules on each and every document or collection.
Typically, with a few ACL is it possible to specify for example that:
- The whole database is read-only for users
- Except specific collections /A and /B and their children which can be read and queried only by privileged users U1 and U2.
- Collection /C can be modified only by certain users which have the role 'manager'.
The order of ACE is important: an ACE can be superseded by a following ACE.
Example (still in informal syntax): here the second ACE supersedes the first one, so admin has the permission to write.
```
   deny all users permission write on collection /system
   grant user admin permission write on collection /system
```
User names and Role names are defined externally by the Servlet container configuration.

In Qizx, ACL are defined using an XML syntax. Example:

<accesscontrol>
  <member path='/'>
    <deny user='*' permissions='Write'/>
    <grant user='*' permissions='Read'/>
    <grant role='manager' permissions='SetContent'/>
  </member>
  <member path='/users/john'>
    <deny user='*' permissions='Read Write'/>  <!-- forbidden to anybody -->
    <grant user='john' permissions='All'/>
    <grant user='jane jim' permissions='Read'/>  <!-- allow friends to read -->
  </member> 
</accesscontrol>

Root element is accesscontrol.
It contains a list of member elements
Each member element contains a list of ACE for a particular collection (or document but this is not encouraged). The attribute path must be present.
An ACE can be grant or deny.
It has a mandatory attribute permissions. The value of this attribute is a list of permission names separated by spaces or commas. See table of permissions hereafter.
A grant/deny must have either a user attribute or a role attribute.
- Value of attribute user can be a list of user names: e.g user='jane jim' .
- Value of attribute user can also be '*': all users.
- Value of role can be a list of role names.

Table 5.1. Permissions
Symbol	Permission
`GetContent`	Read the content of a document, list children of a collection
`SetContent`	Replace content of a document, insert/delete/replace children of a collection.
`GetProperty`	Read and query properties of document or collection.
`SetProperty`	Add/modify/delete properties of document or collection.
`Content`	short name for GetContent + SetContent
`Properties`	short name for GetProperty + SetProperty
`Read`	short name for GetContent + GetProperty
`Write`	short name for SetContent + SetProperty
`All`	short name for all permissions

3.2. Setting ACL in Qizx Server

Setting ACL in a Qizx Server is an administrator operation. It cannot be performed by ordinary users.

In QizxStudio: right-click on a collection and select "Modify Access Rights". A dialog appears and allows modifying the XML representation of access rights for the collection.
In Qizx Server API demo: Use link setacl: Set Access Rights and enter the XML representation of access-rights.
In command-line qizx: Prepare a file containing the ACL in XML. Use option switch -set-acl file to upload the ACL to the server.

A more user-friendly interface should be provided in later versions.

4. Developer Documentation

4.1. API service

The REST API is documented through the demo pages.

Using a web browser, enter the URL: http://myhost:8080/qizx/

(assuming that the servlet container is on that host and port, and that you have named the webapp 'qizx', like in the examples above).

Each request is documented by a page describing it and allowing to execute it:

Purpose of the request
Format of results
Possible errors. The format of errors is described in a dedicated page.
Arguments: each argument corresponds with a form field.
A Submit button can be used to run the request with the provided arguments and see the results.

4.2. XQuery services

XQuery Services is a simple mechanism allowing to call XQuery scripts stored in the server.

Such scripts need only to follow a few conventions and be placed in the appropriate location inside the server storage.

The mapping to an URL is straightforward. Request parameters are automatically bound to XQuery variables with conversion to the declared type.

4.2.1. Protocol

Like other services in Qizx Server, this service uses HTTP.

Supported request formats:

GET with parameters in the URL
POST with form-urlencoded content type.
POST with multipart/form-data content type.

Parameters are retrieved transparently and bound to XQuery variables (see next section).

Restrictions:

anonymous file fields are not supported.
several parameters with same name yield undefined results.
Binary file fields (e.g images) are not supported currently.

4.2.2. Creating services

An elementary service is defined by simply depositing a XQuery script within the xqs directory of a Qizx_Server_Storage.
For example, a script named search.xqs is placed in sub-directory client of Qizx_Server_Storage/xqs. It can be invoked with the following URL (still using the same example host, port, and webapp as before):
```
http://myhost:8080/qizx/xqs/client/search.xqs
```
This xqs directory can be structured in packages. By convention, a package should represent a set of related services.
For example the package client would contain several services on clients like search, retrieve, create, update, delete.

4.2.3. Description of available services

A description of available services in a particular package can be obtained by a GET at the URL of that package.

This will return a XML description of services available in this package.

Example:

http://myhost:8080/qizx/xqs/client/

returning:

<services package="/client">
  <service name="create" result-type="xs:string">
    <parameter name="client-name" type="xs:string"/>
    <output-option name="method">text</output-option>
    <documentation>Create a new client and return the id.</documentation>
  </service>
  ...
</services>

The XML description is straightforward: basically it is a list of service elements wrapped in a top element services.
Example:
```
<services package="/client">

  <service name="create" result-type="xs:string">
    <parameter name="client-name" type="xs:string"/>
    <output-option name="method">text</output-option>
    <documentation>Create a new client and return an id.</documentation>
  </service>
  ...
</services>
```
- Each service has a name and type attribute (type is inferred from the expression).
- Then come parameters with their name (without prefix) and type.
- Then output options with name and value (as content).
- Then the documentation comment if any.

4.2.4. Parameters

Parameters for each elementary request are specified through global XQuery variables:

declare variable $param:client-name as xs:string external;
declare variable $param:year as xs:integer external;
...

The name of a parameter must use the predefined namespace prefix param.
A parameter may have a type declaration. If not defined, the actual type is xs:string.
If the type is specified, the value of the parameter can be converted from the string representation to the specified type.
Special case: if type node() or element() is specified, the value is considered XML and parsed into a node. In practice, only XML elements can be passed this way (because of parsing): attribute, comment, processing-instruction, text nodes are not supported.
A parameter may have a default value:
```
declare variable $param:year as xs:integer := 2010;
```
A parameter with a default value needs not be specified in the request.
An execution error happens on use of a parameter without default value and not specified in the request.

4.2.5. Result type and output options

To define the way results are formatted in the response, XQS uses XQuery options in the declaration part of each query.

Example:

  declare option output:encoding "UTF-8";

Available options:

These are the common serialization options (see section Serialization), preceded by the output: prefix. This prefix is predefined (this is actually a XQuery 1.1 feature).

Most important options are:

method: standard values "XML", "XHTML", "HTML" and "TEXT".
encoding: values like "UTF-8", "ISO-8859-15".
content-type: allows fine control on the mime type of the request response (header Content-Type).
Normally this value is deduced from the output method: for example text/xml if method="XML", etc. but in some circumstances you may want to use application-specific values like image/svg+xml.

4.2.6. Documentation of services

Each request can be documented by a special comment beginning with a tilde character:

Example:

(:~
 :  Create a new client and return an id.
 :)
declare variable $param:client-name as xs:string external;
...

Note: no particular structure is defined yet for these comments.

Prev	Up	Next
Chapter 4. Getting started	Home	Chapter 6. Support of standard XQuery Update