Student projects/SQLite

Note: This page outlines ideas for the XMLDB/SQLite project. It's a specification under construction! If you have any comments or suggestions, please add them to the page comments.

Summary

This project is part of the 2008 edition of Google Summer of Code (GSoC). Mentor: Penny Leach. Student: Andrei Bautu.

The objective of this project is to extend Moodle's database support and features. The main working directions established in the proposal are:

• implement SQLite in Moodle database abstraction layer;
• implement a tool which will allow administrators to copy a live Moodle database and switch into 'testing mode' using a SQLite copy of the live database.

SQLite support

Adding support for SQLite in Moodle requires a careful analysis of compatibility between Moodle's database requirements and SQLite's features (e.g. https://docs.moodle.org/en/Development:XMLDB_problems).

SQLite supports almost all of the features of SQL92 (see http://www.sqlite.org/omitted.html). Some areas that need special attention are:

• proper settings for Unicode support, long field names, numeric/associative fetches, metadata support;
• incompatible SQL commands (e.g. SQLite does not support MySQL's TRUNCATE because it is not part of standard SQL);
• unsupported SQL commands (SQLite does not support GRANT/REVOKE);
• unsupported SQL keywords (SQLite does not support right and full join);
• security issues (in a misconfigured environment, the SQLite database may be downloaded).

A serious limitation of SQLite is the limited ALTER TABLE command. In this case, the general solution for the missing clauses problem consists in replacing the query with a series of queries that will create a second table to replace the original one. This will be done by the database layer.

Testing mode tool

Using this tool, the live database will be copied to a SQLite testing database. This will allow administrators to experiment in testing mode without performing a backup of the live database or worrying about breaking their live site. The tool will use Moodle's XMLDB to replicate the data from the live database to the test database.

Project timeline

• April, 14 – May, 25 – develop specification
  • discus required functionality with the mentor;
  • analyze database incompatibilities and possible solutions;
  • collect feedback from the mentor and community.
• May, 26 – June, 22 – SQLite support
  • implement SQLite support in Moodle database abstraction layer;
  • test and benchmark SQLite support
• June, 23 – July, 31 – Testing mode tool
  • implement database migration mechanism;
  • provide alternative solutions for non-standard SQL commands in Moodle (if necesary)
• August, 1 – August, 18 – remove reported bugs
• September, 3 – submit code and resource files Google

Completion criteria

Moodle will allow users to:

• work with SQLite databases;
• use testing mode (i.e. replicate a live database to a SQlite test database)

Moodle installation with SQLite database

Moodle 2.0 offers support for SQLite3 database installations. In this case, no database setup is required. Before starting a Moodle install with a SQLite database be sure to read the Installing Moodle page for general information. Focusing on the Requirements section, you will notice that the pdo and pdo_sqlite extensions are required for the (experimental) SQLite 3 database support.

The SQLite3 database file will be created during the installation process. By default, the database file will be store in Moodle's data directory (see Creating the data directory). During install, the web server must have write access on the directory where the database file will be stored. After installation, the web server must have read-write access to the database file.

The installation process follows the following steps:

1. Select the language used during the installation process
2. Check your PHP settings
3. Confirm the locations of this Moodle installation

   The default SQLite will use the Moodle's data directory for storing of the database file. The SQLite database file also contains a simple protection mechanism that prevents direct downloads of it. However, as emphasised on the Creating the data directory page, you should do your best to protect the Moodle's data directory (and the SQLite database file) against direct web access.

4. Configure the database
   1. Select the database type (SQLite 3 (PDO))
   2. Fill in the Host Server (optional)

      This setting specifies the directory where the database file will be stored. If it is empty or it is "localhost", then Moodle's data directory will be used. Otherwise, this settings will be used. In order to avoid problems, do not use a path relative to the current working directory (e.g. ../datadir); use instead an absolute path (e.g. /home/abautu/datadir or C:\Inetpub\abautu\datadir).

   3. Fill in a database name, an user and a password (all three are optional)

      The database name, user and password are used to compute the real name of the database file. The database filename is username_database_passwordmd5.sq3.php. If the username is empty, then the username_ prefix is omitted. This prefix allows databases belonging to the same user to be grouped together in a name-ordered directory listing. The passwordmd5 part of the filename offers a simple protection of the password in case the database directory allows web browsing (try to avoid this). The .php filename extension allows the database file to stop direct web downloads in most server setups (don't change it). Notice that the file is still a binary SQLite3 compatible database file. This filenaming scheme allows different users to use the same directory for storing database files, and allows a user to have multiple databases with the same name but different passwords.

   4. Fill in the prefix to use for all table names (optional)
5. Checking your environment
6. Download a language pack for your Moodle site
7. Continue the installation process with the current settings

See also

• GSOC/2008
• Database migration utility on Tracker
• XMLDB/SQLite application abstract
• Using Moodle How to add sqlite-support? forum discussion