|
|
| (19 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| __NOTOC__
| | #redirect [[Development:XMLDB Documentation]] |
| One of the main upcoming features in Moodle 1.7 will be its ability to work with some more [[wikipedia:RDBMS|RDBMS]] ([[wikipedia:MSSQL|MSSQL]] and [[wikipedia:Oracle database|Oracle]]) while maintaining everything working properly with both [[MySQL]] and [[PostgreSQL]]. As Moodle core uses [http://adodb.sourceforge.net/ ADOdb] internally, this possibility has been present since the beginning and, with the current maturity of the project (5 years old baby!), this can be a good moment to sort all this out.
| |
| | |
| Initially, all our tests and preliminary work was to inspect how [http://adodb.sourceforge.net/ ADOdb] was doing its work, and how we could mix together all those 4 RDBMS, whose SQL dialects, although pretty similar, have some differences and idiosyncrasies that force us to do some important changes to our current database code (formerly '''datalib.php''') and how it's used by the rest of Moodle.
| |
| | |
| All the changes to be performed, which primary objective is to enable Moodle to work with more RDBMS must be be filled following the next non-functional requirements:
| |
| | |
| * '''Provide one layer (new) for DB creation/upgrade''' ([[wikipedia:Data_Definition_Language|DDL]]): With this, developers will create their structures in one neutral form, independent of the exact implementation to be used by each RDBMS.
| |
| | |
| * '''Provide one layer (existing) for DB handling''' ([[wikipedia:Data_Manipulation_Language|DML]]): With this, developers will request/store information also in one nuetral form, independent of the RDBMS being used.
| |
| | |
| * '''Easy migration path from previous versions''': The current installation/upgrade system will work until, at least, Moodle 2.0, allowing 3rd part developers to migrate along the time to the new system.
| |
| | |
| * '''Simple, usable and effective''': Until now, the way to upgrade Moodle has been really cool and it has worked pretty fine since the beginning, but it has forced developers to maintain at least two installation and two upgrade scripts for each module/plugin. The new alternative will have only one file to install and one file to upgrade (per modude/plugin too), reducing the possibility of mistakes in an high degree.
| |
| | |
| * '''Conditional code usage must be minimised''': Database libraries must accept 99% of potential SQL sentences, building/transforming them as necessary to work properly under any RDBMS. The number of places using custom (per DB) code should be minimum.
| |
| | |
| * '''Well documented''': All the functions defined, both at DML and DDL level must be well documented, helping the developer to find and use the correct one in each situation.
| |
| | |
| == The Stack ==
| |
| | |
| The next stack shows how Moodle 1.7 code will interact with underlying RDBMS. It will help us to understand a bit more what we are trying to do and will explain some of the points related in the Roadmap (below in this page).
| |
| | |
| [[Image:MoodleDBStack.png|center]]
| |
| | |
| Moodle code will use two ''languages'' to perform its DB actions:
| |
| | |
| * '''XMLDB neutral description files''': To create, modify and delete database objects (DDL: create/alter/drop tables, fields, indexes, constraints...). It consists in a collection of validated, standard, XML files. They will be used to define all the DB objects. New for 1.7.
| |
| * '''Moodle SQL neutral statements''': To add, modify, delete and select database information (DML: insert/update/delete/select records). To modify for 1.7.
| |
| | |
| Please note the '''neutral''' keyword used in the expressions above. It means that both '''languages''' will be 100% the same, independently of the underlying RDBMS being used. And this must be particularly true for the XMLDB part. Point.
| |
| | |
| Obviously it's possible that in the SQL part we found some specialised queries (using complex joins, regular expressions...) that will force us to do some '''Exceptions'''. Well, they can exist (in fact, they exist), but we always must try to provide an alternate path to minimise them using neutral statements and standard libraries.
| |
| | |
| Each one of the '''languages''' above will use its own library to do the work:
| |
| | |
| * '''Moodle DDL Library''' (ddllib.php): Where all the functions needed to handle DB objects will exist. This library in new for 1.7 and will provide developers with an high level of abstraction. As input it will accept some well defined objects and actions and it will execute the proper commands for the RDBMS being used.
| |
| * '''Moodle DML Library''' (datalib.php): This is our old-friend library where all the functions to handle DB contents exist. The library needs some modifications for 1.7 and provides developers with an high level of abstraction. As input it will accept some well-known parameters and it will execute the proper statements for the RDBMS being used.
| |
| | |
| Both this libraries (plus the small '''Exceptions''' bar) will perform all their actions using the '''ADOdb Database Abstraction Library for PHP''' that will receive all the request from them, communicate with the DB ('''MySQL''', '''PostgreSQL''', '''Oracle''' or '''SQL*Server'''), retrieve results and forward them back to originator library.
| |
| | |
| == The process ==
| |
| | |
| * [[XMLDB Roadmap|Roadmap]]: Where the whole process is defined. It has been splitted in small chuncks to be performed and tested easily. Also, such documents should be used to track what's done and what's pending following some easy nomenclature.
| |
| | |
| * [[XMLDB Problems|Problems]]: A comprensive list of matters that need to be determined/solved prior to incorporate them to the [[XMLDB Roadmap|roadmap]].
| |
| | |
| ==The roadmap==
| |
| | |
| Extending DB support to new RDBMS implies a lot of changes in current code, plus the creation of some new libraries/artifacts. In this chapter, the main points of the whole process are related and explained briefly. Please note that points are numbered and should be performed in that order (when parallel work is possible it's pointed).
| |
| | |
| # <p>[[XMLDB Splitting datalib.php|Splitting datalib.php]]: Currently such file includes SQL DDL (table_column()...), SQL DML (select, insert...) and some functions that aren't related directly with ADOdb (category_parent_visible, print_object()...). With more and more functions coming to support new RDBMS (DDL mainly), it could be a good idea to split the file into 3 parts, one to handle DDL (creation/alter/drop.. - could be called ddllib.php, only used on install/upgrade/migration), other to handle DML (where all the current insert, select, update.. functions will go - could continue being datalib.php) and other to store all those non-ADOdb related functions (or move them to their proper lib - course/lib.php, user/lib.php...).</p> | |
| # <p>[[XMLDB Modifying DML functions|Modifying DML functions]]: Some changes have to be performed to extend support for other RDBMS in all this functions, like changes to support LIMIT clauses (that are emulated both under Oracle and MSSQL), insert_record() changes, handling of CLOB/BLOB (large text/binary objects...) and so on. While all this changes are performed in datalib.php, code across Moodle must be modified to perform the correct function calls. This point must be performed '''after point 1 was finished'''. This is really a big point!</p>
| |
| # <p>[[XMLDB Defining one XML structure|XML Defining one XML structure and handling it]]: The new XML structure used to define any DB (used at installation) should be as complex as needed, defining all the info required to transform it into usable DDL statements against any RDBMS. Validation and some sort of edition/inverse engineering will be present. This point can be performed '''in parallel with 1 & 2'''.</p>
| |
| # <p>[[XMLDB Creating new DDL functions|Creating new DDL functions]]: In order to be able to install/upgrade any DB component of Moodle, we need to suppot a bunch of new functions to CREATE/DROP/ALTER table, indexes, constraints, sequences, fields. All this functions should be cross-compatible and generate the proper SQL for each RDBMS. They will reside in the new ddllib.php (point 1) and will be included only in the installation/upgrade processes. This point must be performed '''after point 3 was finished'''.</p>
| |
| # <p>[[XMLDB Modifying the installation/upgrade process|Modifying the installation/upgrade process]]: While compatibility with the old method will be present and any module/block/format/question... could use that approach, 1.7 should use (for the whole official distro) a new approach. This will be based in the version number (like the current one) and will execute any action defined in one new XML file (replacing current *.sql files) and one new upgrade.php file (replacing current *php files). Both MySQL and PostgreSQL installations will be able to run both upgrade systems (first, the old one, then the new one) but new RDBMS only will work with the new one. In any case, the old one will become deprecated and will disappear in 2.0. This point must be performed '''after point 4 was finished'''.</p>
| |
| #<p>[[XMLDB Documenting everything|Documenting everything]]: All the libraries above plus the new install/upgrade system must be completely documented to allow developers one easy migration from the old approach. Things like field types, forbidden words an so on must be well defined in order to allow the quick adoption and better usage of all the system. This point can be performed '''in parallel with points 1-5'''.</p>
| |
| | |
| ==See also==
| |
| * [[XMLDB preliminary links]] - A collection of links about general info, searched and analysed at the initial stages of the project
| |
| * [[XMLDB preliminary notes]] - A collection of notes collected in the early stages of this project, pointing both to some changes required and some problems to solve.
| |
| * [[XMLDB column types]] - Some links about column types inside every RDBMS and their characteristics
| |
| * [[XMLDB reserved words]] - A collection of reserver words inside each RDBMS
| |
| | |
| [[Category:Developer]]
| |
| [[Category:XMLDB]] | |
