Development:Modules and plugins replacement proposal
The Plugins plugin
This is a design proposal for the system that will be created to replace the Modules and plugins database. Development of this system will be undertaken by Moodle HQ after this proposal has reached maturity.
The initial direction
The new system is to completely replace the current modules and plugins database, as well as the themes database. It has been decided that the new system will be a local plugin within Moodle. This gives us the advantage that we can both make use of the Moodle backend, as well as benefit from already having all of the user accounts, capabilities/permissions, and setup of Moodle.org for when this new system gets implemented. Because of this decision the new system will be making use of the Moodle API's wherever possible, this includes, comments, tags, if possible ratings (some improvements required) and of course the Moodle file API. Integration and reliance on these Moodle components will be documented within this proposal along with the database structure, and screens that the system will consist of.
The following are the major design focuses for this new system:
| Categorise contributed code | Categorisation of code based upon the type of the code, and perhaps for unspecific areas such as hacks categorisation based upon the nature of the code as well. |
|---|---|
| Track versions of contributed code | This will allow contributors to maintain multiple versions of thier code, hopefully better supporting the diversity of code between Moodle versions. |
| A trusted review system | A trusted review system consisting of both a general review and criteria based review for versions of contributed code. |
| Complete supported version tracking | Tracking of both major and minor release support by version of contributed code. |
| Full support for all types of contributed code | All native Moodle plugin types including theme's as well as core patches/hacks. |
| Improved user interaction | We hope to get users interacting more with contributors. Providing ratings, commenting, reiviewing, perhaps even contributing themselves. |
| Better search | The ability to easily search for plugins, and plugin versions, comments, and reviews. |
| More comprehensive reporting | Sortable, customisable reports for the an array of things within the new system. |
| Notification services and systems | Choose to be notified when there are new plugins, new versions, new reviews, and changes or any combination of. |
| Support for popular version control systems | Be able to link in to you projects git, cvs, or svn repository. |
| Statistics | Downloads, views, reviews, ratings, activity etc. |
| Awards | There will be awards for plugins that can be used for things such as showing the level of the plugin (gold, silver, bronze), or marking it was a status (featured). |
Categorisation
Obviously one of the big things that we want to achieve with this new system is proper categorisation of contributed code.
It would be great to be able to browse plugins by their category, search within a category, and view information organised/limited by category.
On top of this is tagging, the new system will support tagging pluggins so that we can further compartmentalise and then organise, limit, and view based upon this axis as well.
Plugins, plugin versions, and supported versions of Moodle
This new system will track not only plugins, but versions of plugins. Contributors will be able to create, or register a plugin with us and then as they continue to develope and maintain it release further versions as they desire.
When a new version for a plugin is created the contributor is able to enter detailed information at the version such as release notes, details about the VCS system used to manage the code, the previous version this version builds upon, and importantly the different versions of Moodle this plugin is designed to work with.
This is of course much more comprehensible than the information gathered by the current system where contributors update a plugin for each release and previous release information is lost.
Reviews and awards
Many people have asked for this functionality and we have listened, one of the major new features for this system will be reviews.
Reviews will be completed by trusted people who are known to be knowledgeable in Moodle development.
The reviews themselves will consist of both a general review or the plugin as well as the review of specific criteria/ideals such as security, coding standard, usability, and documentation. The reviewer will then be able to rate each criteria, and of course the plugin as a whole.
In conjunction to this there will also be an awards system whereby when a plugin achieves notable goals it will be given awards to recognise this. This will in the initial release of the new system be a manual undertaking by an administrator.
Through the process of getting trusted reviews and awards we are again able to provide better recommendations and direction to our users.
Version control systems and downloads
With this new system we will start collecting information about the version control system that the contributor is using. We gather this information for each version of a plugin that is created and are able to collect some or all of the following: the new of the vcs system, the url to the repository, the name of the branch, and the name of the tag.
With this information we can provide useful information and links to the end user, and whilst presently we not not attempt any integration of monitoring perhaps in the future.
In regards to downloads we will still require users to upload thier plugin as a package, this allows us to ensure we will always have a valid download to historical versions. Contributors will be able to specify an optional alternative download location as well if they choose, however in order to record accurate statistics we will mediate the download link.
Reporting and notifications
With all the new functionality and information that is going to be put into this new system we are going to need better methods of monitoring what is going on and notify certain parties when things change or are introduced.
For just this purpose the new system will be created with several reports and a notifications subsystem to keep people up to date with what is happening in the system.
Concepts
The following are brief descriptions of the main concepts that will go into this new system.
Plugins
Plugins of course are the main focus for this new system which is being developed specifically to manage our collection of contributed plugins.
The most important thing to note is that while each contributed plugin will have a plugin record describing it in general, the actual plugin itself will be properly described by a plugin version.
Plugins can have a number of plugin versions, awards, and contributors. They will also belong to a category, and 0..n collections.
Plugin versions
Plugin versions relate to an actual version of a contributed plugin. Every time a contributor releases a new version of a plugin in the system they will need to log in and create a new plugin version. The plugin versions record in the database contains information about the specific version it was created for. Once created the uploaded version cannot be changed. The details about the version can still be edited but in order to ensure that awards and reviews are accurate the contributor will not be able to change the uploaded plugin. If they make changes they will need to create a new plugin version.
Plugin versions can have a number or reviews, awards, and supported versions. They always belong to a plugin.
Reviews
Reviews are an integral part of this new system. Each review is done on a plugin version (never a plugin in general) and consists of both a general review as well as a review of specified criteria such as security, usability etc.
Reviews can only be created by privileged users.
Awards
Awards will be created to show that a plugin, or plugin version has achieved something notable. They will only be able to be handed out by privileged users. Awards can be given to either a plugin, or a plugin version. In the future there may be automatic systems created to hand out rewards however at the moment it will be a purely manual task.
Sets
Sets will be used to organise plugins into special collections that can be presented through RSS or used within output widgets created for this new system. Sets cover ideas such as Featured plugins, Staff picks, Partner contributions, and any other matrix not covered by awards or calculation of information.
Database structure
The following is the proposed database structure for this new system.
| local_contrib_plugin | ||
|---|---|---|
| Contributed plugins and/or code | ||
| id | int (10) unsigned not null | |
| categoryid | int (10) unsigned not null | Maps to local_contrib_category.id |
| name | varchar (255) not null | The name of the plugin |
| shortdescription | smalltext | A short plain text description of the module |
| description | mediumtext | A complete description of the plugin |
| descriptionformat | int(2) unsigned default 0 not null | |
| websiteurl | varchar (255) default NULL | A url to a site/page about this plugin |
| sourcecontrolurl | varchar (255) default NULL | A url to a system being used to manage the source for this plugin |
| faqs | mediumtext | Frequently asked questions for/about this plugin |
| faqsformat | int(2) unsigned default 0 not null | |
| documentation | mediumtext | Documenation for/about this plugin |
| documentationformat | int(2) unsigned default 0 not null | |
| documentationurl | varchar (255) default NULL | A url to further documentation about this plugin |
| timecreated | int (10) unsigned | Time the plugin was created within out system |
| timelastmodified | int (10) unsigned | Last time the plugin or any of its versions were modified |
| approved | int(1) unsigned default 0 not null | Flag to track when this module was approved |
| available | int(1) unsigned default 0 not null | Toggle availablility of this module easily. |
| local_contrib_vers | ||
|---|---|---|
| Tracks a version of a plugin | ||
| id | int (10) unsigned not null | |
| pluginid | int (10) unsigned not null | Maps to local_contrib_plugin.id |
| previousversionid | int (10) unsigned default NULL | Id of the version this version supersedes, maps to local_contrib_vers.id |
| version | varchar(255) | The version as a string |
| name | varchar(255) default NULL | The name of the version |
| status | varchar(255) default NULL | The status of the version, e.g. stable, development |
| releasenotes | mediumtext | Notes about this version of the plugin |
| releasenotesformat | int(2) unsigned default 0 not null | |
| websiteurl | varchar (255) default NULL | A url to a site/page about this plugin version |
| altdownloadurl | varchar (255) default NULL | An alternative download location for this plugin version |
| md5sum | varchar (255) not null | The md5 sum for the download |
| vcssystem | varchar (255) default NULL | The version control system used to manage this version (git, cvs, svn) |
| vcsrepositoryurl | varchar (255) default NULL | The url to the VCS used to manage this plugin version |
| vcsbranch | varchar (255) default NULL | The branch for this version within the VCS |
| vcstag | varchar (255) default NULL | The tag for this version within the VCS |
| timecreated | int (10) unsigned | Timestamp when this version was created |
| timelastmodified | int (10) unsigned | Timestamp when this version was last updated in some way |
| approved | int(1) unsigned default 0 not null | Set to 1 when the version has been approved and is available to the public |
| available | int(1) unsigned default 1 not null | When on this version is visible to the public |
| local_contrib_vers_download | ||
|---|---|---|
| Tracks version downloads either through file or link | ||
| id | int (10) unsigned not null | |
| versionid | int (10) unsigned not null | Maps to local_contrib_vers.id |
| timedownloaded | int (10) unsigned not null | Timestamp from when version was downloaded |
| userid | int (10) unsigned default null | Optional id of user who downloaded this plugin |
| downloadmethod | varchar (255) not null default 'file' | The means of download |
| local_contrib_category | ||
|---|---|---|
| Plugin categories | ||
| id | int (10) unsigned not null | |
| parentid | int (10) unsigned not null | Maps to local_contrib_category.id |
| name | varchar (255) not null | The name of this category |
| shortdescription | smalltext | A short plain text description of this category |
| description | mediumtext | A full description of this category. |
| descriptionformat | int(2) unsigned default 0 not null | |
| local_contrib_contributor | ||
|---|---|---|
| Tracks the contributors to a project so that no one gets forgotten. | ||
| id | int (10) unsigned not null | |
| userid | int (10) unsigned not null | Maps to user.id |
| pluginid | int (10) unsigned not null | Maps to local_contrib_plugin.id |
| creator | int(1) unsigned default 0 | If true the user is the creator of the plugin |
| type | varchar (125) unsigned default 'active' | Enum active, inactive, primary |
| timecreated | int (10) unsigned not null | Timestamp when this user became a contribtor |
| Unqiue key userid + pluginid | ||
| local_contrib_set | ||
|---|---|---|
| A collection that plugins can belong to | ||
| id | int (10) unsigned not null | |
| name | varchar(255) | The name of the collection |
| shortdescription | smalltext | A short description of the collection |
| maxplugins | int (10) unsigned default NULL | The maximum number of plugins that can exist in this collection. |
| local_contrib_set_plugin | ||
|---|---|---|
| A collection that plugins can belong to | ||
| id | int (10) unsigned not null | |
| collectionid | int (10) unsigned not null | |
| pluginid | int (10) unsigned not null | |
| userid | int (10) unsigned not null | The id of the user who added this plugin to the collection |
| timeadded | int (10) unsigned not null | Timestamp when this plugin was added to the collection |
| sort | int (10) unsigned not null default 0 | A sortable field |
| Unqiue key collectionid + pluginid | ||
| local_contrib_supported_vers | ||
|---|---|---|
| Contains the different Moodle versions that a plugin version can support | ||
| id | int (10) unsigned not null | |
| name | varchar(255) | Name of the software plus version |
| version | decimal (12,5) not null | Version number for sorting |
| timecreated | int (10) unsigned not null | Timestamp when this version was made available to support |
| contrib_supported_versions | ||
|---|---|---|
| Maps a supported Moodle version to a plugin | ||
| id | int (10) unsigned not null | |
| versionid | int (10) unsigned not null | Maps to local_contrib_vers.id |
| softwareversionid | int (10) unsigned not null | Maps to contrib_software_version.id |
| local_contrib_review | ||
|---|---|---|
| Used to record plugin version reviews | ||
| id | int (10) unsigned not null | |
| userid | int (10) unsigned not null | The user who published the review, maps to user.id |
| versionid | int (10) unsigned not null | The version of the plugin reviewed, maps to local_contrib_vers.id |
| softwareid | int (10) unsigned not null | The version of software used in this review, maps to contrib_software_version.id |
| generalreview | mediumtext | A general review about this plugin version |
| generalreviewformat | int(2) unsigned default 0 not null | |
| timereviewed | int (10) unsigned not null | The time the plugin version was reviewed |
| local_contrib_review_test | ||
|---|---|---|
| Contains the criteria a plugins versions should be reviewed on. | ||
| id | int (10) unsigned not null | |
| name | varchar (255) | The name of the review criteria |
| description | mediumtext | A description of the review criteria |
| descriptionformat | int(2) unsigned default 0 not null | |
| scaleid | int (10) unsigned not null | The scale to rate on, maps to scale.id |
| local_contrib_review_outcome | ||
|---|---|---|
| Contains the outcome and comments of a reviewed criteria | ||
| id | int (10) unsigned not null | |
| reviewid | int (10) unsigned not null | Maps to local_contrib_review.id |
| criteriaid | int (10) unsigned not null | Maps to local_contrib_review_test.id |
| review | mediumtext | The review of this criteria |
| reviewformat | int(2) unsigned default 0 not null | |
| local_contrib_awards | ||
|---|---|---|
| Contains the different awards that can be granted to a plugin and/or plugin version | ||
| id | int (10) unsigned not null | |
| name | varchar (255) | The name of the award |
| description | mediumtext | A description about this award |
| descriptionformat | int(2) unsigned default 0 not null | |
| timecreated | int (10) unsigned not null | Timestamp from when this award was created |
| local_contrib_plugin_awards | ||
|---|---|---|
| Tracks the awards a plugin, or plugin version have been awarded | ||
| id | int (10) unsigned not null | |
| awardid | int (10) unsigned not null | Maps to contrib_award.id |
| pluginid | int (10) unsigned not null | Maps to local_contrib_plugin.id |
| versionid | int (10) unsigned default NULL | Maps to local_contrib_vers.id |
| timeawarded | int (10) unsigned not null | Timestamp from when this was awarded |
| userid | int (10) unsigned not null | The user who awarded this award |
File areas
There are several file areas that will be used within this new system.
Please not this relies on changes to the core system in order to be possible, presently the file API doesn't support local plugins.
| Type | Contextid | Component | Filearea | Itemid | |
|---|---|---|---|---|---|
| Plugin badge | Image [200x200] | CONTEXT_SYSTEM | local_contrib | plugin_badge | Plugin id |
| Plugin icon | Image [16x16] | CONTEXT_SYSTEM | local_contrib | plugin_icon | Plugin id |
| Plugin screenshots | Images 0..n | CONTEXT_SYSTEM | local_contrib | plugin_screenshots | Plugin version id |
| Review images | Images 0..n | CONTEXT_SYSTEM | local_contrib | review_images | Review id |
| Award image | Image [200x200] | CONTEXT_SYSTEM | local_contrib | award_image | Award id |
| Award icon | Image [16x16] | CONTEXT_SYSTEM | local_contrib | award_icon | Award id |
| Collection icon | Image [64x64] | CONTEXT_SYSTEM | local_contrib | collection_icon | Collection id |
File system storage of plugins
The new system won't use the Moodle 2.0 file API to store plugin archives. Instead it will create a directory within the Moodle data directory, most likely contrib and will start storing archives in there. Archives will be both stored and extracted so that we can inspect the version.php and plugin in general for common problems (invalid plugin names, etc) and perhaps potentially inspect the language files associated with the plugin. On the filesystem files will first be organised by plugin id, then version id. The following examples should help clarify this:
- moodledata/contrib/pluginid/versionid/archive.zip
- moodledata/contrib/pluginid/versionid/pluging/plugincontents
In proposing this means of file storage please note that I am no expert on file systems and will be in the near future talking with those in the office who know more about it than me. Please if you have any ideas here share them on the talk page (don't forget to add your signature).
Capabilities
The following are the capabilities that this new local plugin will introduce and utilise.
| Capability | Type | Description |
|---|---|---|
| local/contrib:view | read | View the system |
| local/contrib:viewunapproved | read | View unapproved plugins |
| local/contrib:createplugins | write | Create new plugins within the system |
| local/contrib:editownplugins | write | Edit plugins the user previously created |
| local/contrib:editanyplugin | write | Edit any plugins within the system |
| local/contrib:deleteownplugin | write | Delete a plugin you created |
| local/contrib:deleteanyplugin | write | Delete a plugin |
| local/contrib:deleteownpluginversion | write | Delete a plugin version you created |
| local/contrib:deleteanypluginversion | write | Delete a plugin version |
| local/contrib:approveplugin | write | Approve a plugin |
| local/contrib:approvepluginversion | write | Approve a plugin version |
| local/contrib:autoapproveplugins | write | New plugins created by this user are automatically approved |
| local/contrib:autoapprovepluginversions | write | New plugin versions created by this user are automatically approved |
| local/contrib:publishreviews | write | Create and publish a review |
| local/contrib:editownreview | write | Edit a review you created |
| local/contrib:editanyreview | write | Edit a review written by anyone |
| local/contrib:comment | write | Add a comment on a plugin, plugin version, or review |
| local/contrib:rate | write | Rate a plugin or plugin version |
| local/contrib:editowntags | write | A user can edit the tags associated with a plugin they manage. |
| local/contrib:editanytags | write | A user can edit the tags associated with any plugin. |
| local/contrib:managesupportableversions | write | Manage supportable versions |
| local/contrib:managecollections | write | Manage collections |
| local/contrib:addtocollections | write | Add plugins to collections |
| local/contrib:manageawards | write | Manage awards |
| local/contrib:handoutawards | write | Give plugins or plugin versions awards |
| local/contrib:managecategories | write | Create, edit, and delete categories |
Ratings
There are three rateable areas within this new system:
- Plugin versions
- Any user with the rate capability will be able to rate a plug-in version.
- Review - general
- Reviewers will be able to give a plugin version a rating for the general section of the review.
- Review - criteria
- Reviewers will be able to give each criteria reviewed for a plugin version its own rating.
The plugin versions ratings will be averaged to form an average rating for the plugin version.
The two review ratings will be averaged to form an average rating for the review as a whole.
Finally people have asked for plugins to be rateable, however this will not be so. Instead a plugins rating will come from it most recent version.
Comments
The are several areas within this new system where users with the comment capability will be able to comment, they are as follows:
- Plugin
- Users will be able to comment on a plugins home page
- Plugin version
- Users will be able to comment on specific plugin version.
- Plugin faqs
- Users will be able to comment on the plugins FAQ page.
- Plugin documentation
- Users will be able to comment on the plugins Docs page.
- Reviews
- Users will be able to comment on specific reviews.
Of course the core comments API will be used to manage all comment areas.
Tags
There is only one area within this new system that users will be able to tag and that is plugins.
Any user with either the editowntags or editanytags capability will be able to manage the tags associated with a plugin.
Of course like ratings and comments the core tag API will be used for this.
Settings
The following are the settings that I anticipate the new system will have.
| Setting | Purpose |
|---|---|
| courseidforgroups | Set to a courseid and user group pictures will be used from that course for users throughout the new system. |
| publishtorss | If enabled then RSS feeds will be generated for a matrix of things including category listings, and plugin collections. |
RSS feeds
If you enable RSS feeds for this new system the following feed will be published:
- New plugins This RSS feed will contain just the new plugins within the system.
- Recent updates This RSS feed will contain new plugins, new versions, and updated versions.
- List of categories This RSS feed will contain a summary of each category in the system.
- Category plugins list This RSS feed will contain the more recently updated plugins in the category.
- Collection list This RSS feed will contain the most recently updated plugins in the collection.
- Plugin information This RSS feed will contain information about a plugin and its versions.
Web services
The new system is going to expose several web services that will be designed for both generic, and core use.
Please note these web services will not likely be implemented for the initial release of the system, they are however worth mentioning as there are some great things that could be done with help from them.
- Search plugins
- Search plugins and retrieve summary information about a plugin.
- Get plugin details
- Fetch detailed information about a plugin.
- Check plugin versions
- Given an array of plugin names and version numbers this service will check if there are any applicable updates and return information if there are.
- Get plugin strings
- Fetch the strings used by a plugin (hopefully useful to AMOS).
Screen descriptions and mockups
The following are the navigation screens for the system, these are screens that will be readily accessible to those who are not logged in or have little or no permissions other than view.
- Reading a review
- Downloading a version of a plugin
Plugin administration screens
- Creating a plugin
- Creating a plugin version
- Adding plugin information
System administration screens
- General settings
- Managing categories
- Managing Moodle versions
- Managing awards
- Managing supportable versions
Reports
- Plugin report
- Plugin approval report
- Plugin version approval report
- Recent activity
- Requiring attention
- Review report
Required core changes
One of the goals for this new system was to get it using the core Moodle systems where possible and practical. Presently this proposal relies of several changes been made to core systems in order to get them into a usable state by this plugin. While I understand it is not desirable to have to make core changes I do think that in all cases making these changes would be of general benefit anyway. In all cases I am happy to be responsible to for making the required changes.
Changes to the ratings API
Presently the ratings system is VERY limited. When a rating is stored is associated by a context and an item id. The problem with this of course is that it immediatly limits ratings to be applied to only one structure per context. In the case of this new system which is currently planned as a local plugin it is running under the system context which immediatly makes the rating system unusable. As part of the development of this new system I would like to purpose the following changes at the same time:
- Add two new columns to the rating table in the database that are similar to fields found in the file table
- component (string 255) This is the frankenstyle name of the component the rating relates to e.g. mod_forum, local_contrib
- ratingarea (string 255) This is a unique name that the utilisine component can use to allow several areas within a component to be rated e.g. for the forum you may have post, and discussion.
- Extend the API to require the two new fields but provide backwards compatability to working without them.
- Review all locations using the rating system presently and refactor to make use of these two new fields.
Post release: Automatically install plugins from within your Moodle site
As I'm sure some of those reading this proposal have realised this new system is geared up in such as way that we will be getting very close to being able to create Moodle functionality to search for, install, and update your modules from within your Moodle site, from this new system.
Presently this is a blue skies proposal but hopefully it will in gain momentum in the future and we will see something happen here.
More information
- MDLSITE-406 Modules and Plugins Database - an improved rating system
- http://moodle.org/mod/forum/discuss.php?d=92972 Modules and Plugins Database - an improved rating system
- MDLSITE-571 Research a replacement for the modules and plugins database
