Development:Modules and plugins replacement proposal

The Plugins plugin

This is a design proposal for the system that will be created to replace the Modules and plugins database. Development of this system will be undertaken by Moodle HQ after this proposal has reached maturity.

The initial direction

The new system is to completely replace the current modules and plugins database, as well as the themes database. It has been decided that the new system will be a local plugin within Moodle. This gives us the advantage that we can both make use of the Moodle backend, as well as benefit from already having all of the user accounts, capabilities/permissions, and setup of Moodle.org for when this new system gets implemented. Because of this decision the new system will be making use of the Moodle API's wherever possible, this includes, comments, tags, if possible ratings (some improvements required) and of course the Moodle file API. Integration and reliance on these Moodle components will be documented within this proposal along with the database structure, and screens that the system will consist of.

The following are the major design focuses for this new system:

Categorise contributed code	Categorisation of code based upon the type of the code, and perhaps for unspecific areas such as hacks categorisation based upon the nature of the code as well.
Track versions of contributed code	This will allow contributors to maintain multiple versions of thier code, hopefully better supporting the diversity of code between Moodle versions.
A trusted review system	A trusted review system consisting of both a general review and criteria based review for versions of contributed code.
Complete supported version tracking	Tracking of both major and minor release support by version of contributed code.
Full support for all types of contributed code	All native Moodle plugin types including theme's as well as core patches/hacks.
Improved user interaction	We hope to get users interacting more with contributors. Providing ratings, commenting, reiviewing, perhaps even contributing themselves.
Better search	The ability to easily search for plugins, and plugin versions, comments, and reviews.
More comprehensive reporting	Sortable, customisable reports for the an array of things within the new system.
Notification services and systems	Choose to be notified when there are new plugins, new versions, new reviews, and changes or any combination of.
Support for popular version control systems	Be able to link in to you projects git, cvs, or svn repository.
Statistics	Downloads, views, reviews, ratings, activity etc.
Awards	There will be awards for plugins that can be used for things such as showing the level of the plugin (gold, silver, bronze), or marking it was a status (featured).

Categorisation

Obviously one of the big things that we want to achieve with this new system is proper categorisation of contributed code.

It would be great to be able to browse plugins by their category, search within a category, and view information organised/limited by category.

On top of this is tagging, the new system will support tagging pluggins so that we can further compartmentalise and then organise, limit, and view based upon this axis as well.

Plugins, plugin versions, and supported versions of Moodle

This new system will track not only plugins, but versions of plugins. Contributors will be able to create, or register a plugin with us and then as they continue to develope and maintain it release further versions as they desire.

When a new version for a plugin is created the contributor is able to enter detailed information at the version such as release notes, details about the VCS system used to manage the code, the previous version this version builds upon, and importantly the different versions of Moodle this plugin is designed to work with.

This is of course much more comprehensible than the information gathered by the current system where contributors update a plugin for each release and previous release information is lost.

Reviews and awards

Many people have asked for this functionality and we have listened, one of the major new features for this system will be reviews.

Reviews will be completed by trusted people who are known to be knowledgeable in Moodle development.

The reviews themselves will consist of both a general review or the plugin as well as the review of specific criteria/ideals such as security, coding standard, usability, and documentation. The reviewer will then be able to rate each criteria, and of course the plugin as a whole.

In conjunction to this there will also be an awards system whereby when a plugin achieves notable goals it will be given awards to recognise this. This will in the initial release of the new system be a manual undertaking by an administrator.

Through the process of getting trusted reviews and awards we are again able to provide better recommendations and direction to our users.

Version control systems and downloads

With this new system we will start collecting information about the version control system that the contributor is using. We gather this information for each version of a plugin that is created and are able to collect some or all of the following: the new of the vcs system, the url to the repository, the name of the branch, and the name of the tag.

With this information we can provide useful information and links to the end user, and whilst presently we not not attempt any integration of monitoring perhaps in the future.

In regards to downloads we will still require users to upload thier plugin as a package, this allows us to ensure we will always have a valid download to historical versions. Contributors will be able to specify an optional alternative download location as well if they choose, however in order to record accurate statistics we will mediate the download link.

Reporting and notifications

With all the new functionality and information that is going to be put into this new system we are going to need better methods of monitoring what is going on and notify certain parties when things change or are introduced.

For just this purpose the new system will be created with several reports and a notifications subsystem to keep people up to date with what is happening in the system.

Concepts

The following are brief descriptions of the main concepts that will go into this new system.

Plugins

Plugins of course are the main focus for this new system which is being developed specifically to manage our collection of contributed plugins.

The most important thing to note is that while each contributed plugin will have a plugin record describing it in general, the actual plugin itself will be properly described by a plugin version.

Plugins can have a number of plugin versions, awards, and contributors. They will also belong to a category, and 0..n collections.

Plugin versions

Plugin versions relate to an actual version of a contributed plugin. Every time a contributor releases a new version of a plugin in the system they will need to log in and create a new plugin version. The plugin versions record in the database contains information about the specific version it was created for. Once created the uploaded version cannot be changed. The details about the version can still be edited but in order to ensure that awards and reviews are accurate the contributor will not be able to change the uploaded plugin. If they make changes they will need to create a new plugin version.

Plugin versions can have a number or reviews, awards, and supported versions. They always belong to a plugin.

Reviews

Reviews are an integral part of this new system. Each review is done on a plugin version (never a plugin in general) and consists of both a general review as well as a review of specified criteria such as security, usability etc.

Reviews can only be created by privileged users.

Awards

Awards will be created to show that a plugin, or plugin version has achieved something notable. They will only be able to be handed out by privileged users. Awards can be given to either a plugin, or a plugin version. In the future there may be automatic systems created to hand out rewards however at the moment it will be a purely manual task.

Sets

Sets will be used to organise plugins into special collections that can be presented through RSS or used within output widgets created for this new system. Sets cover ideas such as Featured plugins, Staff picks, Partner contributions, and any other matrix not covered by awards or calculation of information.

Database structure

The following is the proposed database structure for this new system.

Entity relationship diagram

local_contrib_plugin
Contributed plugins and/or code
id	int (10) unsigned not null
categoryid	int (10) unsigned not null	Maps to local_contrib_category.id
name	varchar (255) not null	The name of the plugin
shortdescription	smalltext	A short plain text description of the module
description	mediumtext	A complete description of the plugin
descriptionformat	int(2) unsigned default 0 not null
websiteurl	varchar (255) default NULL	A url to a site/page about this plugin
sourcecontrolurl	varchar (255) default NULL	A url to a system being used to manage the source for this plugin
faqs	mediumtext	Frequently asked questions for/about this plugin
faqsformat	int(2) unsigned default 0 not null
documentation	mediumtext	Documenation for/about this plugin
documentationformat	int(2) unsigned default 0 not null
documentationurl	varchar (255) default NULL	A url to further documentation about this plugin
timecreated	int (10) unsigned	Time the plugin was created within out system
timelastmodified	int (10) unsigned	Last time the plugin or any of its versions were modified
approved	int(1) unsigned default 0 not null	Flag to track when this module was approved
available	int(1) unsigned default 0 not null	Toggle availablility of this module easily.

local_contrib_vers
Tracks a version of a plugin
id	int (10) unsigned not null
pluginid	int (10) unsigned not null	Maps to local_contrib_plugin.id
previousversionid	int (10) unsigned default NULL	Id of the version this version supersedes, maps to local_contrib_vers.id
version	varchar(255)	The version as a string
name	varchar(255) default NULL	The name of the version
status	varchar(255) default NULL	The status of the version, e.g. stable, development
releasenotes	mediumtext	Notes about this version of the plugin
releasenotesformat	int(2) unsigned default 0 not null
websiteurl	varchar (255) default NULL	A url to a site/page about this plugin version
altdownloadurl	varchar (255) default NULL	An alternative download location for this plugin version
md5sum	varchar (255) not null	The md5 sum for the download
vcssystem	varchar (255) default NULL	The version control system used to manage this version (git, cvs, svn)
vcsrepositoryurl	varchar (255) default NULL	The url to the VCS used to manage this plugin version
vcsbranch	varchar (255) default NULL	The branch for this version within the VCS
vcstag	varchar (255) default NULL	The tag for this version within the VCS
timecreated	int (10) unsigned	Timestamp when this version was created
timelastmodified	int (10) unsigned	Timestamp when this version was last updated in some way
approved	int(1) unsigned default 0 not null	Set to 1 when the version has been approved and is available to the public
available	int(1) unsigned default 1 not null	When on this version is visible to the public

local_contrib_vers_download
Tracks version downloads either through file or link
id	int (10) unsigned not null
versionid	int (10) unsigned not null	Maps to local_contrib_vers.id
timedownloaded	int (10) unsigned not null	Timestamp from when version was downloaded
userid	int (10) unsigned default null	Optional id of user who downloaded this plugin
downloadmethod	varchar (255) not null default 'file'	The means of download

local_contrib_category
Plugin categories
id	int (10) unsigned not null
parentid	int (10) unsigned not null	Maps to local_contrib_category.id
name	varchar (255) not null	The name of this category
shortdescription	smalltext	A short plain text description of this category
description	mediumtext	A full description of this category.
descriptionformat	int(2) unsigned default 0 not null

local_contrib_contributor
Tracks the contributors to a project so that no one gets forgotten.
id	int (10) unsigned not null
userid	int (10) unsigned not null	Maps to user.id
pluginid	int (10) unsigned not null	Maps to local_contrib_plugin.id
creator	int(1) unsigned default 0	If true the user is the creator of the plugin
type	varchar (125) unsigned default 'active'	Enum active, inactive, primary
timecreated	int (10) unsigned not null	Timestamp when this user became a contribtor
Unqiue key userid + pluginid

local_contrib_set
A set that plugins can belong to
id	int (10) unsigned not null
name	varchar(255)	The name of the set
shortdescription	smalltext	A short description of the set
maxplugins	int (10) unsigned default NULL	The maximum number of plugins that can exist in this set.

local_contrib_set_plugin
Used to track which plugins are in which sets
id	int (10) unsigned not null
setid	int (10) unsigned not null
pluginid	int (10) unsigned not null
userid	int (10) unsigned not null	The id of the user who added this plugin to the set
timeadded	int (10) unsigned not null	Timestamp when this plugin was added to the set
sort	int (10) unsigned not null default 0	A sortable field
Unqiue key setid + pluginid

local_contrib_supported_vers
Contains the different Moodle versions that a plugin version can support
id	int (10) unsigned not null
name	varchar(255)	Name of the software plus version
version	decimal (12,5) not null	Version number for sorting
timecreated	int (10) unsigned not null	Timestamp when this version was made available to support

contrib_supported_versions
Maps a supported Moodle version to a plugin
id	int (10) unsigned not null
versionid	int (10) unsigned not null	Maps to local_contrib_vers.id
softwareversionid	int (10) unsigned not null	Maps to contrib_software_version.id

local_contrib_review
Used to record plugin version reviews
id	int (10) unsigned not null
userid	int (10) unsigned not null	The user who published the review, maps to user.id
versionid	int (10) unsigned not null	The version of the plugin reviewed, maps to local_contrib_vers.id
softwareid	int (10) unsigned not null	The version of software used in this review, maps to contrib_software_version.id
generalreview	mediumtext	A general review about this plugin version
generalreviewformat	int(2) unsigned default 0 not null
timereviewed	int (10) unsigned not null	The time the plugin version was reviewed

local_contrib_review_test
Contains the criteria a plugins versions should be reviewed on.
id	int (10) unsigned not null
name	varchar (255)	The name of the review criteria
description	mediumtext	A description of the review criteria
descriptionformat	int(2) unsigned default 0 not null
scaleid	int (10) unsigned not null	The scale to rate on, maps to scale.id

local_contrib_review_outcome
Contains the outcome and comments of a reviewed criteria
id	int (10) unsigned not null
reviewid	int (10) unsigned not null	Maps to local_contrib_review.id
criteriaid	int (10) unsigned not null	Maps to local_contrib_review_test.id
review	mediumtext	The review of this criteria
reviewformat	int(2) unsigned default 0 not null

local_contrib_awards
Contains the different awards that can be granted to a plugin and/or plugin version
id	int (10) unsigned not null
name	varchar (255)	The name of the award
description	mediumtext	A description about this award
descriptionformat	int(2) unsigned default 0 not null
timecreated	int (10) unsigned not null	Timestamp from when this award was created

local_contrib_plugin_awards
Tracks the awards a plugin, or plugin version have been awarded
id	int (10) unsigned not null
awardid	int (10) unsigned not null	Maps to contrib_award.id
pluginid	int (10) unsigned not null	Maps to local_contrib_plugin.id
versionid	int (10) unsigned default NULL	Maps to local_contrib_vers.id
timeawarded	int (10) unsigned not null	Timestamp from when this was awarded
userid	int (10) unsigned not null	The user who awarded this award

File areas

There are several file areas that will be used within this new system.

Please not this relies on changes to the core system in order to be possible, presently the file API doesn't support local plugins.

	Type	Contextid	Component	Filearea	Itemid
Plugin badge	Image [200x200]	CONTEXT_SYSTEM	local_contrib	plugin_badge	Plugin id
Plugin icon	Image [16x16]	CONTEXT_SYSTEM	local_contrib	plugin_icon	Plugin id
Plugin screenshots	Images 0..n	CONTEXT_SYSTEM	local_contrib	plugin_screenshots	Plugin version id
Review images	Images 0..n	CONTEXT_SYSTEM	local_contrib	review_images	Review id
Award image	Image [200x200]	CONTEXT_SYSTEM	local_contrib	award_image	Award id
Award icon	Image [16x16]	CONTEXT_SYSTEM	local_contrib	award_icon	Award id
Collection icon	Image [64x64]	CONTEXT_SYSTEM	local_contrib	collection_icon	Collection id

File system storage of plugins

The new system won't use the Moodle 2.0 file API to store plugin archives. Instead it will create a directory within the Moodle data directory, most likely contrib and will start storing archives in there. Archives will be both stored and extracted so that we can inspect the version.php and plugin in general for common problems (invalid plugin names, etc) and perhaps potentially inspect the language files associated with the plugin. On the filesystem files will first be organised by plugin id, then version id. The following examples should help clarify this:

• moodledata/contrib/pluginid/versionid/archive.zip
• moodledata/contrib/pluginid/versionid/pluging/plugincontents

In proposing this means of file storage please note that I am no expert on file systems and will be in the near future talking with those in the office who know more about it than me. Please if you have any ideas here share them on the talk page (don't forget to add your signature).

Capabilities

The following are the capabilities that this new local plugin will introduce and utilise.

Capability	Type	Description
local/contrib:view	read	View the system
local/contrib:viewunapproved	read	View unapproved plugins
local/contrib:createplugins	write	Create new plugins within the system
local/contrib:editownplugins	write	Edit plugins the user previously created
local/contrib:editanyplugin	write	Edit any plugins within the system
local/contrib:deleteownplugin	write	Delete a plugin you created
local/contrib:deleteanyplugin	write	Delete a plugin
local/contrib:deleteownpluginversion	write	Delete a plugin version you created
local/contrib:deleteanypluginversion	write	Delete a plugin version
local/contrib:approveplugin	write	Approve a plugin
local/contrib:approvepluginversion	write	Approve a plugin version
local/contrib:autoapproveplugins	write	New plugins created by this user are automatically approved
local/contrib:autoapprovepluginversions	write	New plugin versions created by this user are automatically approved
local/contrib:publishreviews	write	Create and publish a review
local/contrib:editownreview	write	Edit a review you created
local/contrib:editanyreview	write	Edit a review written by anyone
local/contrib:comment	write	Add a comment on a plugin, plugin version, or review
local/contrib:rate	write	Rate a plugin or plugin version
local/contrib:editowntags	write	A user can edit the tags associated with a plugin they manage.
local/contrib:editanytags	write	A user can edit the tags associated with any plugin.
local/contrib:managesupportableversions	write	Manage supportable versions
local/contrib:managecollections	write	Manage collections
local/contrib:addtocollections	write	Add plugins to collections
local/contrib:manageawards	write	Manage awards
local/contrib:handoutawards	write	Give plugins or plugin versions awards
local/contrib:managecategories	write	Create, edit, and delete categories

Ratings

There are three rateable areas within this new system:

Plugin versions

Any user with the rate capability will be able to rate a plug-in version.

Review - general

Reviewers will be able to give a plugin version a rating for the general section of the review.

Review - criteria

Reviewers will be able to give each criteria reviewed for a plugin version its own rating.

The plugin versions ratings will be averaged to form an average rating for the plugin version.

The two review ratings will be averaged to form an average rating for the review as a whole.

Finally people have asked for plugins to be rateable, however this will not be so. Instead a plugins rating will come from it most recent version.

Comments

The are several areas within this new system where users with the comment capability will be able to comment, they are as follows:

Plugin

Users will be able to comment on a plugins home page

Plugin version

Users will be able to comment on specific plugin version.

Plugin faqs

Users will be able to comment on the plugins FAQ page.

Plugin documentation

Users will be able to comment on the plugins Docs page.

Reviews

Users will be able to comment on specific reviews.

Of course the core comments API will be used to manage all comment areas.

Tags

There is only one area within this new system that users will be able to tag and that is plugins.

Any user with either the editowntags or editanytags capability will be able to manage the tags associated with a plugin.

Of course like ratings and comments the core tag API will be used for this.

Settings

The following are the settings that I anticipate the new system will have.

Setting	Purpose
courseidforgroups	Set to a courseid and user group pictures will be used from that course for users throughout the new system.
publishtorss	If enabled then RSS feeds will be generated for a matrix of things including category listings, and plugin collections.

RSS feeds

If you enable RSS feeds for this new system the following feed will be published:

• New plugins This RSS feed will contain just the new plugins within the system.
• Recent updates This RSS feed will contain new plugins, new versions, and updated versions.
• List of categories This RSS feed will contain a summary of each category in the system.
• Category plugins list This RSS feed will contain the more recently updated plugins in the category.
• Collection list This RSS feed will contain the most recently updated plugins in the collection.
• Plugin information This RSS feed will contain information about a plugin and its versions.

Web services

The new system is going to expose several web services that will be designed for both generic, and core use.

Please note these web services will not likely be implemented for the initial release of the system, they are however worth mentioning as there are some great things that could be done with help from them.

Search plugins

Search plugins and retrieve summary information about a plugin.

Get plugin details

Fetch detailed information about a plugin.

Check plugin versions

Given an array of plugin names and version numbers this service will check if there are any applicable updates and return information if there are.

Get plugin strings

Fetch the strings used by a plugin (hopefully useful to AMOS).

Screen descriptions and mockups

Navigation screens

The following are the navigation screens for the system, these are screens that will be readily accessible to those who are not logged in or have little or no permissions other than view.

The frontpage of the contrib plugin

Viewing a plugin

Viewing a version of a plugin

• Reading a review
• Downloading a version of a plugin

Plugin administration screens

• Creating a plugin
• Creating a plugin version
• Adding plugin information

System administration screens

• General settings
• Managing categories
• Managing Moodle versions
• Managing awards
• Managing supportable versions

Reports

• Plugin report
• Plugin approval report
• Plugin version approval report
• Recent activity
• Requiring attention
• Review report

Required core changes

One of the goals for this new system was to get it using the core Moodle systems where possible and practical. Presently this proposal relies of several changes been made to core systems in order to get them into a usable state by this plugin. While I understand it is not desirable to have to make core changes I do think that in all cases making these changes would be of general benefit anyway. In all cases I am happy to be responsible to for making the required changes.

Changes to the ratings API

Presently the ratings system is VERY limited. When a rating is stored is associated by a context and an item id. The problem with this of course is that it immediatly limits ratings to be applied to only one structure per context. In the case of this new system which is currently planned as a local plugin it is running under the system context which immediatly makes the rating system unusable. As part of the development of this new system I would like to purpose the following changes at the same time:

1. Add two new columns to the rating table in the database that are similar to fields found in the file table
   • component (string 255) This is the frankenstyle name of the component the rating relates to e.g. mod_forum, local_contrib
   • ratingarea (string 255) This is a unique name that the utilisine component can use to allow several areas within a component to be rated e.g. for the forum you may have post, and discussion.
2. Extend the API to require the two new fields but provide backwards compatability to working without them.
3. Review all locations using the rating system presently and refactor to make use of these two new fields.

Post release: Automatically install plugins from within your Moodle site

As I'm sure some of those reading this proposal have realised this new system is geared up in such as way that we will be getting very close to being able to create Moodle functionality to search for, install, and update your modules from within your Moodle site, from this new system.

Presently this is a blue skies proposal but hopefully it will in gain momentum in the future and we will see something happen here.

More information

• MDLSITE-406 Modules and Plugins Database - an improved rating system
• http://moodle.org/mod/forum/discuss.php?d=92972 Modules and Plugins Database - an improved rating system
• MDLSITE-571 Research a replacement for the modules and plugins database