Modules and plugins replacement proposal

Jump to: navigation, search

The Plugins plugin

This is a design proposal for the system that will be created to replace the Modules and plugins database. Development of this system will be undertaken by Moodle HQ after this proposal has reached maturity.

The initial direction

The new system is to completely replace the current modules and plugins database, as well as the themes database. It has been decided that the new system will be a local plugin within Moodle. This gives us the advantage that we can both make use of the Moodle backend, as well as benefit from already having all of the user accounts, capabilities/permissions, and setup of Moodle.org for when this new system gets implemented. Because of this decision the new system will be making use of the Moodle API's wherever possible, this includes, comments, tags, if possible ratings (some improvements required) and of course the Moodle file API. Integration and reliance on these Moodle components will be documented within this proposal along with the database structure, and screens that the system will consist of.

The following are the major design focuses for this new system:

Categorise contributed code Categorisation of code based upon the type of the code, and perhaps for unspecific areas such as hacks categorisation based upon the nature of the code as well.
Track versions of contributed code This will allow contributors to maintain multiple versions of thier code, hopefully better supporting the diversity of code between Moodle versions.
A trusted review system A trusted review system consisting of both a general review and criteria based review for versions of contributed code.
Complete supported version tracking Tracking of both major and minor release support by version of contributed code.
Full support for all types of contributed code All native Moodle plugin types including theme's as well as core patches/hacks.
Improved user interaction We hope to get users interacting more with contributors. Providing ratings, commenting, reiviewing, perhaps even contributing themselves.
Better search The ability to easily search for plugins, and plugin versions, comments, and reviews.
More comprehensive reporting Sortable, customisable reports for the an array of things within the new system.
Notification services and systems Choose to be notified when there are new plugins, new versions, new reviews, and changes or any combination of.
Support for popular version control systems Be able to link in to you projects git, cvs, or svn repository.
Statistics Downloads, views, reviews, ratings, activity etc.
Awards There will be awards for plugins that can be used for things such as showing the level of the plugin (gold, silver, bronze), or marking it was a status (featured).

Categorisation

Obviously one of the big things that we want to achieve with this new system is proper categorisation of contributed code.

It would be great to be able to browse plugins by their category, search within a category, and view information organised/limited by category.

On top of this is tagging, the new system will support tagging pluggins so that we can further compartmentalise and then organise, limit, and view based upon this axis as well.

Plugins, plugin versions, and supported versions of Moodle

This new system will track not only plugins, but versions of plugins. Contributors will be able to create, or register a plugin with us and then as they continue to develope and maintain it release further versions as they desire.

When a new version for a plugin is created the contributor is able to enter detailed information at the version such as release notes, details about the VCS system used to manage the code, the previous version this version builds upon, and importantly the different versions of Moodle this plugin is designed to work with.

This is of course much more comprehensible than the information gathered by the current system where contributors update a plugin for each release and previous release information is lost.

Reviews and awards

Many people have asked for this functionality and we have listened, one of the major new features for this system will be reviews.

Reviews will be completed by trusted people who are known to be knowledgeable in Moodle development.

The reviews themselves will consist of both a general review or the plugin as well as the review of specific criteria/ideals such as security, coding standard, usability, and documentation. The reviewer will then be able to rate each criteria, and of course the plugin as a whole.

In conjunction to this there will also be an awards system whereby when a plugin achieves notable goals it will be given awards to recognise this. This will in the initial release of the new system be a manual undertaking by an administrator.

Through the process of getting trusted reviews and awards we are again able to provide better recommendations and direction to our users.

Version control systems and downloads

With this new system we will start collecting information about the version control system that the contributor is using. We gather this information for each version of a plugin that is created and are able to collect some or all of the following: the vcs system (git, cvs, svn, other), the url to the repository, the name of the branch, and the name of the tag.

With this information we can provide useful information and links to the end user, and whilst presently we not not attempt any integration of monitoring perhaps in the future.

In regards to downloads we will still require users to upload thier plugin as a package, this allows us to ensure we will always have a valid download to historical versions. Contributors will be able to specify an optional alternative download location as well if they choose, however in order to record accurate statistics we will mediate the download link.

Reporting and notifications

With all the new functionality and information that is going to be put into this new system we are going to need better methods of monitoring what is going on and notify certain parties when things change or are introduced.

For just this purpose the new system will be created with several reports and a notifications subsystem to keep people up to date with what is happening in the system.

Concepts

The following are brief descriptions of the main concepts that will go into this new system.

Plugins

Plugins of course are the main focus for this new system which is being developed specifically to manage our collection of contributed plugins.

The most important thing to note is that while each contributed plugin will have a plugin record describing it in general, the actual plugin itself will be properly described by a plugin version.

Plugins can have a number of plugin versions, awards, and contributors. They will also belong to a category, and 0..n sets.

Plugin versions

Plugin versions relate to an actual version of a contributed plugin. Every time a contributor releases a new version of a plugin in the system they will need to log in and create a new plugin version. The plugin versions record in the database contains information about the specific version it was created for. Once created the uploaded version cannot be changed. The details about the version can still be edited but in order to ensure that awards and reviews are accurate the contributor will not be able to change the uploaded plugin. If they make changes they will need to create a new plugin version.

Plugin versions can have a number or reviews, awards, and supported versions. They always belong to a plugin.

Reviews

Reviews are an integral part of this new system. Each review is done on a plugin version (never a plugin in general) and consists of both a general review as well as a review of specified criteria such as security, usability etc.

Reviews can only be created by privileged users.

Awards

Awards will be created to show that a plugin, or plugin version has achieved something notable. They will only be able to be handed out by privileged users. Awards can be given to either a plugin, or a plugin version. In the future there may be automatic systems created to hand out rewards however at the moment it will be a purely manual task.

Sets

Sets will be used to organise plugins into special collections that can be presented through RSS or used within output widgets created for this new system. Sets cover ideas such as Featured plugins, Staff picks, Partner contributions, and any other matrix not covered by awards or calculation of information.

Database structure

The following is the proposed database structure for this new system.

Entity relationship diagram
local_contrib_plugin
Contributed plugins and/or code
id int (10) unsigned not null
categoryid int (10) unsigned not null Maps to local_contrib_category.id
name varchar (255) not null The name of the plugin
shortdescription smalltext A short plain text description of the module
description mediumtext A complete description of the plugin
descriptionformat int(2) unsigned default 0 not null
websiteurl varchar (255) default NULL A url to a site/page about this plugin
sourcecontrolurl varchar (255) default NULL A url to a system being used to manage the source for this plugin
faqs mediumtext Frequently asked questions for/about this plugin
faqsformat int(2) unsigned default 0 not null
documentation mediumtext Documenation for/about this plugin
documentationformat int(2) unsigned default 0 not null
documentationurl varchar (255) default NULL A url to further documentation about this plugin
timecreated int (10) unsigned Time the plugin was created within out system
timelastmodified int (10) unsigned Last time the plugin or any of its versions were modified
approved int(1) unsigned default 0 not null Flag to track when this module was approved
available int(1) unsigned default 0 not null Toggle availablility of this module easily.


local_contrib_vers
Tracks a version of a plugin
id int (10) unsigned not null
pluginid int (10) unsigned not null Maps to local_contrib_plugin.id
previousversionid int (10) unsigned default NULL Id of the version this version supersedes, maps to local_contrib_vers.id
version varchar(255) The version as a string
name varchar(255) default NULL The name of the version
status varchar(255) default NULL or int(10) unsigned default NULL The status of the version, e.g. stable, development TODO See talk page
releasenotes mediumtext Notes about this version of the plugin
releasenotesformat int(2) unsigned default 0 not null
websiteurl varchar (255) default NULL A url to a site/page about this plugin version
altdownloadurl varchar (255) default NULL An alternative download location for this plugin version
md5sum varchar (255) not null The md5 sum for the download
vcssystem varchar (255) default NULL The version control system used to manage this version (git, cvs, svn)
vcsrepositoryurl varchar (255) default NULL The url to the VCS used to manage this plugin version
vcsbranch varchar (255) default NULL The branch for this version within the VCS
vcstag varchar (255) default NULL The tag for this version within the VCS
timecreated int (10) unsigned Timestamp when this version was created
timelastmodified int (10) unsigned Timestamp when this version was last updated in some way
approved int(1) unsigned default 0 not null Set to 1 when the version has been approved and is available to the public
available int(1) unsigned default 1 not null When on this version is visible to the public


local_contrib_vers_download
Tracks version downloads either through file or link
id int (10) unsigned not null
versionid int (10) unsigned not null Maps to local_contrib_vers.id
timedownloaded int (10) unsigned not null Timestamp from when version was downloaded
userid int (10) unsigned default null Optional id of user who downloaded this plugin
downloadmethod varchar (255) not null default 'file' The means of download


local_contrib_category
Plugin categories
id int (10) unsigned not null
parentid int (10) unsigned not null Maps to local_contrib_category.id
name varchar (255) not null The name of this category
shortdescription smalltext A short plain text description of this category
description mediumtext A full description of this category.
descriptionformat int(2) unsigned default 0 not null


local_contrib_contributor
Tracks the contributors to a project so that no one gets forgotten.
id int (10) unsigned not null
userid int (10) unsigned not null Maps to user.id
pluginid int (10) unsigned not null Maps to local_contrib_plugin.id
creator int(1) unsigned default 0 If true the user is the creator of the plugin
type varchar (125) unsigned default 'active' Enum active, inactive, primary
timecreated int (10) unsigned not null Timestamp when this user became a contribtor
Unqiue key userid + pluginid


local_contrib_set
A set that plugins can belong to
id int (10) unsigned not null
name varchar(255) The name of the set
shortdescription smalltext A short description of the set
maxplugins int (10) unsigned default NULL The maximum number of plugins that can exist in this set.


local_contrib_set_plugin
Used to track which plugins are in which sets
id int (10) unsigned not null
setid int (10) unsigned not null
pluginid int (10) unsigned not null
userid int (10) unsigned not null The id of the user who added this plugin to the set
timeadded int (10) unsigned not null Timestamp when this plugin was added to the set
sort int (10) unsigned not null default 0 A sortable field
Unqiue key setid + pluginid


local_contrib_supported_vers
Contains the different Moodle versions that a plugin version can support
id int (10) unsigned not null
name varchar(255) Name of the software plus version
version decimal (12,5) not null Version number for sorting
timecreated int (10) unsigned not null Timestamp when this version was made available to support


contrib_supported_versions
Maps a supported Moodle version to a plugin
id int (10) unsigned not null
versionid int (10) unsigned not null Maps to local_contrib_vers.id
softwareversionid int (10) unsigned not null Maps to contrib_software_version.id


local_contrib_review
Used to record plugin version reviews
id int (10) unsigned not null
userid int (10) unsigned not null The user who published the review, maps to user.id
versionid int (10) unsigned not null The version of the plugin reviewed, maps to local_contrib_vers.id
softwareid int (10) unsigned not null The version of software used in this review, maps to contrib_software_version.id
generalreview mediumtext A general review about this plugin version
generalreviewformat int(2) unsigned default 0 not null
timereviewed int (10) unsigned not null The time the plugin version was reviewed


local_contrib_review_test
Contains the criteria a plugins versions should be reviewed on.
id int (10) unsigned not null
name varchar (255) The name of the review criteria
description mediumtext A description of the review criteria
descriptionformat int(2) unsigned default 0 not null
scaleid int (10) unsigned not null The scale to rate on, maps to scale.id


local_contrib_review_outcome
Contains the outcome and comments of a reviewed criteria
id int (10) unsigned not null
reviewid int (10) unsigned not null Maps to local_contrib_review.id
criteriaid int (10) unsigned not null Maps to local_contrib_review_test.id
review mediumtext The review of this criteria
reviewformat int(2) unsigned default 0 not null


local_contrib_awards
Contains the different awards that can be granted to a plugin and/or plugin version
id int (10) unsigned not null
name varchar (255) The name of the award
description mediumtext A description about this award
descriptionformat int(2) unsigned default 0 not null
timecreated int (10) unsigned not null Timestamp from when this award was created


local_contrib_plugin_awards
Tracks the awards a plugin, or plugin version have been awarded
id int (10) unsigned not null
awardid int (10) unsigned not null Maps to contrib_award.id
pluginid int (10) unsigned not null Maps to local_contrib_plugin.id
versionid int (10) unsigned default NULL Maps to local_contrib_vers.id
timeawarded int (10) unsigned not null Timestamp from when this was awarded
userid int (10) unsigned not null The user who awarded this award

File areas

There are several file areas that will be used within this new system.

Please not this relies on changes to the core system in order to be possible, presently the file API doesn't support local plugins.

Type Contextid Component Filearea Itemid
Plugin badge Image [200x200] CONTEXT_SYSTEM local_contrib plugin_badge Plugin id
Plugin icon Image [16x16] CONTEXT_SYSTEM local_contrib plugin_icon Plugin id
Plugin screenshots Images 0..n CONTEXT_SYSTEM local_contrib plugin_screenshots Plugin version id
Review images Images 0..n CONTEXT_SYSTEM local_contrib review_images Review id
Award image Image [200x200] CONTEXT_SYSTEM local_contrib award_image Award id
Award icon Image [16x16] CONTEXT_SYSTEM local_contrib award_icon Award id
Set icon Image [64x64] CONTEXT_SYSTEM local_contrib set_icon Set id

File system storage of plugins

The new system won't use the Moodle 2.0 file API to store plugin archives. Instead it will create a directory within the Moodle data directory, most likely contrib and will start storing archives in there. Archives will be both stored and extracted so that we can inspect the version.php and plugin in general for common problems (invalid plugin names, etc) and perhaps potentially inspect the language files associated with the plugin. On the filesystem files will first be organised by plugin id, then version id. The following examples should help clarify this:

  • moodledata/contrib/pluginid/versionid/archive.zip
  • moodledata/contrib/pluginid/versionid/pluging/plugincontents

In proposing this means of file storage please note that I am no expert on file systems and will be in the near future talking with those in the office who know more about it than me. Please if you have any ideas here share them on the talk page (don't forget to add your signature).

Capabilities

The following are the capabilities that this new local plugin will introduce and utilise.

Capability Type Description
local/contrib:view read View the system
local/contrib:viewunapproved read View unapproved plugins
local/contrib:createplugins write Create new plugins within the system
local/contrib:editownplugins write Edit plugins the user previously created
local/contrib:editanyplugin write Edit any plugins within the system
local/contrib:deleteownplugin write Delete a plugin you created
local/contrib:deleteanyplugin write Delete a plugin
local/contrib:deleteownpluginversion write Delete a plugin version you created
local/contrib:deleteanypluginversion write Delete a plugin version
local/contrib:approveplugin write Approve a plugin
local/contrib:approvepluginversion write Approve a plugin version
local/contrib:autoapproveplugins write New plugins created by this user are automatically approved
local/contrib:autoapprovepluginversions write New plugin versions created by this user are automatically approved
local/contrib:publishreviews write Create and publish a review
local/contrib:editownreview write Edit a review you created
local/contrib:editanyreview write Edit a review written by anyone
local/contrib:comment write Add a comment on a plugin, plugin version, or review
local/contrib:rate write Rate a plugin or plugin version
local/contrib:editowntags write A user can edit the tags associated with a plugin they manage.
local/contrib:editanytags write A user can edit the tags associated with any plugin.
local/contrib:managesupportableversions write Manage supportable versions
local/contrib:managesets write Manage sets
local/contrib:addtosets write Add plugins to sets
local/contrib:manageawards write Manage awards
local/contrib:handoutawards write Give plugins or plugin versions awards
local/contrib:managecategories write Create, edit, and delete categories

Ratings

There are three rateable areas within this new system:

Plugin versions 
Any user with the rate capability will be able to rate a plug-in version.
Review - general 
Reviewers will be able to give a plugin version a rating for the general section of the review.
Review - criteria 
Reviewers will be able to give each criteria reviewed for a plugin version its own rating.

The plugin versions ratings will be averaged to form an average rating for the plugin version.

The two review ratings will be averaged to form an average rating for the review as a whole.

Finally people have asked for plugins to be rateable, however this will not be so. Instead a plugins rating will come from it most recent version.

Comments

The are several areas within this new system where users with the comment capability will be able to comment, they are as follows:

Plugin 
Users will be able to comment on a plugins home page
Plugin version 
Users will be able to comment on specific plugin version.
Plugin faqs 
Users will be able to comment on the plugins FAQ page.
Plugin documentation 
Users will be able to comment on the plugins Docs page.
Reviews 
Users will be able to comment on specific reviews.

Of course the core comments API will be used to manage all comment areas.

Tags

There is only one area within this new system that users will be able to tag and that is plugins.

Any user with either the editowntags or editanytags capability will be able to manage the tags associated with a plugin.

Of course like ratings and comments the core tag API will be used for this.

Settings

The following are the settings that I anticipate the new system will have.

Setting Purpose
courseidforgroups Set to a courseid and user group pictures will be used from that course for users throughout the new system.
publishtorss If enabled then RSS feeds will be generated for a matrix of things including category listings, and plugin collections.

RSS feeds

If you enable RSS feeds for this new system the following feed will be published:

  • New plugins This RSS feed will contain just the new plugins within the system.
  • Recent updates This RSS feed will contain new plugins, new versions, and updated versions.
  • List of categories This RSS feed will contain a summary of each category in the system.
  • Category plugins list This RSS feed will contain the more recently updated plugins in the category.
  • Set list This RSS feed will contain the most recently updated plugins in the set.
  • Plugin information This RSS feed will contain information about a plugin and its versions.

Web services

The new system is going to expose several web services that will be designed for both generic, and core use.

Please note these web services will not likely be implemented for the initial release of the system, they are however worth mentioning as there are some great things that could be done with help from them.

Search plugins 
Search plugins and retrieve summary information about a plugin.
Get plugin details 
Fetch detailed information about a plugin.
Check plugin versions 
Given an array of plugin names and version numbers this service will check if there are any applicable updates and return information if there are.
Get plugin strings 
Fetch the strings used by a plugin (hopefully useful to AMOS).

Screen descriptions and mockups

Navigation screens

The following are the navigation screens for the system, these are screens that will be readily accessible to those who are not logged in or have little or no permissions other than view.

The frontpage of the contrib plugin
Viewing a plugin
Viewing a version of a plugin


  • Reading a review
  • Downloading a version of a plugin

Plugin administration screens

  • Creating a plugin
  • Creating a plugin version
  • Adding plugin information

System administration screens

  • General settings
  • Managing categories
  • Managing Moodle versions
  • Managing awards
  • Managing supportable versions

Reports

  • Plugin report
  • Plugin approval report
  • Plugin version approval report
  • Recent activity
  • Requiring attention
  • Review report

Required core changes

One of the goals for this new system was to get it using the core Moodle systems where possible and practical. Presently this proposal relies of several changes been made to core systems in order to get them into a usable state by this plugin. While I understand it is not desirable to have to make core changes I do think that in all cases making these changes would be of general benefit anyway. In all cases I am happy to be responsible to for making the required changes.

Changes to the ratings API

Presently the ratings system is VERY limited. When a rating is stored is associated by a context and an item id. The problem with this of course is that it immediatly limits ratings to be applied to only one structure per context. In the case of this new system which is currently planned as a local plugin it is running under the system context which immediatly makes the rating system unusable. As part of the development of this new system I would like to purpose the following changes at the same time:

  1. Add two new columns to the rating table in the database that are similar to fields found in the file table
    • component (string 255) This is the frankenstyle name of the component the rating relates to e.g. mod_forum, local_contrib
    • ratingarea (string 255) This is a unique name that the utilisine component can use to allow several areas within a component to be rated e.g. for the forum you may have post, and discussion.
  2. Extend the API to require the two new fields but provide backwards compatability to working without them.
  3. Review all locations using the rating system presently and refactor to make use of these two new fields.

Post release: Automatically install plugins from within your Moodle site

As I'm sure some of those reading this proposal have realised this new system is geared up in such as way that we will be getting very close to being able to create Moodle functionality to search for, install, and update your modules from within your Moodle site, from this new system.

Presently this is a blue skies proposal but hopefully it will in gain momentum in the future and we will see something happen here.

More information