Repository API

This page describes the specification for a future feature, currently being worked on for Moodle 2.0. This spec is STILL UNDER CONSTRUCTION.

See MDL-13766 to track the status of the implementation.

The page is open for everyone so everyone can help correct mistakes and help with the evolution of this document. However, if you have questions to ask, problems to report or major changes to suggest, please add them to the page comments, or start a discussion in the Repositories forum. We'll endeavour to merge all such suggestions into the main spec before we start development.

Note that parts of this document have been now split off into a separate File_API

Objectives

1. Allow all Moodle users to easily bring content into Moodle from external repositories
2. Provide a consistent interface to any external repository, for any Moodle module

Use cases

Teacher adding an external file as a new resource

1. Teacher wants to add a new resource to a course
2. Teacher clicks the "Choose a resource" button
3. Teacher is presented with a simple file picker to choose a file (with a menu to switch between multiple configured repositories)
4. Teacher chooses a file in an external repository
5. File is COPIED into Moodle and stored by the resource module
6. File is marked as owned by that user
7. Whenever someone wants to view that file, the resource module controls access (see File API )

Teacher linking to an external file as a new resource (think video repository)

1. Teacher wants to display a file in the repository
2. Teacher clicks the "Choose a resource" button
3. Teacher is presented with a simple file picker to choose a file (with a menu to switch between multiple configured repositories)
4. Teacher chooses a file in an external repository
5. Link to the file is COPIED into Moodle and stored by the resource module
6. Link is marked as owned by that user
7. Whenever someone wants to follow that link, the resource module controls access (see File API )

Student submitting an assignment

1. Student needs to submit an assignment and presses the "Choose files" button
2. Student sees a "file picker" where they can see files listed on any of several configured repositories (file picker login, file picker browser, file picker search)
3. Student chooses MySpace from the list
4. Student is prompted to enter MySpace username/password (if admin allows it, a checkbox could be there to "remember this for next time" but remember security)
5. Student sees their files in MySpace and chooses one or more
6. Files are copied from MySpace to Moodle
7. Assignment module controls the permissions so that only the Student and assignment graders can see the file (other students would not have permission).

Student attaching an image to a forum

1. Student needs to attach an image and presses the "Choose files" button in the posting screen
2. Student sees a "file picker" where they can see files listed on any of several configured repositories
3. Student chooses Mahara from the list
4. Student is prompted to enter Mahara username/password
5. Student sees their files in Mahara and chooses one image
6. Image is copied to Moodle
7. Image file is attached to forum post by Forum module (by reference)
8. Forum module controls permissions so that anyone who can read that forum can see that file

Student attaching the same image in another forum

1. Student needs to submit an assignment and presses the "Choose files" button
2. Student sees a "file picker" where they can see files listed on any of several configured repositories
3. Student chooses "Local files" from the list and sees all the files they've uploaded before
4. A COPY of the image file is attached to forum post by Forum module
5. Forum module controls access to this file.

Please add more use cases in this same format

Mock screenshots

When you first call up the file picker and choose a repository, you might be asked to log in (if saving of passwords is not allowed):

Browsing files could look something like this:

And you can also search:

General architecture

Each repository plugin (a standard Moodle plugin stored under /repository/xxx) will subclass the standard API and override methods specific to that repository.

As is usual in Moodle, there will be admin settings to disable/enable certain repository plugins as standard, as well as user settings so that users can add their own personal repositories to the standard list (eg Yahoo Briefcase or Google Docs) and to select their default repository.

Once a repository has been used the file will usually be copied into Moodle there and then. However there will also be options to:

• only return the URL to the file if it's desired to keep it external (but this does present security and integrity risks), or
• refresh the local file copy regularly and automatically
• refresh the file manually if desired

Once in Moodle, it is subject to the File API for access control like any other file.

Repository requirements

From the Moodle point of view, each repository is just a hierarchy of nodes.

The repository MUST provide:

1. A URI to download each node (eg file).
2. A list of the nodes (eg files and directories) under a given node (eg directory). This allows Moodle to construct a standard browse interface (much like a standard OS file picker). However some repository plugins may choose to completely override the repository_browse() method and implement their own interface, that's OK, as long as they end up with a URL for the file.

The repository can OPTIONALLY:

1. Require some authentication credentials
2. Provide more metadata about each node (mime type, size, dates, related files, dublin core stuff, etc)
3. Describe a search facility (so that Moodle can construct a search form)
4. Provide copyright and usage rules (or just information about the rules)

Repository plugins

Some plugins I'd like to see developed for the first version are:

• local - very similar to the current course-based file manager, except user-based
• moodle - an interface to another Moodle site, accessed over a secure mnet connection
• jsr170 - an interface that can talk to anything that supports jsr170 (eg Alfresco)
• oki - an OKI emulator allowing us to access things with OKI interfaces,like Fedora
• briefcase - an interface to Yahoo Briefcase
• myspace - an interface to MySpace files (perhaps via this MySpace API)
• googledocs - an interface to Google Docs
• s3 - an interface to Amazon S3
• skydrive - an interface to Microsoft's SkyDrive files
• box - an interface to box.net
• facebook - an interface to Facebook files
• merlot - an interface to the learning materials in Merlot.org
• flickr - an interface to flickr
• youtube - an interface to YouTube
• mahara - an interface to a Mahara installation
• Dspace - a repository from MIT
• DOOR - another popular open source repository
• SMB shares - An interface for windows shares e.g. personal folders on network drives. Would need to link with LDAP as usernames will often be wholly/partially the same as network folder names. This could be done using SAMBA, but would also need to work on windows machines natively. See this block for a linux implementation.
• WebDAV - to access arbitrary external WebDAV servers

Tables

repository

This table contains one entry for every configured external repository instance.

Field	Type	Default	Info
id	int(10)		autoincrementing
repositoryname	varchar 255		A custom name for this repository (non-unique)
repositorytype	varchar(255)		The name of the plugin being used (1 special case: "local" which covers all files uploaded from desktop)
userid	int(10)		The person who created this repository instance
contextid	int(10)		The context that this repository is available to ( = system context for site-wide ones)
username	varchar(255)		username to log in with, if required (almost never!)
password	varchar(255)		password to log in with, if required (almost never!)
data1	text		Other information useful to the plugin
data2	text		Other information useful to the plugin
data3	text		Other information useful to the plugin
data4	text		Other information useful to the plugin
data5	text		Other information useful to the plugin
timecreated	int(10)		The time this repository was created
timemodified	int(10)		The last time the repository was modified

File types

The context at which someone is inserting a file may require certain file types (eg uploading a new user profile image is only looking for images).

To support this, the calling code needs to be able to specify the required mimetypes, and the listing code should be able to filter the results based on these mimetypes. Ideally the repository itself can do the filtering for ultimate speed (though not all repositories will support this).

We will have to develop special new mimetypes for Moodle files like backups (application/vnd.moodle.backup) and IMS learning design (application/vnd.moodle.imsld) etc

Technical walkthrough

There are two cases where the repository API will be used: as part of a Moodleform to add a file and as part of the HTML editor to add a media element into some HTML).

Case 1: As part of a Moodleform with Javascript

1. Moodle code must pre-create a context. We propose the notion of "draft" contexts for cases like adding an activity where we don't know if the activity will actually be added.

2. Moodle module code calls a "fileopen" moodleform item whenever a file is required, which includes the following information to pass to the File API:

$mform->addElement('fileopen' ...)

a) the filearea reference that the file will end up in (See File API)
b) the current Moodle context

3. When rendering the form, Moodle will display a read-only filename field with an "add file" button next to it. There will also be a hidden field to store a file reference later (this is what actually gets used, the filename field is just for users).

4. When the add file button is pressed, the form will be "replaced" in the page by a larger resizeable area containing an AJAX file picker. (After picking the display will shrink back again to what it was before). (There could be a user option to make this a popup window instead, if required)

4. The AJAX file picker interface will list all the active repositories as a menu, and list files in one of several formats (like Windows/Mac/Linux): Details, Names, Icons.

5. For each plugin, the AJAX interface will prompt the user to login first (if required) asking the plugin to log in behind the scenes. It'll also ask the plugin to return listing data in response to clicks and searches.

6. Finally, when the user selects a file and clicks the "Select" button, the AJAX interface will trigger a method in the plugin that will fetch the file and call the File API to store the file using the filearea and context information we already had. While this is happening, the interface should show some sort of progress bar (ideally) or at least a "loading file" image/sign/message.

7. After a file was selected the picker can shrink down and leave our read-only filename field renaming (with the hidden field containing a file reference).

8. Submitting the form will trigger the mform processing for this field, which should call the File API to mark the context as "Not draft" any more.

9. Cron jobs in file api should automatically delete any files attached to draft contexts that are older than 24 hours or move them into a trash can in the user's file area (perhaps).

Case 2: As part of a Moodleform without Javascript

Coming soon

Case 3: As part of a HTML editor

1. The moodleform for a textarea (HTML editor) will require the same parameters as for a native moodleform file element:

a) the filearea reference that the file will end up in (See File API)
b) the current Moodle context

2. There are passed to the editor similar to how we used to pass courseid (the current editors in HEAD do this).

3. The editor passes on this information to all plugins (perhaps they are javascript variables?)

4. When adding an image or other media element, HTML editor always require a URL (with perhaps other info like height and width etc) in a modal dialog. We can add the same AJAX repository picker to these dialogs (as a tab or a popup etc) to allow people to pick from any repository and choose files to download. The repository picker is responsible for downloading the file in real-time and returning a URL back to the dialog text input before closing.

Repository plugins

Each repository plugin is required to contain the following elements:

class repository()

This class implements the interface to a particular repository, for browsing, selecting and updating files. The base class (repository) is defined in /repository/lib.php, while each repository defines an inherited class (eg repository_alfresco) in /repository/repositoryname/repository.class.php

Repositories can redefine any of these methods as required (and in some instances, MUST redefine them):

get_file($path)

Given a URL, get a file from there.

get_listing($parent='/', $search=''')

Given a path, and perhaps a search, get a listing of files.

print_login()

Show the login screen, if required

print_listing

Given a listing from get_listing, print it.

print_search

Print the search form.

store_login($username, $password, $userid=''')

If you do want to cache login details for various repositories, then use this method.

cron()

Defines operations that happen occasionally on cron.

icon.png

A logo that represents the repository. Ideally square but we should handle all sizes.

See also

• Repository Administration Specification
• Repository Interface for Moodle/Course/User
• Repository Plugins
• Repository File Picker
• File API
• Portfolio API