Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

Portfolio API: Difference between revisions

From MoodleDocs
Line 48: Line 48:
===Exporting===
===Exporting===


*User is viewing a page that calls portfolio_add_button().  This checks to see if there are any configured portfolio plugin instances, and also (maybe) any permissions related to portfolios, and what the user's portfolio settings are, and then displays either a single 'add to portfolio' button, or a drop down menu of the available systems with the add button.
*User is viewing a page that calls new portfolio_add_button().  This checks to see if there are any configured portfolio plugin instances, and also (maybe) any permissions related to portfolios, and what the user's portfolio settings are, and then displays either a single 'add to portfolio' button, or a drop down menu of the available systems with the add button.
*When this button is pressed, the user is redirected to portfolio/add.php, with some post data containing the responsible area (activity module or something like course or blog) callback file and function.
*When this button is pressed, the user is redirected to portfolio/add.php, with some post data containing the responsible area (activity module or something like course or blog) callback file and callback arguments, as well as optionally some information about what type of content it is.
*On this page, the user is presented with a form to enter metadata about the item, and configure any options. At this point if there are multiple formats available for export (based on the intersection of what the plugin and module support), the user can select which format they want.  The plugin and module can both export mform elements for this page.  The user can at this point also select to send the data and wait (with a warning it might take awhile), or queue it for processing if it's larger.
*On this page, the user is presented with a form to enter metadata about the item, and configure any options. At this point if there are multiple formats available for export (based on the intersection of what the plugin and module support), the user can select which format they want.  The plugin and module can both export mform elements for this page.  The user can at this point also select to send the data and wait (with a warning it might take awhile), or queue it for processing if it's larger.  This is determined by the size of the content to be exported.
*When the user has submitted the form, they are displayed a summary of what they're about to export, with 'confirm' and 'edit' buttons.  Edit just relaunches the form, and confirm goes to the next step.
*When the user has submitted the form, they are displayed a summary of what they're about to export, with 'confirm' and 'cancel' buttons.  Cancel cancels the request, and cleans up any temporary data, and returns the user to where they came from, while confirm goes to the next step.
*At this point, the portfolio plugin (or transport plugin) might need to take control for a step. For example, facebook or flickr might require the user to log in for the first time and confirm moodle is allowed to access their API
*At any point, the portfolio plugin might need to take control for a step. For example, facebook or flickr might require the user to log in for the first time and confirm moodle is allowed to access their API.
*When the user has confirmed their summary, a 'portfolio_send' event will be triggered.  At this point, one of two things happen.   
*When the user has confirmed their summary, a 'portfolio_send' event will be triggered.  At this point, one of two things happen.   
# If the user has elected to wait, the 'instant' event is fired, and when the caller gets control again, it displays the status to the user.
# If the user has elected to wait, the 'instant' event is fired, and when the caller gets control again, it displays the status to the user.
# If the user has elected to queue, the delayed event is fired and the user is notified.
# If the user has elected to queue, the delayed event is fired and the user is notified.
*The user is given the option to continue to their portfolio, or return to where they were
*The user is given the option to continue to their portfolio, or return to where they were
*
*When the event is handled (either through the cron or instant event), the following happens:
*When the event is handled (either through the cron or instant event), the following happens:
*The event handler is invoked. This is almost certainly going to be a function in lib/portfoliolib.php rather than any handler in the portfolio plugins.  The event handler in turn calls the original callback that was passed to portfolio/add.php (module//lib.php or course/lib.php or something), which prepares the data in the format the user selected and returns control to the event handler.
*The event handler is invoked. This reawakens the transfer and defers control to the caller and then the portfolio to prepare and send the package.
*At this point we have all the metadata we need, and we have the content to send, so it's time to start transport negotiation, which happens in stages. Note that some plugins might not implement all stages, and 3 and 4 might be interchangeable in order, depending on the external system.
*When this is complete, we return control to the event handler (which, if it's an 'instant' one, will return true to the caller.
#request
#authentication
#content
#metadata (some control is passed to the portfolio plugin for packaging - either to just write out an xml file and zip up the whole package, or send a series of requests)
#completion
*When this is complete, we return control to the event handler (which, if it's an 'instant' one, will return true to the caller, which will be the ajax request)


===Storage===
===Storage===

Revision as of 15:22, 11 September 2008

Moodle 2.0

This page describes the specification for a future feature, currently being worked on for Moodle 2.0. This spec is STILL UNDER CONSTRUCTION.

Overview

The Portfolio API is a core set of interfaces that all Moodle code will/should use so that we can easily publish files to all kinds of external document repository systems.

It's important to remember that portfolios are generally treated as WRITE-ONLY. All we are doing in Moodle is grabbing stuff and pushing it out to somewhere. Management of the files and further combining/reflecting is done through the native interface provided by the portfolio system. Reading of files from a repository is handled by the Repository API.

A typical user story:

  1. When portfolios are enabled, every page or major piece of content in Moodle has a little "Save" button beside it.
  2. User clicks one of these buttons
  3. User is able to choose from a list of configured portfolios (this step will be skipped if there's only one).
  4. User may be asked to define the format of the captured content (eg pdf, IMS LD, HTML, XML ...)
  5. User may be asked to define some metadata to go with the captured content (some will be generated automatically).
  6. The content and metadata is COPIED to the external portfolio system
  7. User has an option to "Return to the page you left" or "Visit their portfolio".

Note this will be just as useful for teachers as for students.

The formatting possibilities will vary depending on the context of the button and the type of external portfolios. So for example, the "Save" button on the course page would allow the user to capture the whole course in IMS LD or Moodle backup format, which you would not have on a forum page.

Architecture

Here is how it will work:

Plugins and libraries

There will be one type of plugins

  1. Portfolio (eg Mahara/Elgg/OSP/Facebook/Download) - this will be portfolio/type/xxx

The transport layer (eg mnet/http/scp/cp/dav etc) or clients (eg boxnet/flickr) will be written as libraries, to be shared by both repository and portfolio.

Then there will be different formats that plugins will support (and the part of moodle exporting content must support as well), eg IMS, moodle native, mahara native, pdf, encrypted pdf. These will have good libraries supporting them.

Admin

It is important to be allowed to have multiple instances of (some) plugins. The workflow for adding a new one is:

  • Admin navigates to portfolio config
  • Selects from the list of available portfolio plugins, and clicks 'add a new external portfolio' (some may be disabled if there is an instance already and the plugin doesn't support multiple instances)
  • Configure the plugin - select which transport and content types to use if there are multiple supported and installed, urls, authentication keys, etc.
  • Set permissions (maybe) - handled by roles.

This is not necessary for every type of portfolio, because many will just require the user to authenticate directly and if we do ever want to retain settings for each user we just use user preferences.

Exporting

  • User is viewing a page that calls new portfolio_add_button(). This checks to see if there are any configured portfolio plugin instances, and also (maybe) any permissions related to portfolios, and what the user's portfolio settings are, and then displays either a single 'add to portfolio' button, or a drop down menu of the available systems with the add button.
  • When this button is pressed, the user is redirected to portfolio/add.php, with some post data containing the responsible area (activity module or something like course or blog) callback file and callback arguments, as well as optionally some information about what type of content it is.
  • On this page, the user is presented with a form to enter metadata about the item, and configure any options. At this point if there are multiple formats available for export (based on the intersection of what the plugin and module support), the user can select which format they want. The plugin and module can both export mform elements for this page. The user can at this point also select to send the data and wait (with a warning it might take awhile), or queue it for processing if it's larger. This is determined by the size of the content to be exported.
  • When the user has submitted the form, they are displayed a summary of what they're about to export, with 'confirm' and 'cancel' buttons. Cancel cancels the request, and cleans up any temporary data, and returns the user to where they came from, while confirm goes to the next step.
  • At any point, the portfolio plugin might need to take control for a step. For example, facebook or flickr might require the user to log in for the first time and confirm moodle is allowed to access their API.
  • When the user has confirmed their summary, a 'portfolio_send' event will be triggered. At this point, one of two things happen.
  1. If the user has elected to wait, the 'instant' event is fired, and when the caller gets control again, it displays the status to the user.
  2. If the user has elected to queue, the delayed event is fired and the user is notified.
  • The user is given the option to continue to their portfolio, or return to where they were
  • When the event is handled (either through the cron or instant event), the following happens:
  • The event handler is invoked. This reawakens the transfer and defers control to the caller and then the portfolio to prepare and send the package.
  • When this is complete, we return control to the event handler (which, if it's an 'instant' one, will return true to the caller.

Storage

Obviously during this process, state is going to be lost between webserver requests and also between user input and event handling. Some data will be stored in the user's session (the result of the metadata form, for example) and some in temporary files on the file system. These will be stored in dataroot/temp/portfolio/$userid-$timestamp/

Access/Permissions

  • Portfolio plugin must be allocatable per context (site/course/group etc)
  • How can we assume all users will have an account on the remote system? We could try and silently authenticate them (or at least see if it's possible to) in the background at the first call to portfolio_add_button (and then cache the result in the session)
  • Check MNet policy - it assumes everyone with the role can have an account (assuming they_sso_in or whatever it's called is on) - this doesn't scale though, other transport mechanisms and or plugin types might not follow this assumption. I think this really must be defined by the portfolio plugin only, not the transport plugin.

Event API

It is probably worth using the event api to handle the sending of data, mostly just because it provides a handy way of detached processing. However, any portfolio_send event will be 'owned' by particular "instances" of plugins - and each plugin that subscribes to portfolio_send event (all of them by definition) will have to check if the event data belongs to any of their instances, which is potentially quite messy.



Technical

Abstract Portfolio Baseclass: portfolio_plugin_base

Mixes providing some basic functionality by means of its own functions, with a number of abstract functions plugins must implement, and with some functions that plugins can also optionally override.

See also: Writing_a_Portfolio_Plugin for a full list of all methods you must/can/shouldn't override, as well as associated instructions for what else you need to do to create a new portfolio plugin

Abstract Caller Baseclass : portfolio_caller_base

Whenever somewhere in Moodle wants an 'add to portfolio' button, they must subclass this.

See also: Adding_a_Portfolio_Button_to_a_page for a full list of all methods you must/can/shouldn't override as well as the associated instructions for how to call portfolio_add_button.

Database Tables

The actual information about plugins that are installed is just stored in mdl_config.

Additionally, as we're configuring instances of plugins, rather than just one config set per plugin, we're not using mdl_config_plugin, but instead our own set of tables:

portfolio_instance:

Field Datatype Comment
id integer sequence
plugin varchar(50) name of plugin (should match directory in portfolio/type)
name varchar(255) name of this plugin instance
visible smallint 0 or 1


portfolio_instance_config:

Field Datatype Comment
id integer sequence
instance integer (pseudo)fk to portfolio_instance
name varchar(255) config name
value text config value


portfolio_instance_user:

Field Datatype Comment
id integer sequence
instance integer (pseudo)fk to portfolio_instance
userid integer (pseudo)fk to mdl_user
name varchar(255) config name
value text config value

portfolio_log:

Field Datatype Comment
id integer sequence
userid integer (pseudo) fk to mdl_user
time integer unix timestamp of transfer
portfolio integer (pseudo) fk to mdl_portfolio_instance
caller_class varchar(150) name of caller class (used in the case of duplicates to display information)
caller_file varchar(255) file that contains the definition of caller_class
caller_sha1 varchar(255) sha1 information of export

portfolio_tempdata

Field Datatype Comment
id integer sequence
data text serialized representation of export data
expirytime integer time this data (and the transfer) expires (and the record (and associated files)) will be deleted

All plugins can also implement their own database tables as needed, by creating a db/install.xml and db/upgrade.php inside portfolio/type/xxx/ (See Writing_a_Portfolio_Plugin) for more information.

Portfolio Plugins

  1. mahara (will be done for the initial implementation)
  2. download (should be done for the initial implementation)

transport types and formats should be able to be found in a shared location for multiple plugins of both portfolio and repository type to use, but also might be specific to one type of plugin which means that moodle should support looking in multiple locations for these plugins. (eg mahara native format will be in the mahara portfolio plugin, but pdf format will be in a shared library)

Possible Transport Types

  1. mnet (will be done for the initial implementation as part of the Mahara Portfolio Plugin)
  2. download (will be done for the initial implementation as part of the Download Portfolio Plugin)
  3. http
  4. filesystem (could be local/nfs/samba whatever) (cp)
  5. ssh based (eg scp - should find and re-use the elgg block code as it deals with using ssh keys nicely)
  6. webdav
  7. open social? (http://code.google.com/apis/opensocial/)

Possible Export Formats

(Note that we don't necessarily want more than one of these for the initial implementation)

  1. html
  2. pdf
  3. encrypted pdf
  4. ims?
  5. leap/piop? (http://wiki.cetis.ac.uk/LEAP_2.0) -- looking increasingly possible Mahara will speak this, so hopefully for the intial implementation
  6. moodle native?
  7. mahara native?
  8. Dublin Core (Enovation implemented this)

Testing

Nico to help (see Development Approach for TDD-related approach)

MNET

This section has moved to MNET_Roadmap

Duplication of Data

Moodle will keep track of what content it transfers and when. It'll remember the url of the original export location (alphabetisising the url parameters and removing the sessionkey) and a sha1 hash of the data, so that if the user tries to export the same content, Moodle can warn the user (along with a notification of whether it has changed or not).

At this point we can build support into the API for plugins to offer to differentiate between 'replace' and 'add', but won't implement it for any we write for the initial implementation.

Assumptions

  • Using Moodle's built in Event API

Save points in Moodle

moved to http://tracker.moodle.org/browse/MDL-15758 during development

Outstanding questions

  • Dependance of transport plugins - eg filesystem might depend on http for sending request/finished pings

Development Approach

  • create two base classes
  • create mahara portfolio plugin, relying on mnet
  • create portfolio/add.php and portfolio/lib.php
  • start adding calls to the portfolio_add_button function to various places in moodle, with their callbacks.
  • create a dummy listener in mahara until the mahara side is specified

After a conversation with Nico about testing, we decided the most sensible ordering to start with would be:

  1. Write portfolio/add.php
  2. Write *skeleton functions* to support it in portfolio/lib.php and the class structure
  3. Write tests for those methods that are necessary
  4. Implement the functions to make the tests pass
  5. Keep adding module implementations

See also