Difference between revisions of "File Converters"

Jump to: navigation, search
(Add docs for file_conversion API)
 
Line 2: Line 2:
  
 
Users submit a wide range of files, and it is a common requirement to convert these to alternative formats.
 
Users submit a wide range of files, and it is a common requirement to convert these to alternative formats.
The biggest example of where this currently happens in core is in the assignfeedback_editpdf plugin which allows for conversion from a variety of document types into PDF to facilitate annotation.
+
The biggest example of where this currently happens in core is in the <tt>assignfeedback_editpdf</tt> plugin which allows for conversion from a variety of document types into PDF to facilitate annotation.
  
 
The file converters distributed with Moodle currently are:
 
The file converters distributed with Moodle currently are:
- Unoconv
+
* Unoconv
- Google Drive
+
* Google Drive
  
 
The file converter API deals allows for conversion via multiple plugins and will automatically fallback to another suitable plugin upon failure.
 
The file converter API deals allows for conversion via multiple plugins and will automatically fallback to another suitable plugin upon failure.
Line 16: Line 16:
 
A file conversion is performed by the <tt>core_files\converter</tt> API and a single conversion is represented by the <tt>core\files\conversion</tt> class.
 
A file conversion is performed by the <tt>core_files\converter</tt> API and a single conversion is represented by the <tt>core\files\conversion</tt> class.
  
A file conversion can be started by calling the <tt>start_conversion</tt> function on the converter API and passing in an existing <tt>stored_file</tt> record, along with the target format.
+
Individual file conversions should always be accessed by the <tt>core_files\converter</tt> API.
 +
 
 +
A file conversion is fetched or created by calling the <tt>start_conversion</tt> function on the converter API and passing in an existing <tt>stored_file</tt> record, along with the target format.
  
 
  <nowiki>
 
  <nowiki>
 
$converter = new \core_files\converter();
 
$converter = new \core_files\converter();
$conversion = $converter->start_conversion($file, 'pdf');
+
$conversion = $converter->start_conversion($file, 'pdf');</nowiki>
</nowiki>
 
  
 
If an existing file conversion matching the file and target format exists, the conversion record for this file will be returned, otherwise a new conversion is created and returned.
 
If an existing file conversion matching the file and target format exists, the conversion record for this file will be returned, otherwise a new conversion is created and returned.
 +
 
To force a fresh conversion, a third boolean parameter can be passed, though this should not normally be necessary.
 
To force a fresh conversion, a third boolean parameter can be passed, though this should not normally be necessary.
  
 
  <nowiki>
 
  <nowiki>
 
$converter = new \core_files\converter();
 
$converter = new \core_files\converter();
$conversion = $converter->start_conversion($file, 'pdf', true);
+
$conversion = $converter->start_conversion($file, 'pdf', true);</nowiki>
</nowiki>
 
  
 
==Polling for updates on an existing conversion==
 
==Polling for updates on an existing conversion==
Since the <tt>start_conversion</tt> function both kicks off a new conversion, and fetches an existing one, there is no need to explicitly poll the status of a conversion, however this is still possible.
+
When the <tt>start_conversion</tt> function is called, it automatically polls for any update on the conversion so it should not normally be necessary to poll the status separately.
 +
 
 +
It is however possible to do so:
 
  <nowiki>
 
  <nowiki>
    $converter = new \core_files\converter();
+
$converter = new \core_files\converter();
    $conversion = $converter->start_conversion($file, 'pdf');
+
$conversion = $converter->start_conversion($file, 'pdf');
    $converter->poll_conversion($conversion);
+
$converter->poll_conversion($conversion);</nowiki>
</nowiki>
 
  
 
==Checking status of a conversion==
 
==Checking status of a conversion==
Line 46: Line 48:
 
* <tt>STATUS_FAILED</tt> - All attempts to convert the file have failed.
 
* <tt>STATUS_FAILED</tt> - All attempts to convert the file have failed.
  
The conversion API provides a way to check the status of the conversion:
+
The conversion API provides a way to check the status of the conversion with the <tt>$conversion->get_status()</tt> function:
  
 
  <nowiki>
 
  <nowiki>
Line 60: Line 62:
 
         case \core_files\conversion::STATUS_FAILED:
 
         case \core_files\conversion::STATUS_FAILED:
 
             // Permanent failure - handle to the user.
 
             // Permanent failure - handle to the user.
     }
+
     }</nowiki>
</nowiki>
 
  
 
==Fetching the target file==
 
==Fetching the target file==
Line 69: Line 70:
 
if ($conversion->get_status() === \core_files\conversion::STATUS_COMPLETE) {
 
if ($conversion->get_status() === \core_files\conversion::STATUS_COMPLETE) {
 
     $file = $conversion->get_destfile();
 
     $file = $conversion->get_destfile();
}
+
}</nowiki>
</nowiki>
 
  
 
=Developing file conversion plugins=
 
=Developing file conversion plugins=
Line 81: Line 81:
  
 
The following notes apply:
 
The following notes apply:
* <tt>are_requirements_met</tt> should return  a status of whether the ''system'' requirements are met. For example, whether appropriate API keys are present.
+
* <tt>are_requirements_met</tt> should return  a status of whether the ''system'' requirements are met. For example, whether appropriate API keys are present. It should be lightweight to call and cache where required.
It should be lightweight to call and cache where possible.
 
 
* <tt>start_document_conversion</tt> should kick off the conversion, whilst <tt>poll_conversion_status</tt> should poll for any status update:
 
* <tt>start_document_conversion</tt> should kick off the conversion, whilst <tt>poll_conversion_status</tt> should poll for any status update:
 
** If any failures occur, it should set the conversion status to <tt>conversion::STATUS_FAILED</tt> and immediately return.  There is no need to update the <tt>$conversion</tt> record in this situation.
 
** If any failures occur, it should set the conversion status to <tt>conversion::STATUS_FAILED</tt> and immediately return.  There is no need to update the <tt>$conversion</tt> record in this situation.
** When the conversion process starts, the status should be set to
+
** When the conversion process starts, the status should be set to <tt>conversion::STATUS_IN_PROGRESS</tt> and the record '''must''' be updated. This ensures that, should the process take a long time, the current status is accurately reflected.
<tt>conversion::STATUS_IN_PROGRESS</tt> and the record '''must''' be updated.
 
This ensures that, should the process take a long time, the current status is accurately reflected.
 
 
** Upon successful completion, the status should be updated to <tt>conversion::STATUS_COMPLETE</tt> and the newly created <tt>stored_file</tt> should be stored against the conversion using either the <tt>store_destfile_from_string</tt> or <tt>store_destfile_from_path</tt> function as appropriate.
 
** Upon successful completion, the status should be updated to <tt>conversion::STATUS_COMPLETE</tt> and the newly created <tt>stored_file</tt> should be stored against the conversion using either the <tt>store_destfile_from_string</tt> or <tt>store_destfile_from_path</tt> function as appropriate.
 
* The <tt>supports</tt> function should return whether the supported source and target conversion is possible.
 
* The <tt>supports</tt> function should return whether the supported source and target conversion is possible.
 
* The <tt>get_supported_conversion</tt> function is used for informational display to administrators only.
 
* The <tt>get_supported_conversion</tt> function is used for informational display to administrators only.

Revision as of 04:14, 27 March 2017

Introduction

Users submit a wide range of files, and it is a common requirement to convert these to alternative formats. The biggest example of where this currently happens in core is in the assignfeedback_editpdf plugin which allows for conversion from a variety of document types into PDF to facilitate annotation.

The file converters distributed with Moodle currently are:

  • Unoconv
  • Google Drive

The file converter API deals allows for conversion via multiple plugins and will automatically fallback to another suitable plugin upon failure.

The API is designed to be called asynchronously as many cloud services offering document conversion offer an asynchronous API themselves.

Using the file converter API

A file conversion is performed by the core_files\converter API and a single conversion is represented by the core\files\conversion class.

Individual file conversions should always be accessed by the core_files\converter API.

A file conversion is fetched or created by calling the start_conversion function on the converter API and passing in an existing stored_file record, along with the target format.

$converter = new \core_files\converter();
$conversion = $converter->start_conversion($file, 'pdf');

If an existing file conversion matching the file and target format exists, the conversion record for this file will be returned, otherwise a new conversion is created and returned.

To force a fresh conversion, a third boolean parameter can be passed, though this should not normally be necessary.

$converter = new \core_files\converter();
$conversion = $converter->start_conversion($file, 'pdf', true);

Polling for updates on an existing conversion

When the start_conversion function is called, it automatically polls for any update on the conversion so it should not normally be necessary to poll the status separately.

It is however possible to do so:

$converter = new \core_files\converter();
$conversion = $converter->start_conversion($file, 'pdf');
$converter->poll_conversion($conversion);

Checking status of a conversion

File conversions can have one of four states:

  • STATUS_PENDING - The conversion has not yet started;
  • STATUS_IN_PROGRESS - A conversion has been picked up by a file converter and is currently in progress;
  • STATUS_COMPLETE - The conversion was successful and the converted file is available; and
  • STATUS_FAILED - All attempts to convert the file have failed.

The conversion API provides a way to check the status of the conversion with the $conversion->get_status() function:

    $converter = new \core_files\converter();
    $conversion = $converter->start_conversion($file, 'pdf');

    switch ($conversion->get_status()) {
        case \core_files\conversion::STATUS_COMPLETE:
            // The file is ready. Do something with it.
        case \core_files\conversion::STATUS_PENDING:
        case \core_files\conversion::STATUS_IN_PROGRESS:
            // Conversion still ongoing. Display spinner to the user.
        case \core_files\conversion::STATUS_FAILED:
            // Permanent failure - handle to the user.
    }

Fetching the target file

Following a conversion, the target file is stored as a stored_file record and can be fetched for consumption elsewhere in your API:

if ($conversion->get_status() === \core_files\conversion::STATUS_COMPLETE) {
    $file = $conversion->get_destfile();
}

Developing file conversion plugins

Plugin structure

File conversion plugins are located in the files/converter directory and follow the standard structure of a Moodle Plugin.

They must provide a converter plugin which implements the \core_files\converter_interface interface and be named as fileconverter_PLUGINNAME\converter.

The following notes apply:

  • are_requirements_met should return a status of whether the system requirements are met. For example, whether appropriate API keys are present. It should be lightweight to call and cache where required.
  • start_document_conversion should kick off the conversion, whilst poll_conversion_status should poll for any status update:
    • If any failures occur, it should set the conversion status to conversion::STATUS_FAILED and immediately return. There is no need to update the $conversion record in this situation.
    • When the conversion process starts, the status should be set to conversion::STATUS_IN_PROGRESS and the record must be updated. This ensures that, should the process take a long time, the current status is accurately reflected.
    • Upon successful completion, the status should be updated to conversion::STATUS_COMPLETE and the newly created stored_file should be stored against the conversion using either the store_destfile_from_string or store_destfile_from_path function as appropriate.
  • The supports function should return whether the supported source and target conversion is possible.
  • The get_supported_conversion function is used for informational display to administrators only.