Improved support for external File content

Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.

File synching
Project state	Planning
Tracker issue	TODO
Discussion	TODO
Assignee	Martin Dougiamas

Moodle 2.2

Problems with Files in 2.0

• It is not currently easy to use a file in multiple places throughout Moodle and update them all at once
• It is not currently easy to create a simple shared "course repository" for teachers to use

Example use cases that will become possible

A teacher wants to upload a file once and use it in multiple courses. When they update the file, it should be updated in all their courses automatically.

1. The teacher uploads the file to their private file area (or other repository).
2. In each place they want to add the file, they use the file picker to select the file from their private files area (or other repo) and select "link to latest version".
3. Replacing the file will automatically mean the linked copies use it.
4. Deleting the original file will force the linked copies to be static copies (the teacher will be informed of all the copies before proceeding with the delete).

Several teachers want to create a shared repository of files together

1. One teacher adds a course repository block to the course
2. Using the block, the teacher creates an instance of a "?filesystem? repository" inside the course (essentially a folder).
3. The content of the repository can be edited via the "Course repositories" block.
4. All of the teachers in that course can now see that appear in their filepickers and select files
5. Files can be "linked" as above

A student wants to submit a linked file as an assignment, so they can continue updating it after the assignment due date.

1. This will not be possible, because the assignment will not allow linking to files.
2. The student is forced to upload a copy of a file and this is protected by the assignment module.

Proposed solution summary

The basic idea is to allow the CONTENT of files to be stored outside of the filepool, while all the metadata and access is controlled by Moodle exactly the same as it is now.

1. Extend Files API with a new "source" concept, with all files now having a "sourcerepositoryid" (the repository instance that the file came from) and a "sourcereference" (the address in that repository of the content of the file).
2. All files copied into Moodle have a "sourcerepositoryid" of "0", but others can specify a UUID (usually a URL with file-specific tokens in it, created by the original user who placed the file).
3. Improve the filesystem repository plugin to make it easier to create folder repositories on the fly, via a block.
4. Add support to the filepicker UI to add "linking" to more repositories that support it, including the server files and filesystem browser.
5. Virtual files are served via pluginfile.php URLs just like normal files:
   1. pluginfile.php uses module callback to determine access (as now), and if it passes then
   2. pluginfile.php calls a file logging subsystem to log the fact that this file is being served (useful for copyright reporting, for example)
   3. If the sourcerepository is not 'local', then pluginfile.php uses the relevant repository callback to get the content of the file and streams it to the user with appropriate mimetypes etc
   4. The repositories have a way to cache this content in the normal Moodle filepool, to avoid repeated downloads.
   5. If an external repository is down or not configured, then the repository plugin can choose to just serve the local cached version (useful for restored backups and disaster tolerance)
6. When a file is linked to another another file in the local location, then it has a location of "filepool". We need to pay attention to these during:
   1. Garbage cleanups
   2. Deleting the original file should create real copies of that file where required (user will be informed).

Note that the original URL of files in external repositories are never revealed to users.