Resource module file API migration

Moodle 2.0

Please use talk page for comments and proposals, thanks.

Current problems

• plain text and html resource types separate
• embedded images and files in general are stored in course files without appropriate access control
• local files and Internet links mixed
• is HarvestRoad Hive repository type maintained? The current SSO hooks are not compatible with 1.9 auth improvements
• embedded images are lost during backup & restore which moves the course to a new site. -- Matt Gibson 08:53, 18 May 2009 (UTC)

Types of resources

• Directory of files
• Resource page - using new editor element
• External resource link - some file stored elsewhere
• Single uploaded images and files
• Uploaded html files
• IMS type with optional repository support
• HarvestRoad Hive repository integration
• Other 3rd party plugins in contrib

Upgrade planning

Each type requires different upgrade code. Because the files are stored in different places, require special handling, settings and options use different text encoding in reference, alltext and options db fields.

We can not reliably find all relatively linked files, such attempts would be also time consuming because we would have to do that recursively. Alternative solution is to let browsers do the work - resource code may detect files missing in resource files and copy them automatically on the fly from course files. We have named this feature automigration of relatively linked resource files. This feature would be enabled separately in each resource, new instances created in 2.0 will not need this anymore because all files would be self contained.

Directory type

Recursively copy directory to resource area, skip backup and moddata contents. No special upgrade steps required.

Links to course files

1. Copy linked file (image, sound, pdf, html, etc.) to resource file area
2. In case of html files find all absolute links to current course files and copy to resource file area
3. Use file automigration described above for dealing with relative links to files that may contain other relative links (html, flash, java, etc.)

External links

Includes links to other courses and site files. This does not require any upgrade. The url is stored in reference field.

Text and web page

1. Convert to one plug-in using new editor forms element
2. Copy directly linked files that may not contain other relative links to resource area
3. Parse html text and copy all directly linked files to resource module file area, those are always absolute links to current course files (except backupdata and moddata)
4. Use file automigration described above for dealing with flash, Java and other embedded elements that may require files linked with relative paths

IMS type

1. copy files from moddata to new resource file area
2. copy original backup of package file to another new resource area

No other special upgrade steps required.

hive

• PROBLEM: remove from core completely?

3rd party

• needs docs

Solution of caching problems

Teachers often do not understand that files may be cached in browsers, se the same mechanism already implemented in SCORM module.

1. Internally store all files as itemid=0
2. Add new revision db field into resource table
3. When serving files always construct links with itemid==revision
4. Ignore itemid for resource files in pluginfile.php

Code refactoring proposal

Technically it would be difficult to split current Resource into multiple modules. The idea of module subplugins was only implemented only partially, it should be much easier to split unmaintained plugins into separate modules in contrib. Considering long term maintenance and support costs it might be better to create a minimal resource-like module skeleton in contrib instead of encouraging people to create new unofficial resource plugins.

Subplugin related problems

1. no standard subplugin APIs
2. all internal module API frozen - no internal refactoring allowed
3.  subplugin regression more likely
4. sloppy db table structures - many universal columns with
5. no standard way to upgrade subplugins - main version file needs to be bumped up
6. no standard way to uninstall subplugin - db table leftovers
7. capability in subplugins not supported
8. subplugin upgrade is not integrated into main module upgrade - subplugin upgrade is executed after the main module upgrade (critical)
9. subplugins can not have own lang packs
10. subplugins do not have own log actions
11. instead of reducing code duplication advanced plugins may duplicate a lot more core
12.  it is not easy to move unmaintained subplugins into contrib
13. unofficial subplugins not reported during upgrade
14. no support for subplugins in file api

Option 1: split Resource into multiple modules

• mod/resource - formatted text pages
• mod/resourcefile - one or more uploaded files
• mod/resourcedir - directory structure
•  mod/resourcelink - links to external files

+1 to shorter names, like "rsrcfile, rsrcdir ..." due to DB restrictions mainly. --Eloy Lafuente (stronk7) 16:20, 19 May 2009 (UTC)

Activities block would have to be updated to group all resources onto one page instead of listing all resource modules separately. This can be accomplished easily by adding new field into core modules table.

Benefits:

• easier maintenance in future
• easier upgrade (no problems with new db fields)
• showcase of good coding style for beginners
• more contrib modules may behave like the real resources (from users' POV)
• no need for solving of any subplugin problems

Option 2: keep Resource submodules/types

Resource module

• mod/resource/type/page - replaces Plain text, wiki and HTML resources
• mod/resource/type/file - one or more uploaded files one is marked as the main file
• mod/resource/type/directory - one or more uploaded files
• mod/resource/type/link - links to external files
• mod/resource/type/ims - IMS packages, maintained by Eloy

Problems:

• each type needs different db structure
• we should solve all subplugin problems above (a lot of work, hard to maintain)

Moving to contrib

•  New module/type for special mod/resource/type/file/localfile.php (win32 only tweak) - originally intended for large files stored on local networks or CDs
• (No idea what to do with hive repositories)

See also

• Using the file API
• Repository API
• Portfolio API
• File API
• MDL-14589 - File API Meta issue
• MDL-16089 - Resource module conversion