Development:Resource module file API migration: Difference between revisions
Line 64: | Line 64: | ||
# Each resource file in the database will have a flag to mark whether this resource is a new 2.0 resource, a 1.9 unmigrated resource or a 1.9 migrated resource. All upgraded resources start off as being marked "unmigrated". | # Each resource file in the database will have a flag to mark whether this resource is a new 2.0 resource, a 1.9 unmigrated resource or a 1.9 migrated resource. All upgraded resources start off as being marked "unmigrated". | ||
# When the browser requests relatively-linked files for a particular unmigrated resource, we grab all the related files that could be included in that resource, move them to the resource fileare and then mark the resource as "migrated". | # When the browser requests relatively-linked files for a particular unmigrated resource, we grab all the related files that could be included in that resource, move them to the resource fileare and then mark the resource as "migrated". | ||
: "copy" is the way, not "move" (we don't know if the file has other uses). Also note the mark is intended to be a "manual" flag (at lest until now). How/who will manage that flag can lead us to different approaches. For example, I can imagine one simple "Test this resource", so the "already migrated" flag is enforced and teacher has the possibility to check if everything woks as expected (multiple pages, complex contents...) with a "mark this resource as working ok" button to perform the flagging (or reverting it). Could work, just one idea. | |||
: Also one utility to handle duplicate files, both in course file area and resources file areas could be handy. So, it will cross-report all the course files that are also being used in resource file areas (already migrated), asking if there are more uses or no, allowing to delete them. In fact, we could "register" any use (access) to course files from any file.php script in order to detect it's being in use and report that information in the utility (to give the teacher more info about detected uses). | |||
: Just two suggestions ~~–––– | |||
Revision as of 10:40, 27 May 2009
Please use talk page for comments and proposals, thanks.
Current problems
- plain text and html resource types separate
- embedded images and files in general are stored in course files without appropriate access control
- local files and Internet links mixed
- HarvestRoad Hive repository type is not maintained.
- embedded images are lost during backup & restore which moves the course to a new site. -- Matt Gibson 08:53, 18 May 2009 (UTC)
- can't move a "whole" web page properly from one course to another (eg course import).
Types of resources
Plugin | Moodle 1.9 | Moodle 2.0 |
---|---|---|
directory | a directory of files on disk | a collection of files in database |
html | A html page, containing media from course files or external | A html page, containing associated media or external |
text | A plain text page in some format | ? |
file | A single file, from course files or external URL | A single file, associated file or external URL |
ims | An IMS resource package from course file or repository web root | An IMS resource package from associated files |
repository | Links to external files in repositories (ie Hive) | (na) |
There are other third party plugins in contrib that need to be migrated:
- mod/book - displays multiple pages of HTML etc
- mod/resource/type/digitalnz - browser and add items listed in http://www.digitalnz.org/
- mod/resource/type/globe - browser and add items in Ariadne repositories
- mod/resource/type/jmol - display chemistry structures from a jmol format file
- mod/resource/type/rss - display one RSS feed on a page
- mod/resource/type/slideshow - display a directory of images as a slideshow
Upgrade planning
Each resource type requires different upgrade code because the files are stored in different places that require special handling. They also have different settings and options with different text encoding in reference, alltext and options db fields.
In several cases, we need to move files from course files into the filearea for the resource (for example, imagine an image included in a HTML file). It's very difficult (impossible?) to cater for 100% of the cases where relative links to other files are being used, so a complete solution during upgrade is not ideal.
An alternative solution is "resource automigration" which upgrades each resource the first time someone looks at it. There are two parts to this solution:
- Each resource file in the database will have a flag to mark whether this resource is a new 2.0 resource, a 1.9 unmigrated resource or a 1.9 migrated resource. All upgraded resources start off as being marked "unmigrated".
- When the browser requests relatively-linked files for a particular unmigrated resource, we grab all the related files that could be included in that resource, move them to the resource fileare and then mark the resource as "migrated".
- "copy" is the way, not "move" (we don't know if the file has other uses). Also note the mark is intended to be a "manual" flag (at lest until now). How/who will manage that flag can lead us to different approaches. For example, I can imagine one simple "Test this resource", so the "already migrated" flag is enforced and teacher has the possibility to check if everything woks as expected (multiple pages, complex contents...) with a "mark this resource as working ok" button to perform the flagging (or reverting it). Could work, just one idea.
- Also one utility to handle duplicate files, both in course file area and resources file areas could be handy. So, it will cross-report all the course files that are also being used in resource file areas (already migrated), asking if there are more uses or no, allowing to delete them. In fact, we could "register" any use (access) to course files from any file.php script in order to detect it's being in use and report that information in the utility (to give the teacher more info about detected uses).
- Just two suggestions ~~––––
Directory type
Recursively copy directory to resource area, skip backup and moddata contents. No special upgrade steps required. Requires full file manager, current forms elements would not be enough.
Links to course files
- Copy linked file (image, sound, pdf, html, etc.) to resource file area
- In case of html files find all absolute links to current course files and copy to resource file area
- Use file automigration described above for dealing with relative links to files that may contain other relative links (html, flash, java, etc.)
External links
Includes links to other courses and site files. This does not require any upgrade. The url is stored in reference field.
Text and web page
- Convert to one plug-in using new editor forms element
- Copy directly linked files that may not contain other relative links to resource area
- Parse html text and copy all directly linked files to resource module file area, those are always absolute links to current course files (except backupdata and moddata)
- Use file automigration described above for dealing with flash, Java and other embedded elements that may require files linked with relative paths
IMS type
- copy files from moddata to new resource file area
- copy original backup of package file to another new resource area
No other special upgrade steps required.
hive
- PROBLEM: remove from core completely?
3rd party
- needs docs
Solution of caching problems
Teachers often do not understand that files may be cached in browsers, se the same mechanism already implemented in SCORM module.
- Internally store all files as itemid=0
- Add new revision db field into resource table
- When serving files always construct links with itemid==revision
- Ignore itemid for resource files in pluginfile.php
Code refactoring proposal
Technically it would be difficult to split current Resource into multiple modules. The idea of module subplugins was only implemented only partially, it should be much easier to split unmaintained plugins into separate modules in contrib. Considering long term maintenance and support costs it might be better to create a minimal resource-like module skeleton in contrib instead of encouraging people to create new unofficial resource plugins.
- no standard subplugin APIs
- all internal module API frozen - no internal refactoring allowed
- subplugin regression more likely
- sloppy db table structures - many universal columns with
- no standard way to upgrade subplugins - main version file needs to be bumped up
- no standard way to uninstall subplugin - db table leftovers
- capability in subplugins not supported
- subplugin upgrade is not integrated into main module upgrade - subplugin upgrade is executed after the main module upgrade (critical)
- subplugins can not have own lang packs
- subplugins do not have own log actions
- instead of reducing code duplication advanced plugins may duplicate a lot more core
- it is not easy to move unmaintained subplugins into contrib
- unofficial subplugins not reported during upgrade
- no support for subplugins in file api
Option 1: split Resource into multiple modules
- mod/resource - formatted text pages
- mod/resourcefile - one or more uploaded files
- mod/resourcedir - directory structure
- mod/resourcelink - links to external files
- +1 to shorter names, like "rsrcfile, rsrcdir ..." due to DB restrictions mainly. --Eloy Lafuente (stronk7) 16:20, 19 May 2009 (UTC)
Activities block would have to be updated to group all resources onto one page instead of listing all resource modules separately. This can be accomplished easily by adding new field into core modules table.
Benefits:
- easier maintenance in future
- easier upgrade (no problems with new db fields)
- showcase of good coding style for beginners
- more contrib modules may behave like the real resources (from users' POV)
- no need for solving of any subplugin problems
Option 2: keep Resource submodules/types
- mod/resource/type/page - replaces Plain text, wiki and HTML resources
- mod/resource/type/file - one or more uploaded files one is marked as the main file
- mod/resource/type/directory - one or more uploaded files
- mod/resource/type/link - links to external files
- mod/resource/type/ims - IMS packages, maintained by Eloy
Problems:
- each type needs different db structure
- we should solve all subplugin problems above (a lot of work, hard to maintain)
Moving to contrib
- New module/type for special mod/resource/type/file/localfile.php (win32 only tweak) - originally intended for large files stored on local networks or CDs
- (No idea what to do with hive repositories, we should also remove the obsoleted auth hive SSO support)
See also
- Development:Using the file API
- Development:Repository API
- Development:Portfolio API
- Development:File API
- MDL-14589 - File API Meta issue
- MDL-16089 - Resource module conversion