Converted from the Word document posted at http://moodle.org/mod/forum/discuss.php?d=152574 by Sam Marshall. --Frank Ralf 11:57, 21 June 2010 (UTC)

Moodle 2.0

Structured Content -> Moodle 2

Introduction

This is a rough log of actions taken in converting the oucontent (Structured Content) module to run inside standard Moodle 2.

The total development time was about 4 full days. On the minus side, the module isn’t quite 100% complete (see later); on the plus side, I also did about half the work on the ousearch block/fulltext search library..

Preparation

In order to get ready for development, I did the following (which would not be necessary for a ‘real’ update):

Cloned Tim’s git repository and made a new branch in it.
Made a new Eclipse project based on the oucontent folder from this repository.
Set up FileCopier instances to make a copy of the current core Moodle 2.0 onto my dev server, and of this oucontent folder to go inside that.

Install

Running the Moodle install process revealed the following necessary changes. After making the changes below (10 minutes), install completed successfully.

version.php

Plugins (also blocks) only seem to install with that file present. Needs only to contain one line:

$plugin->version = 2007101509;

--Frank Ralf 11:48, 14 November 2010 (UTC)

access.php

Change array name to $capabilities only. [Warning only]
Remove all references to "admin" from access.php as it is not needed. [Warning only]
Rename "legacy" to "archetypes" in line with core code. [Optional]
Add new "manager" role. --Frank Ralf 11:22, 14 November 2010 (UTC)

Example: \blocks\online-users\access.php --Frank Ralf 11:17, 14 November 2010 (UTC)

Language file change

Rename language folder inside the plugin from "en_utf8" to "en".
Change $a in language files to {$a}, also {$a->param} etc.
- (except where it already has curly brackets around.)
Change \" withinin single-quoted strings to ".
- (Moodle 1.9 required an extra backslash before any double-quote in a lang string; Moodle 2 doesn't.)
Popup help files in the help folder need to be migrated to _help language strings (and shortened). However, there was only one popup help file in Structured Content.
- Before: help content in file
```
mod/MODNAME/lang/en/helpidentifier.html
```
- After: help content in
```
$string['helpidentifier_help']
```
  , in the module's lang file.
The language file must contain a line

$string['pluginname']   = 'Plugin name';

edit_form.php

Change help buttons from setHelpButton to addHelpButton
- Before:
```
setHelpButton( 'elementname', array('helpidentifier', "Printable title string", 'helpmodule'))
```
- After:
```
addHelpButton('elementname', 'helpidentifier', 'helpmodule')
```
- Note you'll also need to move the help contents from a standalone html file to a standard lang string (as mentioned in the lang file changes section)

settings.php

Seems to replace config_global.html for block, see example on talk page. --Frank Ralf 18:42, 15 November 2010 (UTC)

Database changes

I searched for database queries that needed updating using the regular expression given in the migration documentation on Moodle wiki (which I improved slightly). This found 154 matches. I updated them to the new format, which took a couple of hours.

When updating, I changed most calls to use the MUST_EXIST parameter, which enabled me to remove several lines of code that checked for errors. This needs a bit of care for cases where an API must return true or false rather than throwing exceptions; I believe it’s OK to throw exceptions in most core Moodle API areas, but there were some points where this is used and shouldn’t halt execution i.e. returning false is okay.
I deleted the previous upgrade changes and left the upgrade file blank, since there is no point wasting time updating this code.
The new database API should use ? for parameters rather than adding them into SQL strings. Where parameters were numeric and don’t need escaping, I didn’t always bother making this change.
One function (related to <olink>) I noticed would have poor performance, and would have been a bit difficult to fix the database function as is, so I rewrote it using a better-performing technique rather than just changing the database calls.
18 of the reported issues I left alone. These were in backup and restore and in pagelib, which I ignored because these will need to be heavily changed later.

File system changes

Because of the new filesystem and because OU content uses files fairly heavily, I needed to make major changes. This took about 2 full days.

Convert file processing

All uses of file storage need to be changed to use the new file API.

The XML file and asset files must be stored using file API.
- For previews, these files are stored in a global location because previews are not course-related. This location is in the system context in a file area called oucontent with the item ID from the oucontent table, e.g. 341. (TBC, may change if this doesn’t work.)
- When saved these files are stored in the module context id in a file area called oucontent, also with the instance ID as item id.
The alternative format files must be stored using file API. For convenience, these files are stored in the same area in a suitably-named folder, just like the current system.
User files (currently, just from voice recorder) are stored separately using file API so as to more logically implement backup/restore and the restricted access to these files. These are stored in an area called oucontent_userdata within the same context as above (depending on whether it’s preview or live). If the preview ID is 341 and the user ID is 11 then files for that user will be in folder /11/ within this area, which will have item ID 341.
Media library files are stored in the oucontent_shared area at system level with item ID 0 and in the path /medialibrary/.
I ditched support for the old SMB preview mechanism, as the new client now uses HTTP preview exclusively. We still use SMB (among other mechanisms) to collect assets.
The function oucontent_get_asset obtains data from a variety of sources (SMB, HTTP, Documentum). I refactored this function to include some code within it about checking whether the file exists, because this was done every time it was called, which was repetitive; also moving it inside the function means I can concentrate the file API changes there.
At time of writing, slight changes were still being made to the file API. This will mean that one piece of work I would otherwise have had to do does not need doing, and slight changes may need to be made to a few other pieces of code (changing area name).
Rather than fixing it, I deleted the export folder which is going to be removed from the 1.9 version of the oucontent module as well. I also deleted other export-related functionality (a few other files) because there is not much point porting it when it will be reimplemented differently.

Upgrade existing files

Because we are planning to install Moodle 2 as a separate system, we do not need to implement upgrade from existing files. (This would be a moderately significant chunk of work otherwise.)

New interface

The code for doing interface output has changed. You can actually still use the old version but it will display developer warnings and doesn’t support many Moodle 2 features. Consequently it is best to change all code at least to use the new header and footer methods.

I renamed the CSS file styles.php to styles.css, and did search-replace to change #mod-oucontent-whatever IDs to #page-mod-oucontent-whatever.
I changed the header and footer display in upload.php and managed to get this script to work. This was repeated on all pages that display to user.
I fixed some issues in view.php but then had to make a number of significant changes.
- The script requires YUI libraries. I had to change it to use a new way to require these libraries. This probably doesn’t work, need to come back to it later.
- Because this is now using standard Moodle page display, I had to display the left-hand column (which looks like blocks) in a different way. This now needs to use the $PAGE->blocks->add_pretend_block method, which means the information needs to be organised differently (into actual blocks).
I had an issue with breadcrumb trail – I wasn’t able to get the breadcrumb trail printed the way I wanted (this is not resolved yet).
In edit_formats.php the slow_redirect function prints out a very sketchy page with a meta redirect tag to redirect after 30 seconds, and a JavaScript to print dots (users! easily entertained). I can’t see how to do this with $PAGE->requires, so to make life easy, I changed it to not use any Moodle output on this page and print it out as pure HTML instead.
The ‘confirm delete’ page when deleting an embed template was bloody useless. It said ‘Are you sure you want to delete this template?’ – without telling you what template you were deleting. I put in the name of the template while I was there.

JavaScript

Converting a JavaScript to the new system took a whole day. Doing it properly would require rewriting everything for YUI3 and reorganising it entirely, so I didn’t take that approach. On the other hand, I could not include it the old way, because Moodle now includes YUI libraries at the bottom of the page so they are not available if I include my script at the top.

I renamed the script to module.js and added a stub object in the global M container (where the real code ought to be) which can be used to launch it. This is the standard Moodle 2 way of doing things.
Moodle 2 uses a complex mechanism to express required JavaScript modules. You can include them in a ‘requires’ list – it is not obvious what the module names are, so you have to search to find other source code that does this, most notably find_module in outputrequirementslib.php. This handles only YUI3 and YUI2 libraries. For SWFObject (below), this has to be included a different way; I searched for existing code that used it and copied from that.
I changed all the parts in the XSL code where it generated inline JavaScript function calls (this no longer works because the JS is included at end of page). If possible I removed these, adding classes if necessary; if not possible I used JavaScript to add data information to the DOM nodes that would be used later in function calls. I then changed the stub object so that it found these elements using YUI3 calls and called the same functions.
Moodle now uses SWFObject instead of UFO to embed Flash. SWFObject uses a different format for its flashvars parameter: an array instead of a string. I changed the code so it worked again.
Moodle has a way to include language strings in JavaScript but I did not use this because Structured Content requires its own language facility (an extra layer on top of Moodle’s language files). However, it was easy to pass through my own strings to the init function without having to manually escape them, which was nice.

Reduce local changes

Local changes, such as utility libraries, outside the module folder are a potential problem.

Reduced local library usage

By changing modules to avoid calling local methods, I was able to reduce the usage of local libraries.

I found code using the OU function mkdir_recursive and changed it to use core function check_dir_exists. (This change, and removing the OU function, could be done within 1.9.)
I removed a call to ou_force_current_theme which is no longer needed in 2.0 as it will ship with current theme only.
I changed transaction_wrapper calls to use the new Moodle transaction API start_deferred_transaction.

Continued local library usage

oucontent still needs to use the following local libraries:

SMB support; I moved this from local/smb_connection.php to local/smb/smb_connection.php. The library needed minor changes (2 lines) to remove dependencies on OU-specific utilities, and some support files moved and modified from other areas of OU Moodle to provide a standard way to set some necessary options.
References support in local/references. (Library may need upgrading.)
Full-text search support in blocks/ousearch. (I upgraded the library itself; rest of the block still needs upgrading.)
Support for redirecting requests to Library servers; I moved this from a function in local/utils.php to a new folder local/oulibrary/lib.php (and renamed the function to make it clearer).The intention would be that this folder can also be used for other crazy library-related hacks. Other than tidying it up, this library probably does not need upgrading.
Internal system dashboard/TTM log integration in admin/report/dashboard.
When reimplemented, OU mobile support in local/mobile will be required. I removed all mobile support as it will need entirely reimplementing for Moodle 2 (hopefully in a less horrible way).
Flash player in local/mediaplayer.swf – this should perhaps be moved into a subfolder or even included as part of a custom filter plugin which is done instead of our OU-specific changes to the media filter.
When reimplemented, there will probably also be references from within oucontent to the new export libraries.

Tidy up

Some parts of code are just a mess. This is a good opportunity to fix them while making other changes.

Removed formatterlib.php, placing one function into oucontent.php and the other function into inline code in the single script that uses it. (This latter function was unsuitable as a function anyway because it relied on many global variables.)
I deleted some obsolete and unused files.

Missing pieces

I tested most features and they work correctly. The following exceptions would need extra work before release:

Images and other asset files do not work in preview mode (they work after you save to a course). This can’t be resolved until after core make a promised change (MDL-21227) which will probably also require changes to file API functions in here.
Breadcrumbs are not displayed correctly because I haven’t figured out how to do this; I’m waiting on a response from Moodle forums.
I did not implement backup and restore; it doesn’t seem useful to implement this while restore is not done yet in core (so there’s a chance backup will change too).
I stripped out the export features. These are not really used much in VLE, but need to be reimplemented at least for OCI. There will need to be slight changes because of how the JavaScript has changed.
The references feature is still there and I converted all the related functions within OU content but, because I didn’t want to convert the references library, I presume it won’t actually work yet, so I didn’t bother testing it at all.
As noted mobile support has removed. (I think we can implement some aspects of mobile support more generically as part of the theme, but some parts will need reimplementing.)
Side notes and anything in the right-hand column does not display correctly. I don’t really know why and couldn’t fix it yet. It might be something to do with the theme, however, so I thought it might make sense to wait until we have an OU theme.

Conclusion

Here’s my overall notes on the different parts of the experience.

Converting database calls and transactions (from OU transaction_wrapper which is now obsolete) was easy and quite close to mechanical. The exception was when SQL queries were built up in parts using functions. In these cases I found it easiest to make minimal changes, which results in code that isn’t quite best-practice for Moodle 2 (for example, with values hard-coded into the query instead of using ?) but works OK.The database API is high-quality and much nicer to use than before.
Converting file calls was challenging. However, once I had understood the system and developed several helper functions, the process became fairly easy (albeit not as mechanical as the database conversion).The file API is high-quality and logical but at the time of writing is not really quite finished.
- The file picker tool can be inserted in Moodle forms easily. There is a process for automatically making it put the uploaded files in the right place, but it wasn’t suitable for my use, so I didn’t try it in this case. However, doing it manually was straightforward and it was easy to replace some old file input elements. I did notice a couple of possible UI issues with the result:
  - The upload page should be the default page of the file picker, but it is not. This means three extra clicks (click pick file; click upload option; click browse, select file, OK dialog, OK picker) compared to a standard file input field (click browse, select file, OK dialog), instead of two extra clicks. A fair difference and it definitely feels like it’s slowing you down.
  - When you have clicked ‘OK’ on the file picker it sort of seems like that should be all you have to do, rather than you having to also click save on the actual form.
Converting display output is easier than it might be. Generally, it involves adding a bunch of $PAGE->set calls at the top of the page and changing print_header and _footer to the new $OUTPUT varieties. I was able to reduce duplicated code by doing a lot of those $PAGE->set calls in the shared init.php.
- I didn’t figure out how to do breadcrumbs yet so can’t comment on how hard this is…
Converting JavaScript was difficult because it requires changes in the model used for JavaScript. (Of course, if the existing code had already happened to use the ‘correct’ model, there wouldn’t have been a problem.)
While doing the conversion I wanted to make a number of other changes to improve the code but tried to resist because this investigation is parallel to continuing development of the 1.9 module, and I’d like it to merge later. However, when updating other code, we have a good opportunity to make minor improvements to code quality such as fixing whitespace and other coding style violations, correcting spelling errors, adding missing PHP documentation, and other minor tasks that don’t add significantly to the update time but will improve future maintenance.
- In order to achieve this advantage we would need to write explicit guidelines and ensure code review of all such changes (whether internal or outsourced).

A screenshot

Just to prove it did work, here’s a screen of one of the documents from the SC website. You can just see an ‘interactive’ activity at the bottom…

Documentation

Migrating contrib code to 2.0/Experience of converting a module to Moodle 2

Contents