UTF-8 contrib

UTF-8 migration > Contrib

How to make your contrib module work for the UTF-8 migration process

The UTF-8 migration script traverse Moodle modules same way as the module upgrade process.

For the purpose of making Moodle totally UTF-8, every field of type (varchar, text, enum) will need to be converted to UTF-8.

To do this, you need to put 2 files in the db folder, where the mysql.sql and postgres.sql files are.

• migrate2utf8.xml defines what fields to be converted. Every field needs a type and a length, and if applicable, index information (see below). There are 3 methods associated with each field:

1. -NO_CONV is for those fields that could only contain English Chars, written by Moodle.
2. -PLAIN_SQL_UPDATE is used when you can find both the user language and course language for that record using 1 SQL conveniently. If you use this method you must include both sqls for the user lang and the course lang.
3. -PHP_FUNCTION is used when the detection of userlang or courselang is more difficult, or more sophisticated methods are needed for the conversion (e.g. serialized objects, or when you need to append _utf8 to a lang field etc)

• migrate2utf8.php defines each php function included to do the conversion.

Please look at forum/db for the exact formats of these files.

All indexes need to be dropped prior to (or at the same time) the processing this field. This can be done using dropindex or dropprimary. After the field is processed, you need to add the indexes back using addindex, adduniqueindex, or addprimary. If you don't have enough fields in the table to do addindex or dropindex, you can add arbitrary number of dummy fields (see scorm/db/migrate2utf8.xlm)

To test the migration, please don't forget to turn debug on.

Contrib modules which don't contain migration scripts

Flashcard Object

Please list names of other modules here...

Manually converting a table to UTF-8

If you have extra tables that contain text in English or UTF encoding from Moodle 1.5, and you want to convert them manually to be used under 1.6, you can do it using PhpMyAdmin (or direct SQL) here is a procedure: (Please make a backup first)

1. For every textual field in the table, (i.e. varchar, char, text, longtext, enum etc), you need to change the column encoding to UTF8.

Before you can change the column encoding to UTF8, you have to make sure that this column is not associated with any index. If it is, you need to drop the index(es), and add them back later, so please remember what the index is before proceeding.

The SQL to drop an index is

ALTER TABLE 'tablename' DROP INDEX 'indexname';

Example,

ALTER TABLE mdl_forum_posts DROP INDEX forum_posts_subject_idx;

The SQLs to change column encoding to UTF8 is

ALTER TABLE 'tablename' CHANGE 'fieldname' 'fieldname' LONGBLOB;

ALTER TABLE 'tablename' CHANGE 'fieldname' 'fieldname' original_type CHARACTER SET utf8 NOT NULL DEFAULT defaultvalue;

Example,

ALTER TABLE mdl_forum_posts CHANGE subject subject LONGBLOB;

ALTER TABLE mdl_forum_posts CHANGE subject subject varchar(255) CHARACTER SET utf8 NOT NULL DEFAULT ;'

Then, if applicable, you need to add the index back.

ALTER TABLE ADD INDEX 'indexname' 'indexvar';

Example,

ALTER TABLE mdl_forum_posts ADD INDEX forum_posts_subject_idx(subject(255));

2. After all columns have been converted to UTF8 encoding, you can change the table encoding to UTF8.

ALTER TABLE 'tablename' CHARACTER SET utf8;

Example,

ALTER TABLE mdl_forum_posts CHARACTER SET utf8;

3. And you are done!