Converting files to UTF-8

Szablon:Moodle 1.6 Some files, like Moodle import and export files and custom language packs or language files from third party modules need to be converted or treated as UTF-8 before they may be used with Moodle 1.6 and later.

*nix like computers (including Mac OS X)

Generally, this may be done with the iconv command on Unix, Linux or a Mac.

iconv -f original_charset -t utf-8 originalfile > newfile

see also the windows explanation - the script there is one for *nix computers, but used in a cygwin environment

Windows computers

For Windows, there are three methods of performing the conversion.

Method 1

• Open the flat file in PSPad (a freeware editor) : http://www.pspad.com
• Click on Format, UTF-8
• Save the file

Method 2

Download the Windows version of the iconv program. Download the "Complete package, except source" and run the setup program. The executable is located in the bin folder. Run from the command prompt (Start -> Run -> cmd) and follow the instructions as above.

Method 3

The conversion may also be done by using Cygwin, a Linux-like environment for Windows, and excecuting the iconv command in that environment. Here is an example of a working solution on Windows with Cygwin:

• Create a text file, named ToUtf8.txt
• Fill it with the following code

#!/bin/bash
FROM=iso-8859-1
TO=UTF-8
ICONV="iconv -f $FROM -t $TO"
# Convert
find ToUTF/ -type f -name "*" | while read fn; do
cp ${fn} ${fn}.bak
$ICONV < ${fn}.bak > ${fn}
rm ${fn}.bak
done

Two things should be changed for your local situation:

1. FROM is the originating encoding (the one your original files are in)
2. ToUTF is the foldername where the files that need to be converted are in. This folder may contain subfolders. Make sure you have a backup!

• Start Cygwin.
• With the cd foldername, cd.., ls commands, go to the folder on your windows machine where the ToUtf8.txt script and the ToUTF8 folder are in.
• Execute the script by typing sh ToUtf8.txt and your files will be converted.

Saving files directly as UTF-8

Windows

Sorry, I don't have access to a Windows computer right now. Please, someone, write something intelligent here.

Mac OS X

The built in text edit application has a 'Plain text encoding' option in the Save as... dialogue.

Linux

The standard Gnome Text Editor defaults to UTF-8 and has character set options when loading and saving.