Backup and restore FAQ: Difference between revisions

Latest revision as of 07:19, 10 January 2013

How do I backup a course?

See Course backup and Automated backup setup.

How do I restore a course?

See Course restore.

How do I backup my site?

See Site backup.

What are the pros and cons of course versus site backups?

Site backups are recommended in order to have all data saved with the best confidence and the shortest recovery time.

For a site administrator, automated course backups are more expensive in terms of time, CPU usage and storage. The recovery time to have a site running again takes longer than a site backup. However, teachers and site administrators might find a course backups as a way to create a "fresh" copy of a course that can be re-used (in older versions of Moodle, in newer versions see Import course data) or as a method to distribute a course(s) to other Moodle sites.

Why is my automated course backup much smaller in size than my manual course backup?

This is an intentional design decision. Because of the way files are stored in Moodle 2.x, there is no need to include the files in the backup if you are planning to restore them to the same Moodle site. Leaving them out saves huge amounts of disc space and makes the backup procedure much faster.

What data is not contained in course backups?

By selecting all the options when setting up the backup you can include almost all the data in the course. However you should be aware of the fact that some things are not backed up:

Quiz questions are only backed up if at least one question from their category has been added to a quiz.
Scales are only backed up if they are used by at least one activity.
Users' passwords are not backed up when the "Include enrolled users" option is selected.

Why is there no "all/none" feature when selecting items to backup?

This was enabled in MDL-32705 and is available in Moodle 2.3.2 onwards.

The process ends with: "Error: An error occurred deleting old backup data". What should I do?

This part of the backup (or restore) procedure tries to delete old info, used in previous executions, performing the following tasks:

Delete old records from "backup_ids" table: Check the table exists, repair it and try again.
Delete old records from "backup_files" table: Check the table exists, repair it and try again.
Delete old files from "moodledata/temp/backup": Delete the dir completely and try again.

Backup error message

For points 1 & 2, there are various ways of repairing tables, including using MySQL Admin.

For point 3 see below:

The error message states that the "directory not empty" and gives the path to that directory. If you go there with an FTP program you can see what is there and clean up. It could be just some empty subfolders that were leftover. Deleting these has been able to help. One can also delete the dir "moodledata/temp/backup" completely. That can take a bit longer but may solve several problems at once.

The process ends with: "XML error: not well-formed (invalid token) at line YYYY". What can I do?

This problem can appear at any point in the restore process. It's caused when the XML parser detects something incorrect in the backup file that prevent correct operation. Usually, it's caused by some "illegal" characters added in the original course due to some copy/paste of text containing them (control characters, or invalid sequences...).

The best method to handle this issue is:

Unzip the problematic backup file under one empty folder.

Open the moodle.xml with Firefox. It will show you where (exact char) the problem is happening.

Edit the moodle.xml file with some UTF8-compatible editor and delete such characters. Save changes.

Test the moodle.xml file again with Firefox until no error was displayed.

Zip everything again (all the folder contents but not the folder itself!).

Restore the course. It should work now.

Restore still not working? See the next question.

Also, if possible, it's highly recommended to solve those problems in the original course too from Moodle itself. Once "repaired" there, problems will be out if you create new backup files in the future.

The process ends with: "moodle xml not found at root level of zip file". What can I do?

If you are restoring from a zip file backup make sure the moodle.xml file is at the root level. To ensure this:

Unzip the backup file of the course (example: mycourse.zip)
Once the file is unzipped, open the folder (example: mycourse).
Select the folders within the mycourse folder AND the moodle.xml file and create a zip of those item (example: mycourse_new.zip)
Upload the new zip file (example: mycourse_new.zip) and restore from that.

If the backup file is guaranteed to be correct, check paths to external files (zip, unzip). Incorrect settings also lead to this error message (see the Using Moodle forum discussion moodle.xml not found in root... and MDL-14812).

The process ends with: "An error occurred while copying the zip file..."

This problem is most likely caused by a permissions issue in the destination directory. Backup files are copied to "XXX/backupdata" under your dataroot directory (where XXX is the id of the course being backed up).

The problem could also be caused by a disk being full, though this is far less likely.

To obtain precise information about what's happening, you can enable debug messages in Administration > Server > Debugging (select the maximum level - DEVELOPER) and/or check the web server error logs.

I Still get an XML error. How can I clean the borked XML file?

In some cases XML backup files may contain characters causing the restore process to abort, even after the steps described in the previous question. In such cases you may want to try the following:

Download the Atlassian XML Cleaner Utility from the JIRA Atlassian site.

Unzip the problematic Moodle backup file under one empty folder. Moodle will create the course file folders as long as the unclean moodle.xml file. Please unzip using the Moodle unzip feature.

Rename the unclean moodle.xml file to moodle-unclean.xml.

If you don't have access to your Moodle server's command prompt, using the Moodle zip feature, zip the moodle-unclean.xml file only, download the zip file locally and unzip it. It is very important to download the xml file in zipped format to avoid unwanted character encoding when transferring from an operating system to another.

Move the downloaded Atlassian XML Cleaner Utility in the same folder where is your moodle-unclean.xml file.

Issue the following command from the command prompt:

java -jar atlassian-xml-cleaner-0.1.jar moodle-unclean.xml > moodle.xml

If you launched the utility on your local computer, zip the just created (and hopefully cleaned) moodle.xml file and upload it in the same place from where you downloaded the moodle-unclean.xml file. Once uploaded, unzip it using the Moodle unzip feature.

Zip everything again (all the folder contents but the folder itself!).

Restore the course. It should work now.

What does "Some of your courses weren't saved!!" mean?

There are three possible causes of this problem:

Error - this happens when the backup procedure has found an error and so hasn't finished the backup of a particular course. These are "controlled" errors and the scheduled backup continues with the next course.
Unfinished - this happens when the backup procedure dies without knowing why. When the cron is next executed it detects that the last execution went wrong, and continues skipping the problematic course. A possible solution would be to raise the PHP/Apache limit in your installation (memory, time of execution...). By taking a look to your log tables you should be able to see if the "crash" is happening at exact time intervals (usually a problem with the max_execution_time php's variable), or if there is some exact point were all the courses are breaking (generally internal zip libraries, try to switch to external executables instead).
Skipped - this happens when a course is unavailable to students and has not been changed in the last month (31 days). This isn't an error situation - it's a feature, especially useful for sites with many unavailable old courses, saving process time.

Why are some courses being skipped?

Course backups automatically skip courses which are unavailable to students and have not been changed in the last month.

The Hide courses excluded from automatic backup discussion in the Using Moodle forum describes a way that you can disable or override this thirty day check. This could be useful if, for example, you wanted to change the period from thirty days to six months.

Why does restore stop, rather than completing?

Attempting to restore a course to an older version of Moodle than the one the course was backed up on can result in the restore process failing to complete. To ensure a successful restore, make sure that the version of Moodle you are restoring the course to is the same, or newer, than the one the course was backed up on.

If it stop unexpectedly with no errors shown try again with Debugging switched on. Any errors you now see can help experts in the support forums diagnose your problem. You can also check the discussion links in the See also section below for further advice.

Restore stops with the message "Trying to restore user xxxx from backup file will cause conflict"

Error message in Moodle 2.0

This message is displayed when:

Your target site has one xxxx user (xxxx being the username)
The backup being restored also has one xxxx user
After various comparisons, Moodle has detected that the xxxx user in 1. and the xxxx user in 2. aren't the same user.

If 1, 2 and 3 above happens, restore stops before performing any action as far as restoring that user will end associating "things" (create activities, posts, attempts... whatever...) to the xxxx user in the target site, when those "things" weren't performed by the same user at all.

Those checks and behaviour were introduced in Moodle 1.9.x and continue being valid under 2.0. Usually the xxxx user in the messages is the "admin" user (that exists in practically all Moodle installations).

There are two possible solutions to make both xxxx users in 1 and 2 to match (and avoid the conflict):

a) Modify the backup's users.xml file and make the email or firstaccess fields match with the ones in target site.
b) Modify the target site and set the user email or firstaccess fields to match with the ones in backup's user.xml file.

Method a) is recommended so the restore process will match both xxxx users and every "thing" in the backup file belonging to xxxx will be associated to the already existing xxxx user in the target site.

 NOTE: When using method a) be aware that the moodle-filename-backup.mbz is a zip file and can be renamed to moodle-filename-backup.zip and unzipped. 
 When editing is complete, rezip and then rename using the original file name with the "*.mbz" extention.

Why are certain course links broken in a restored course?

Inter-activity links must be absolute (full) URLs e.g. http://site.com/mod/resource/view.php?id=xxx in order to be processed properly during backup and restore.

Any relative URLs e.g. /mod/resource/view.php?id=xxx, ../resource/view.php?id=xxx or view.php?id=xxx will result in broken links when the course is restored.

@@ Line 1: / Line 1: @@
-{{FAQ}}
+{{Backup}}
-;Site backups
+==How do I backup a course?==
-:Site backups, as explained in [[Upgrading_Moodle#Backup_important_data|upgrading Moodle]], are recommended in order to have all data saved with the best confidence and the shortest recovery time.
-;Course backups
+See [[Course backup]] and [[Automated backup setup]].
-:Course backups, configured on the [[admin/backup|backup]] page, are more expensive in terms of time and CPU usage. The recovery time to have your site running again is longer. Course backups are useful for obtaining "fresh" copies of courses to be re-used or distributed individually, however they should never be used as a primary backup system (unless your hosting doesn't allow the preferred site backups). In order to make scheduled backups, you have to set up CRON to run periodically. Please refer to the [[Cron|cron instructions]].
+==How do I restore a course?==
-==How do I backup my whole Moodle site?==
+See [[Course restore]].
-There are two main things you need to make a copy of - the database and the uploaded files. The Moodle scripts themselves are less important, since you can always download a fresh copy if you have to.
+==How do I backup my site?==
-There are many ways to do such backups. Here is an outline of a little script you can run on Unix to backup the database (it works well to have such a script run daily via a cron task):
+See [[Site backup]].
- cd /my/backup/directory
+==What are the pros and cons of course versus site backups?==
- mv moodle-database.sql.gz moodle-database-old.sql.gz
- mysqldump -h example.com -u myusername --password=mypassword -C -Q -e -a mydatabasename > moodle-database.sql
- gzip moodle-database.sql
-For the files, you can use rsync regularly to copy only the changed files to another host:
+[[Site backup|Site backups]] are recommended in order to have all data saved with the best confidence and the shortest recovery time.
-  rsync -auvtz --delete -e ssh mysshusername@example.com:/my/server/directory /my/backup/directory/
+For a site administrator, [[Automated course backup|automated course backups]] are more expensive in terms of time, CPU usage and storage. The recovery time to have a site running again takes longer than a site backup. However,  teachers and site administrators might find a course backups as a way to create a "fresh" copy of a course that can be re-used (in older versions of Moodle, in newer versions see [[Import course data]]) or as a method to distribute a course(s) to other Moodle sites.
-'''Character Encoding'''
+==Why is my automated course backup much smaller in size than my manual course backup?==
-When dumping the entire Moodle database, Administrators should be careful to watch for possible character encoding issues. In some instances, backups created with mysqldump or phpmyadmin may not properly encode all of the data resulting in spurious A characters. One solution is to use mySQL Administrator 1.1 or another tool that will force a UTF-8 dump of the data.
+This is an intentional design decision. Because of the way files are stored in Moodle 2.x, there is no need to include the files in the backup if you are planning to restore them to the same Moodle site. Leaving them out saves huge amounts of disc space and makes the backup procedure much faster.
-==How do I restore a backup of my whole Moodle site?==
-If you have followed the above instructions and created a backup of a Moodle site, you may need to know how to restore the site backup you created.  Here is a set of basic steps that make up the restore process.
-. Rename the original moodle directory to something different (so you still have it) and copy the backed up moodle directory or a newly downloaded moodle directory in its place.
-. If you are running mysql, a backup of the database should be a .sql, .gz or .tar.gz file.  If it is .tar.gz or .gz you need to extract it until it is an sql file.
- tar -xzvf moodlesqlfile.tar.gz
-. If you are running mysql, import the sql file back into a newly created database on the mysql server.  Be careful here, some backups try to import right back into the same working database that moodle is connected to.  This causes database problems that damage a Moodle installation.  The best thing to do is make a new database, restore the backed up database into it, and change the Moodle config.php file to connect to this new database (this way you still have the original database).
-once you have created the new database:
- mysql -p new_database < moodlesqlfile.sql
 ==What data is not contained in course backups?==
@@ Line 46: / Line 27: @@
 * Quiz questions are only backed up if at least one question from their category has been added to a quiz.
 * Scales are only backed up if they are used by at least one activity.
+* Users' passwords are not backed up when the "Include enrolled users" option is selected.
+==Why is there no "all/none" feature when selecting items to  backup?==
+This was enabled in MDL-32705 and is available in Moodle 2.3.2 onwards.
-==Error: An error occurred deleting old backup data==
+==The process ends with: "Error: An error occurred deleting old backup data". What should I do?==
 This part of the backup (or restore) procedure tries to delete old info, used in previous executions, performing the following tasks:
-* Delete old records from "backup_ids" table: Check the table exists, repair it and try again.
+# Delete old records from "backup_ids" table: Check the table exists, repair it and try again.
+# Delete old records from "backup_files" table: Check the table exists, repair it and try again.
-* Delete old records from "backup_files" table: Check the table exists, repair it and try again.
+# Delete old files from "moodledata/temp/backup": Delete the dir completely and try again.
-* Delete old files from "moodledata/temp/backup": Delete the dir completely and try again.
+[[Image:BackupProblem.gif|thumb|Backup error message]]For points 1 & 2, there are various ways of repairing tables, including using MySQL Admin.
+For point 3 see below:
-There are various ways of repairing tables, including using MySQL Admin.
+The error message states that the "directory not empty" and gives the path to that directory. If you go there with an FTP program you can see what is there and clean up. It could be just some empty subfolders that were leftover. Deleting these has been able to help. One can also delete the dir "moodledata/temp/backup" completely. That can take a bit longer but may solve several problems at once.
-==XML error: not well-formed (invalid token) at line YYYY==
+==The process ends with: "XML error: not well-formed (invalid token) at line YYYY". What can I do?==
 This problem can appear at any point in the restore process. It's caused when the XML parser detects something incorrect in the backup file that prevent correct operation. Usually, it's caused by some "illegal" characters added in the original course due to some copy/paste of text containing them (control characters, or invalid sequences...).
@@ Line 77: / Line 63: @@
 * Restore the course. It should work now.
-* Restore still not working? See the next paragraph.
+* Restore still not working? See the next question.
 Also, if possible, it's highly recommended to solve those problems in the original course too from Moodle itself. Once "repaired" there, problems will be out if you create new backup files in the future.
-==Still XML error: how to clean xml borked file==
+==The process ends with: "moodle xml not found at root level of zip file". What can I do?==
+If you are restoring from a zip file backup make sure the moodle.xml file is at the root level. To ensure this:
+#Unzip the backup file of the course (example: mycourse.zip)
+#Once the file is unzipped, open the folder (example: mycourse).
+#Select the folders within the mycourse folder AND the moodle.xml file and create a zip of those item (example: mycourse_new.zip)
+#Upload the new zip file (example: mycourse_new.zip) and restore from that.
+If the backup file is guaranteed to be correct, check paths to external files (zip, unzip). Incorrect settings also lead to this error message (see the Using Moodle forum discussion [http://moodle.org/mod/forum/discuss.php?d=140355 moodle.xml not found in root...] and MDL-14812).
+==The process ends with: "An error occurred while copying the zip file..."==
+This problem is most likely caused by a permissions issue in the destination directory. Backup files are copied to "XXX/backupdata" under your dataroot directory (where XXX is the id of the course being backed up).
+The problem could also be caused by a disk being full, though this is far less likely.
+To obtain precise information about what's happening, you can enable debug messages in ''Administration > Server > [[Debugging]]'' (select the maximum level - DEVELOPER) and/or check the web server error logs.
+==I Still get an XML error. How can I clean the borked XML file?==
-In some cases XML backup files may contain characters causing the restore process to abort, even after the steps described in the previous paragraph. In such cases you may want to try the following:
+In some cases XML backup files may contain characters causing the restore process to abort, even after the steps described in the previous question. In such cases you may want to try the following:
-* Download the [http://repository.atlassian.com/atlassian-xml-cleaner/jars/atlassian-xml-cleaner-0.1.jar Atlassian XML Cleanr Utility] from the [http://confluence.atlassian.com/display/JIRA/Removing+invalid+characters+from+XML+backups JIRA Atlassian site].
+* Download the [http://repository.atlassian.com/atlassian-xml-cleaner/jars/atlassian-xml-cleaner-0.1.jar Atlassian XML Cleaner Utility] from the [http://confluence.atlassian.com/display/JIRA/Removing+invalid+characters+from+XML+backups JIRA Atlassian site].
 * Unzip the problematic Moodle backup file under one empty folder. Moodle will create the course file folders as long as the unclean moodle.xml file. Please unzip using the Moodle unzip feature.
@@ Line 91: / Line 94: @@
 * Rename the unclean moodle.xml file to moodle-unclean.xml.
-* If you don't have access to your Moodle server's command prompt, using the Moodle zip feature, zip the moodle-unclean.xml file only, download the zip file locally and unzip it. It is very important to download the xml file in zipped format to avoid unwanted charachter encoding when transferring from an operating system to another.
+* If you don't have access to your Moodle server's command prompt, using the Moodle zip feature, zip the moodle-unclean.xml file only, download the zip file locally and unzip it. It is very important to download the xml file in zipped format to avoid unwanted character encoding when transferring from an operating system to another.
 * Move the downloaded Atlassian XML Cleaner Utility in the same folder where is your moodle-unclean.xml file.
@@ Line 105: / Line 108: @@
 * Restore the course. It should work now.
-==Some of your courses weren't saved!!==
+==What does "Some of your courses weren't saved!!" mean?==
 There are three possible causes of this problem:
@@ Line 112: / Line 115: @@
 # Skipped - this happens when a course is unavailable to students and has not been changed in the last month (31 days). This isn't an error situation - it's a feature, especially useful for sites with many unavailable old courses, saving process time.
-==Restoring pre 1.6 non-ISO-8859-1 backups to Moodle 1.6 - Unicode==
+==Why are some courses being skipped?==
-Any backup files with contents which are not 100% ISO-8859-1 will be a problem to restore to Moodle 1.6 (and upwards) running under Unicode. Instead, please try the following:
+Course backups automatically skip courses which are unavailable to students and have not been changed in the last month.
-# Make a clean install of Moodle 1.5.x (the latest version available)
+The [http://moodle.org/mod/forum/discuss.php?d=80367#p714733 Hide courses excluded from automatic backup] discussion in the Using Moodle forum describes a way that you can disable or override this thirty day check. This could be useful if, for example, you wanted to change the period from thirty days to six months.
-# Restore all your courses there (they should work if they were working originally)
-# Upgrade your site to Moodle 1.6 and run the UTF-8 migration script
-# Backup your courses again
-This will produce a new set of backup files that will be 100% UTF-8 and you will be able to use them with Moodle 1.6 without any problems.
+==Why does restore stop, rather than completing?==
-==Why are some courses being skipped?==
+Attempting to restore a course to an older version of Moodle than the one the course was backed up on can result in the restore process failing to complete. To ensure a successful restore, make sure that the version of Moodle you are restoring the course to is the same, or newer, than the one the course was backed up on.
+If it stop unexpectedly with no errors shown try again with [[Debugging]] switched on. Any errors you now see can help experts in the support forums diagnose your problem. You can also check the discussion links in the See also section below for further advice.
+==Restore stops with the message "Trying to restore user xxxx from backup file will cause conflict"==
+[[Image:Moodle 2.0 Restore breaks.png|thumb|Error message in Moodle 2.0]]
+This message is displayed when:
+# Your target site has one xxxx user (xxxx being the username)
+# The backup being restored also has one xxxx user
+# After various comparisons, Moodle has detected that the xxxx user in 1. and the xxxx user in 2. aren't the same user.
+If 1, 2 and 3 above happens, restore stops before performing any action as far as restoring that user will end associating "things" (create activities, posts, attempts... whatever...) to the xxxx user in the target site, when those "things" weren't performed by the same user at all.
+Those checks and behaviour were introduced in Moodle 1.9.x and continue being valid under 2.0. Usually the xxxx user in the messages is the "admin" user (that exists in practically all Moodle installations).
+There are two possible solutions to make both xxxx users in 1 and 2 to match (and avoid the conflict):
+a) Modify the backup's '''users.xml''' file and make the ''email'' or ''firstaccess'' fields match with the ones in target site.<br />
+b) Modify the '''target site''' and set the user ''email'' or ''firstaccess'' fields to match with the ones in backup's user.xml file.
+Method a) is recommended so the restore process will match both xxxx users and every "thing" in the backup file belonging to xxxx will be associated to the already existing xxxx user in the target site.
-From 1.6 onwards, course backups automatically skip courses which are unavailable to students AND have not been changed in the last month.
+  '''NOTE:''' When using method a) be aware that the ''moodle-filename-backup.'''mbz''''' is a zip file and can be renamed to ''moodle-filename-backup.'''zip''''' and unzipped.
+  When editing is complete, rezip and then rename using the original file name with the "*.mbz" extention.
-== See also ==
+==Why are certain course links broken in a restored course?==
-*Using Moodle [http://moodle.org/mod/forum/view.php?f=128 Backup and Restore forum]
+Inter-activity links must be absolute (full) URLs e.g. <code><nowiki>http://site.com/mod/resource/view.php?id=xxx</nowiki></code> in order to be processed properly during backup and restore.
-*[[Backup restore]] Version 1.6 description of screens for both
-*[[Restore]] almost same as backup_restore
-*[[Roll courses forward]] new Version 1.7 option
-*Using Moodle [http://moodle.org/mod/forum/discuss.php?d=66708 Scheduled backup fails] forum discussion including possible solution to "An error occured while copying files".
-== External links ==
+Any relative URLs e.g. <code>/mod/resource/view.php?id=xxx</code>, <code>../resource/view.php?id=xxx</code> or <code>view.php?id=xxx</code> will result in broken links when the course is restored.
-*[http://www.databasejournal.com/features/mysql/article.php/10897_3300511_2 Repairing Database Corruption in MySQL]
+== See also ==
-==See also==
+*Using Moodle [http://moodle.org/mod/forum/view.php?f=128 Backup and Restore forum]
+*[http://www.databasejournal.com/features/mysql/article.php/10897_3300511_2 databasejournal.com article on repairing database corruption in MySQL]
+* [[Site backup]]
+* [[Moodle migration]]
+* [[Beginning Moodle 2.0 Administration]]
-*[http://youtube.com/watch?v=ufAmf_jm_p8 How to backup a whole Moodle site video]
+Using Moodle forum discussions:
+*[http://moodle.org/mod/forum/discuss.php?d=142720 Trying to restore user 'admin' from backup file will cause conflict]
+*[http://moodle.org/mod/forum/discuss.php?d=167471 Where is the Moodle 2.0 "Course Backup Filearea"?]
 [[Category:FAQ]]
-[[Category:Backup]]
 [[es:FAQ Backup]]
+[[pl:Backup FAQ]]
 [[fr:FAQ de sauvegarde]]
 [[ja:バックアップFAQ]]
+[[pt:FAQ sobre cópias de segurança]]

Documentation