Development:Backup 2.0

Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.

Template:Moodle 2.0

Drop in ideas

• Support incremental backups & restore.
• Support 1-activity backup.
• Backup/restore one topic only (not just one activity) --Samuli Karevaara 21:29, 5 March 2009 (CST)
• Support anonymisation of personal data on backup.
• Separate process and progress. (Perhaps these classes I wrote can help)--Tim Hunt 19:41, 1 March 2009 (CST)
• XML format doesn't need to be radical different (IMO).
• Support restore of old (1.9 only?) backups.
• Hook in backup & restore to intercept & support other formats (BB, IMS-CC...)
• One code base both for manual and scheduled backup
• Scheduled restore
• Fileless export/import
• Export/import over mnet
• OOP.
• Secure (don't handle non-allowed data, user matching on restore, salted passwords..)
• Safe (some sort of "rollback" on failure - requires to have everything annotated somewhere - backup_ids or so).
• file-less backups - if you have separate backups, import/export etc.
• Allow to backup one category of courses (at the same time, one zip per course). MDL-17187
• Allow to mark courses to be excluded from scheduled backup manually.
• Avoid anti-timeout output when not running from browser. MDL-17282
• Review related caps, cleaning them and adding missing bits, improving sec. consistency.
• Log all backups (both manual and scheduled). Improve logging in general.

General prerequisites

Improve XML parsing

One of the major bottlenecks in backup and, specially, in restore reliability under Moodle 1.x has been the big amount of memory needed to handle those operations (see bugs like MDL-14302, MDL-15489, MDL-9838... and many others). While the whole XML file (moodle.xml) is parsed in a SAX way (hence, "streamed" and requiring small amounts of memory), we use to group some XML contents into "parts" in order to delegate the operations over those "parts" to different plugins (modules, blocks...). And problems arrive when some of those "parts" is big enough and processing them with the xml_parse_into_struct() function and the xmlize library uses exaggerated amounts of memory (for example, to process a 12.5MB file requires 311MB of memory, crazy!). So we need to switch to an alternate method to parse those "parts" using much less memory and to build the corresponding in-memory object with one acceptable throughput (speed).