View source for Perth Hackfest October 2012/Backup

{{Template:WillNotMigrate}}

* Asynchronous backup + restore ('''Important''')
** want to see a schedule list of what is due to run and progress bar, start/end times, email completion success/failure
** queueing system, dispatch many or just one
** use event api? can still be bad if next cron run has a ton of backups to run
*** elis has improved cron system that can prevent long running cron tasks and provides scheduling
*** Alternative, have processes be able to run outside of cron, so that system admins can run their own cron tools
**** Problem with dependencies (e.g. enrollments update before forum emails)
**** cron script to take an argument for which bit of cron to run (OU has something similar already for dealing with previous failures)
**** need to log lastcronrun for everything
* Large backups end up timing out
* run backup on a separate webserver than user facing
* configuration option to disable temporarily (for busy times)
** 
*** maybe by capabilities so having separate set of capabilities for asynch backup to live backup so you can enable/disable each separately for different sets of users?
* Restore of M19 user data
** Would take an entire 6 month release cycle to implement it. Not priority at all. 
** There are current workarounds
* Blue Sky
** Run backup/restore on separate server
** Ability to backup course before a delete
*** On delete, a flag is set and then course is backed up using automated system and then deleted.
*** Or just have flag with option to really bulk delete courses with that flag 
*** While flagged delete need care over unique ids which will have to be munged like user email etc
** when restoring to same site, avoid using file pool as file i/o is expensive
** Display potential size of backup file before actual backup to implement backup file size limit
* Improved UI for backup/restore
** OU build one click backup/restore solution
** Backup profiles e.g ‘standard with userdata’, ‘standard without userdata’?
** admin configurable default settings
*** This already exists at “Site administration ▶ Courses ▶ Backups ▶ General backup defaults”
* Improving Development of custom code:
** e.g. https://moodle.org/mod/forum/discuss.php?d=212549

=== Decisions ===
* Now (2.5)
** Adding hooks for asynchronous backup/restore (e.g. events, etc)
** Asynchronous backup/restore plugin
*** To disable existing backup/restore system, remove existing capabilities
*** Would need some hooks into core
*** Capability to do instant backup/restore or only queued
*** '''Spec contributors: '''Kris Stokking, Mike Churchward, Ashley Holman, Matt Sharpe
* Soon (2.6)
** Improved Backup/Restore UI
*** Profiles
*** One step process
*** Backup size
*** '''Spec contributors: '''Max Kan, Justin Filip, Kris Stokking, Loic Jeannin
* Later (2.7+)
** Asynchronous backup/restore core
** “Deleted” flags / Recycle bin?
*** '''Spec contributors: '''Shane Elliott, Andrew Nicols, Max Kan
** 

== Decisions ==
<nowiki>* Local plugin</nowiki>

<nowiki>* Where do we need to use this? Is it just in the “Backup/Restore” menu (from the settings block)?</nowiki>

<nowiki>* cli interface to allow separate cron schuduler to pick jobs off queue - should be able to set up multiple pickups to multi threading </nowiki>

<nowiki>* capability controls: addtoqueue</nowiki>

<nowiki>* desire to use observer pattern for monitoring the progress</nowiki>

<nowiki>* Core changes - add a plugin type for “backup executors”</nowiki>

<nowiki>* queue needs to know who made request, all prefs chosen. Need to check may all be in serialized backup controller</nowiki>

<nowiki>* jobs in queue should be able to be deleted / aborted</nowiki>

<nowiki>* The plugin should allow parallel backup/restore (no core changes needed - could be handled in the plugin)</nowiki>

== How to implement this: ==
=== Required ===
* Admin Tool Plugin (Spec required)
* Option 1
** Hides backup/restore links in settings block
** Add’s it’s own backup/restore links
** Recreate the backup/restore UI by re-using the existing UI classes
** At the end of the process - add the backup/restore controller to it’s own queue table
** Provide links to it’s own view pages to show the status of the current backup/restore processes
* Option 2
** Use existing UI and add hooks to run a plugin at the end of the backup/restore process (when added to core?)
* Queue processing
** Cli driven - run “automated” backups or “user” backups
** Cron starting workers - but not waiting for them to finish? Easier to just run separate CLI - can run on separate server.
* Plugin UI Questions
** Can 2 users queue backups for the same course? Yes - but not with all the same settings (? hash of all settings?)

'''Spec 2.6'''

'''UI Improvements for Backup/Restore'''

Decisions

* Just go with two hard coded profiles, with user data and without user data.
* There is too few setting that can be saved in a profile to make it worthwhile. Most settings are specific to course and depend on activities.
* Not necessary to display the estimate backup size before the backup start

''Backup UI improvements''

* Backup option 1: Backup everythings
* Backup option 2: Backup everythings except user data
* Backup option 3: Advance backup options

''Restore UI improvements''

* Selection UI to select an option to restore into a new course, or merging into a new course should be simplified. 
* Show three options in first screen “restore into a new course”, “merge”, “replace” and based on the button clicked show only the requested form on the next screen.
* The form discussed above needs error messages.
* One click restore all data with default settings (Skip 3 screens in restore UI).

'''Action Items: '''

Damyon - Convert this to wiki spec and continue discussion