Note: You are currently viewing documentation for Moodle 3.11. Up-to-date documentation for the latest stable version of Moodle may be available here: Migrations.

Migrations: Difference between revisions

From MoodleDocs
No edit summary
(Re-added "Import of legacy" section)
 
(13 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Workplace}}
{{Workplace}}
Moodle Workplace provides a powerful '''Migration''' tool that lets you copy data and elements between tenants and sites. It also lets you import and export data from external systems.
== Overview ==
== Overview ==
This tool, through an intuitive step-by-step process, offers the capacity to export different parts of a Moodle Workplace instance and import them the same or a different site.
The migration tool can export various parts of your Moodle Workplace instance and import them into the same site or different sites.
The following diagram shows a high-level migration overview, visualising the typical workflows of exporting and importing Moodle Workplace data:
[[File:Migration - Overview schema.png|border|center|frameless|900x900px|alt=]]
The [[Migrations#Migration Export|export process]] contains the following key steps:
# [[Migrations#Step 1 - General settings|Selecting the exporter]]: Initiating the respective wizard of the selected exporter
# [[Migrations#Step 2 - Options|Export options]]: This step covers two parts, namely '''content''' (what data must be exported) and '''instances''' (which elements should be exported)
# [[Migrations#Step 3 - Review|Review and export]]: Executing/scheduling the actual export and creating the export file
The [[Migrations#Migration Import|import process]] contains the following steps:
# [[Migrations#Step 1 - Select source|Selecting a source]]: Choosing the import file you will be dealing with.
# [[Migrations#Step 2 - General settings|Selecting a tenant]]: Choosing the tenant where the data must be imported to.
# [[Migrations#Step 3 - Options|Import options]]: This step covers two parts: '''content''' (what data must be imported) and '''instances''' (which elements should be imported).
# [[Migrations#Step 4 - Conflicts|Conflict resolution]]: If the import file contains any inconsistencies, manual intervention is required.
# [[Migrations#Step 5 - Review|Review and importing]]: Executing/scheduling the actual import.
Migration exporters and importers can be chained together – that way, import and export workflows can be created. For example, the tenant exporter makes use of all other exporters through internal cascading.


[[File:Migration.gif]]
Third-party plugins are fully supported, if the activity supports Moodle's backup and restore mechanism.
== Export ==
== Migration Export ==
[[File:Export.gif]]
You can access the migration tool via '''Site administration > Migration''' or directly via the '''Migration''' icon in the Workplace launcher. On the '''Exports''' tab, you will see a list of any exports, which contains the following columns:
* '''Date''' (filter): Date and time the export was created
* '''Created by''': User account that initiated the export
* '''Exporter''' (filter): Type of exporter
* '''Tenant''' (filter): Name of the tenant that the export was created for
* '''Size''': Size of the export file
* '''Status''' (filter): Scheduled / In progress / Success / Error
* '''Actions''':
** '''New import from this file''': Use the export file as input and start the migration importer. This will take you directly to [[Migrations#Step 2 - General settings|Step 2]] of the import process.
** '''Download''': Download the export file
** '''View export''': Show a status report that includes a status or error log
** '''Delete''': Remove the export file
[[File:Migration - Exports.png|border|center|frameless|900x900px|alt=]]
To create a new migration export, select the '''+ Export''' button on the '''Exports''' tab. You will then be guided through the 3-step export process.
=== Step 1 - General settings ===
=== Step 1 - General settings ===
Select which exporter you want to use from the options displayed. On the following steps, you will be able to narrow down your selection and specify which elements you want to export.
The first step of the migration process is to choose an exporter.
[[File:Migration - Export Step 1.png|border|center|frameless|900x900px|alt=]]
Some exporters offer a '''Site / Current tenant''' switch, which allows you to define whether you want to export entities from the whole site, or only those associated with the current tenant.
{| class="wikitable"
|'''Exporter'''
|'''Site / Current tenant switch'''
| colspan="2" |'''Description'''
|-
|'''Certificates'''
|Yes
| colspan="2" |Issued certificates and templates
|-
|'''Certifications'''
|No
| colspan="2" |Certifications with their associated programs, courses, user allocations and component dynamic rules
|-
|'''Cohorts'''
|Yes
| colspan="2" |Cohorts, including cohort members without user data
|-
|'''Course categories'''
|Yes
| colspan="2" |Course categories and subcategories
|-
|'''Courses'''
|Yes
| colspan="2" |Courses without user data, using [https://docs.moodle.org/311/en/Course_backup#General_backup_defaults default course backup configuration] (see details below table)
|-
|'''Custom reports'''
|No
| colspan="2" |Custom reports, including audience and schedule data
|-
|'''Dynamic rules'''
|No
| colspan="2" |Dynamic rules definition, conditions and actions
|-
|'''Organisation structure frameworks'''
|No
| colspan="2" |Frameworks with the whole hierarchy for departments and/or positions
|-
|'''Organisation structure jobs'''
|No
| colspan="2" |Jobs with their associated department and position frameworks
|-
|'''Programs'''
|No
| colspan="2" |Programs with their courses, user allocations and component dynamic rules
|-
|'''Site'''
|No
| colspan="2" |Full site content including all tenants along with all entities contained within them. Course backups will include all content and user data except logs.
|-
|'''Tenants'''
|Yes
| colspan="2" |Tenants along with all entities contained within them
|-
|'''Users'''
|Yes
| colspan="2" |Site and tenant users
|}
When courses are included in the export, the following default course backup configuration settings are taken into account:
* Include filters
* Include groups and groupings


[[File:export - step 1.png]]
 
When all course content is exported, for example, when exporting the entire site or a tenant, the following default course backup configuration settings are also taken into account:
* Include users
* Include role assignments
* Include activities and resources
* Include blocks
* Include comments
* Include badges
* Include calendar events
* Include user completion information
* Include histories
* Include question bank
* Include competencies
* Include content bank content
=== Step 2 - Options ===
=== Step 2 - Options ===
Specify which elements you want to export by selecting them. Exporters can be slightly different from each other.
During the second step, you can narrow down your selection and specify which elements you want to export. The exporter options always contain the following two elements (except when exporting the entire site):
==== Content ====
* '''Content''': Which attributes and settings may or may not be included in the export file.
Attributes and settings that may or not be included in the exported file.
* '''Instances''': Which elements will be included in the export file.
==== Instances ====
The content and instances available depend on the chosen exporter. Details about each setting are displayed in the balloon help indicated by a question mark.
Select which instances will be included in the exported file.
[[File:Migration - Export Step 2.png|border|center|frameless|900x900px|alt=]]
The table below lists all '''Content''' and '''Instances''' settings (locked entries in italics) for each exporter plugin.
{| class="wikitable"
|'''Exporter'''
|'''Content'''
| colspan="2" |'''Instances'''
|-
|'''Certificates'''
|''Certificate template details''
Issued certificates
| colspan="2" |Select all certificate templates
Select categories and subcategories manually…
 
Select certificate templates manually...
|-
|'''Certifications'''
|''Settings''
Associated programs
 
Course structure
 
Include course content
 
Certification user allocations
 
Dynamic rules
| colspan="2" |Select all active certifications
Select active and archived certifications
 
Select manually…
 
Include shared entities
|-
|'''Cohorts'''
|''Cohort details''
Cohort members
| colspan="2" |All cohorts
All system cohorts
 
Select categories and subcategories manually...
 
Select manually...
|-
|'''Course categories'''
|''Course category details''
Course structure
 
Include course content
 
Certificate templates
 
Cohort details including members
| colspan="2" |Select categories and subcategories manually
|-
|'''Courses'''
|''Course structure''
Include course content
| colspan="2" |Select categories and subcategories manually…
Select courses manually...
|-
|'''Custom reports'''
|''Report definitions''
Audiences
 
Schedules
| colspan="2" |Export all custom reports
Export specific custom reports...
|-
|'''Dynamic rules'''
|''Rule definitions, conditions and actions''
| colspan="2" |Select all Dynamic rules (excluding archived)
Select all enabled Dynamic rules
 
Select all Dynamic rules (including archived)
|-
|'''Organisation structure frameworks'''
|''Descriptions, hierarchy and permissions''
| colspan="2" |Select all department and position frameworks
Select all department frameworks
 
Select all position frameworks
 
Select manually...
 
Include shared entities
|-
|'''Organisation structure jobs'''
|''Job assignments''
Department and position frameworks
| colspan="2" |Select all active jobs
Select all active and past jobs
 
Select all jobs in any of the selected frameworks...
|-
|'''Programs'''
|''Settings''
Course structure
 
Include course content
 
Program user allocations
 
Dynamic rules
| colspan="2" |Select all active programs
Select all active and archived programs
 
Select manually...
 
Include shared entities
|-
|'''Site'''
|''Details''
''Appearance''
 
''Users''
 
''Content: courses, categories, programs and certifications''
 
Cohort details including members
 
Certificate templates
 
Organisation structure
 
Dynamic rules
 
Custom reports
| colspan="2" |
|-
|'''Tenants'''
|''Details''
Appearance
 
Users
 
Course categories, with cohorts and course structure
 
Include course content
 
Certificate templates
 
Programs
 
Certifications
 
Organisation structure
 
Dynamic rules
 
Custom reports
| colspan="2" |Select all tenants (excluding archived)
Select all tenants (including archived)
 
Select tenants manually...
|-
|'''Users'''
|''User profiles''
User pictures


[[File:export - step 2 - certifications.png]]
Include suspended users
| colspan="2" |Select all users
Select users manually...
|}
=== Step 3 - Review ===
=== Step 3 - Review ===
In this page, you can check at a glance if everything is correct before exporting the file.
During the third and final step, you need to review and execute the export process. You will be presented with a summary of the selected and unselected options for content and instances.
The export can only be started when the resulting file will be non-empty; that is, at least one instance has been selected. For instance, if you choose a course category that does not contain any courses, the Export button will remain deactivated. Finally, you have to confirm the execution of the export.
[[File:Migration - Export Step 3.png|border|center|frameless|900x900px|alt=]]
Once the export has been initiated, its status will be set to '''Scheduled'''. The export will commence on the subsequent [[Cron|cron]] execution. While the export file is being generated, the status will be changed to '''In progress'''. This might take some time on larger exports. When the export file is ready for download or import, the status changes again to '''Success'''. You will also receive a notification via the usual on-board mechanism. If an '''Error''' occurred during the export process, details would be displayed on the status page.
== Migration Import ==
You can access the migration tool via '''Site administration > Migration''' or directly via the '''Migration''' icon in the Workplace launcher.
 
On the '''Imports''' tab, you will see a list of any imports which contains the following columns:
* '''Date''' (filter): Date and time the import was created
* '''Created by''': User account that initiated the import
* '''Importer''' (filter): Type of importer
* '''Tenant''' (filter): Name of the tenant that the import was created for
* '''Size''': Size of the import file
* '''Status''' (filter): Scheduled / In progress / Success / Error
* '''Actions''':
** '''Download''': Download the import file
** '''View import''': Show a status report that includes a status or error log
** '''Delete''': Remove the import file
[[File:Migration - Imports.png|border|center|frameless|900x900px|alt=]]
To create a new migration export, select the '''+ Import''' button on the '''Imports''' tab. You will then be guided through the 5-step import process.
=== Step 1 - Select source ===
In step 1, you must provide an input file: You will need to upload either a migration export file or a CSV file.
[[File:Migration - Import Step 1.png|border|center|frameless|900x900px|alt=]]
By default, the importer automatically detects what export type has been used before proceeding to step 2. In addition to the '''Detect automatically''' option, you can also select the '''File format''', choosing between '''Workplace format''' and '''CSV'''. When the latter is used, make sure the '''CSV delimiter''' and '''Encoding''' settings are correct. The CSV import format only supports departments and positions.
=== Step 2 - General settings ===
The second step differs depending on what type of import file you have uploaded.
==== Step 2a - General settings (Importing a file in Workplace format) ====
The following two sections are available when importing a file in Workplace format:
* '''About this file''': Metadata about the file and its content is displayed
* '''Destination''': Select the tenant the file has to be imported to. You can either opt for the current tenant or select a tenant from your site. This section is not available when you wish to import a tenant import file. For some import files, the option '''Available for all tenants''' is offered; that is, it will be imported into Shared space.
[[File:Migration - Import Step 2a.png|border|center|frameless|900x900px|alt=]]
==== Step 2b - General settings (Importing a file in CSV format) ====
The following three sections are available when importing a file in CSV format:
* '''About this file''': A preview of the first three rows is displayed, along with an option to show additional records.
* '''Select importer''': Moodle Workplace supports a '''Departments importer''' and a '''Positions importer'''. The formats of those files will be shown in [[Migrations#Step 3b - Options (Importing a file in CSV format)|Step 3b - Options]].
* '''Destination''': Select the tenant the file has to be imported to. You can either opt for the current tenant or select a tenant from your site.
[[File:Migration - Import Step 2b.png|border|center|frameless|900x900px|alt=]]
=== Step 3 - Options ===
During the third step, you can narrow down your selection and specify which elements you want to import.
==== Step 3a - Options (Importing a file in Workplace format) ====
When importing a file in Workplace format, the importer options always contains at least the following two elements (except when importing tenants or the entire site):
* '''Content''': Which attributes and settings may or may not be included when importing the file.
* '''Instances''': Which elements will be included when importing the file.
The content and instances available depend on the chosen importer. Details about each setting are displayed in the balloon help indicated by a question mark.
[[File:Migration - Import Step 3a.png|border|center|frameless|900x900px|alt=]]
The table below lists all '''Content''' and '''Instances''' settings (locked entries in italics) for each importer plugin.
{| class="wikitable"
|'''Importer'''
|'''Content'''
| colspan="2" |'''Instances'''
|-
|'''Certificates'''
|Certificate template details
Issued certificates
| colspan="2" |Select all certificate templates in this file
Select certificate templates manually…
 
Course category…
|-
|'''Certifications'''
|Settings
Associated programs


[[File:export - step 3 - certifications.png]]
Course backups excluding user data
=== Exporting statuses ===
After going through the step-by-step process, you’ll be redirected to the export report page where you can check, among other attributes, its status, size and date of creation.


[[File:export - success.png]]
Select course category...
==== Scheduled ====
An export set as “scheduled” will run on the next cron execution. (How much it takes to start will depend on the cron settings for your site). It’s safe to continue navigating on the site.
==== In progress ====
The export file is being generated.
==== Completed ====
The file is ready to be downloaded or imported again. More information about the file should be displayed at this point, and the user will receive a notification about it.
== Import ==
[[File:Import.gif]]


Certification user allocations (if available)


=== Step 1 - Select source ===
Dynamic rules
Upload a file containing either a Moodle Workplace export or a CSV file.
| colspan="2" |Select all certifications in this file
You can skip this step if you are importing a Moodle Workplace export from your current site: in this case, click on the button “New import from this file” directly from the “Export” tab.
Select manually...
|-
|'''Cohorts'''
|Cohort details
Cohort members
| colspan="2" |Select all cohorts in this file
Select manually...
 
Import all in system context
 
Import all in the selected category...
|-
|'''Course categories'''
|Course category details
Course backups excluding user data
 
Certificate templates (if available)
 
Cohort details including members (if available)
| colspan="2" |Select all course categories in this file
Select categories and subcategories manually…
 
Parent category...
|-
|'''Courses'''
|Course backups excluding user data
| colspan="2" |Select all courses in this file
Select courses manually…
 
Course category...
|-
|'''Custom reports'''
|Report definitions
Audiences
 
Schedules
| colspan="2" |Import all custom reports
Import specific custom reports...
|-
|'''Dynamic rules'''
|Rule definitions, conditions and actions
| colspan="2" |Select all Dynamic rules in this file
Select manually...
|-
|'''Organisation structure frameworks'''
|Descriptions, hierarchy and permissions
| colspan="2" |Select all department and position frameworks
Select all department frameworks
 
Select all position frameworks
 
Select manually...
|-
|'''Organisation structure jobs'''
|Job assignments
Department and position frameworks
| colspan="2" |Select all jobs in this file
Select all jobs from specific frameworks...
|-
|'''Programs'''
|Settings
Course backups excluding user data
 
Course category...
 
Dynamic rules
| colspan="2" |Select all the programs in this file
Select manually...
|-
|'''Site'''
|Details
Appearance
 
Users
 
Content: courses, categories, programs and certifications
 
Cohort details including members
 
Certificate templates
 
Organisation structure
 
Dynamic rules
 
Custom reports
| colspan="2" |
|-
|'''Tenants'''
|Details
Appearance
 
Users
 
Course categories, with cohorts and course structure
 
Certificate templates
 
Programs
 
Certifications
 
Organisation structure


[[File:import - step 1.png]]
Dynamic rules
=== Step 2 - General settings ===
Check if the file being imported is the correct one. Importers can be slightly different from each other, but most of them will offer options regarding tenant destination and date shifting. You will be able to specify which elements you want to import in the next step.


[[File:import - step 2 - organisation structure jobs.png]]
Custom reports
=== Step 3 - Options ===
| colspan="2" |Merge into existing tenant...
Again, these can vary depending on the importer, but these are the most common ones:
Create new tenants
==== Content ====
Select which attributes and settings available on the exported file will be imported.
==== Instances ====
Select which instances available on the exported file will be imported.


[[File:import - step 3 - organisation structure jobs.png]]
Select all tenants
=== Step 4 - Conflicts ===
Many times the import process will generate conflicts that will require some manual handling. If no conflict is found, this step will be skipped automatically. For each conflict found, the following elements will be displayed:
==== Instances ====
It indicates which instances, from the file being imported, are generating the conflict.
==== If affects ====
It shows all components that are being affected by the conflict generated by these instances.
==== Solution ====
It offers a list of different available options that will solve the conflict.


[[File:import - step 4 - organisation structure jobs.png]]
Select tenants manually...
=== Step 5 - Review ===
|-
Here, an overview of the import process is displayed. Check if everything is correct before importing the file.
|'''Users'''
|User profiles
User pictures
| colspan="2" |Select all users
Select users manually...
|}
==== Step 3b - Options (Importing a file in CSV format) ====
The following three sections exist when you're importing either departments or positions via CSV files:
* '''Position/Department framework''': Either create a new (generic) framework where a name will be provided automatically (Import <name of import file>) or select an existing framework.
* '''Hierarchy''': Positions and departments can either be arranged hierarchically or as a list. If the organisation structure is hierarchical, a position identifier must be provided, which will be used to identify the parent.
* '''Field mapping''': The import file contains the following fields:
** '''name''': Name of the position or department
** '''idnumber''': ID number of the position or department
** '''description''': Textual description of the position or department
** '''descriptionformat''': 1 = Moodle auto format (default), 2 = Plain text format, 3 = HTML format, 4 = Markdown format
** '''parent''': ID number of the parent. The mapping for this field cannot be the same as the position identifier provided in the hierarchy section.
** '''manager''' (positions only): Toggle whether the manager role permissions are all available (1) or not (0)
** '''departmentlead''' (positions only): Toggle whether the department lead role permissions are all available (1) or not (0)


[[File:import - step 5 - organisation structure jobs.png]]
=== Importing statuses ===
After going through the step-by-step process, you’ll be redirected to the import report page where you can check, among other attributes, its status, size and date of creation.


[[File:import - success.png]]
When the import file does not follow the notation laid out here, you have the option to select CSV columns manually.
==== Scheduled ====
[[File:Migration - Import Step 3b.png|border|center|frameless|900x900px|alt=]]
An import set as “scheduled” will run on the next cron execution. (How much it takes to start will depend on the cron settings for your site). It’s safe to continue navigating on the site.
Sample departments import file:<syntaxhighlight lang="text">
==== In progress ====
idnumber,parent,name,description
The content is being imported into its destination.
SAL,,Sales,Sales department
==== Completed ====
SAD,SAL,Sales development,Sales development department
The import process is over, and its content should be available on its destination. More information about the file should be displayed at this point, and the user will receive a notification about it.
CSC,SAL,Customer Success,Customer Success department
EDI,,Editorial,Editorial department
AND,EDI,Art and design,Art and design department
PRO,EDI,Production,Production department
RNE,EDI,Research and Education,Research and Education department
MED,RNE,Medical advising,Medical advising department
MHR,RNE,Mental Health research,Mental Health research department
ONR,RNE,Oncology research,Oncology research department
MKT,,"Marketing, Promotion and Advertising","Marketing, Promotion and Advertising department"
PUB,MKT,Publicity,Publicity department
MH,MKT,Mental Health,Mental Health department
ONC,MKT,Oncology,Oncology department
IT,,Information and Technology,Information and Technology department
SUP,IT,ICT Support,ICT Support department
ASP,IT,Applications support,Applications support department
ADE,IT,Applications development,Applications development department
INF,IT,Infraestructure,Infraestructure department
HR,,Human Resources,Human Resources department
LND,HR,Learning and Development,Learning and Development department
ADM,,Administration,Administration department
LNC,ADM,Legal and Contracts,Legal and Contracts department
SRI,LNC,Subsidiary Rights,Subsidiary Rights department
FNA,ADM,Finance and Accounting,Finance and Accounting department
</syntaxhighlight>Sample positions import file:<syntaxhighlight lang="text">
idnumber,parent,name,description,manager
CEO,,Chief Executive Officer,Chief Executive Officer position,1
COO,CEO,Chief Operating Officer,Chief Operating Officer position,1
HMD,COO,Head of Marketing and Design,Head of Marketing and Design position,1
DES,HMD,Designer,Designer position,0
MCO,HMD,Marketing Consultant,Marketing Consultant position,0
OM,COO,Office Manager,Office Manager position,0
PM,COO,Project Manager,Project Manager position,1
EDI,PM,Editor,Editor position,0
MEW,PM,Medical Writer,Medical Writer position,0
CTO,CEO,Chief Technology Officer,Chief Technology Officer position,1
LDE,CTO,Lead Developer,Lead Developer position,1
DEV,LDE,Developer,Developer position,0
LDO,CTO,Lead DevOps,Lead DevOps position,1
DEO,LDO,DevOps,DevOps position,0
SA,CTO,Sysadmin,Sysadmin position,1
OPE,SA,Operator,Operator position,0
CFO,CEO,Chief Financial Officer,Chief Financial Officer position,1
AC,CFO,Accountant,Accountant position,0
DPO,CFO,Data Privacy Officer,Data Privacy Officer position,0
LEA,CFO,Legal advisor,Legal advisor position,0
HHR,CFO,HR Manager,HR Manager position,1
ODA,HHR,Organisational Development advisor,Organisational Development advisor position,0
CCO,CEO,Chief Commercial Officer,Chief Commercial Officer position,1
AMG,CCO,Account Manager,Account Manager position,0
SAR,CCO,Sales Representative,Sales Representative position,0
</syntaxhighlight>
=== Step 4 - Conflicts ===
If the import file contains any inconsistencies that the migration tool cannot resolve automatically, the conflict resolution screen will be shown, and action must be taken.
[[File:Migration - Import Step 4.png|border|center|frameless|900x900px|alt=]]
Typical conflicts are that a record already exists (for instance, the username or an organisation ID) or an associated user account does not exist. Depending on the conflict, two types of resolution mechanisms are offered: either a solution is provided, or the record has to be skipped.
=== Step 5 - Review ===
During the fifth and final step, you need to review and execute the import process. You will be presented with a summary of the content that will be imported.
[[File:Migration - Import Step 5.png|border|center|frameless|900x900px|alt=]]
Once the import has been initiated, its status will be set to '''Scheduled'''. The import will commence on the subsequent [[Cron|cron]] execution. While the import is being processed, the status will be changed to '''In progress'''. This might take some time on larger imports. When the import is completed, the status changes again to '''Success'''. You will also receive a notification via the usual on-board mechanism. If an '''Error''' occurred during the import process, details would be displayed on the status page.
== Command-line interface ==
== Command-line interface ==
The following commands will allow you to perform export and import from CLI:
The following commands will allow you to perform export and import from CLI:
<pre>
php admin/tool/wp/cli/export.php
php admin/tool/wp/cli/import.php
</pre>
Both commands have help that explains how to use them. They also have interactive mode that allows to choose some options and display available options for exporters and importer.


Exporter/importer-specific parameters are currently passed as JSON-encoded string. In the future versions we will be working on internationalisation of the CLI (translating dialogues) and more interactivity in the exporter/importer settings.
<code>php admin/tool/wp/cli/export.php</code>
 
<code>php admin/tool/wp/cli/import.php</code>
 
Use the <code>--help</code> parameter to display all the available options, alongside a short description of each parameter. Exporter and importer-specific parameters are passed as JSON-encoded strings.
 
The two migration CLIs support interactive and non-interactive modes.
 
During '''interactive mode''', you will be asked various questions, such as the program name to be exported, and you have to confirm actions, such as the usage of default values. This is particularly useful during one-off operations and when testing new processes.
 
In '''non-interactive mode''', no questions are asked, no confirmations are needed, and default values are used. It is intended for usage in scripts to automate regular processes fully.
 
When run in interactive mode, the non-interactive counterpart, that is, the valid command will be displayed. That way, you can create your command step by step and, once completed, use it in your shell scripts.


[[File:workplace_migration_cli.mov|600px]]
[[File:workplace_migration_cli.mov|600px]]
==Import of legacy data==
[https://docs.moodle.org/dev/Moodle_Workplace_3.11_release_notes#3.11 Moodle Workplace 3.11] introduced some changes in the audience and schedules for Report Builder to support more types of audiences and link schedules to existing audiences. Due to these improvements, audiences and schedules from versions prior to 3.11 need to be upgraded following this process:
*'''Audiences''': report audiences based on job department/position will be converted to the new "job" audience type.
*'''Schedules''': New audiences will be automatically created for previously defined report schedule recipients for job department, position and manually added users. Each of these audiences will automatically be added to the new schedule as recipients.
Note <b>recipient emails are no longer supported in schedules</b>. Where previous schedules used these, the person who created the schedule will receive an email notifying them of any changes made. The email will include a list of previous email recipients, with a link to this page. As an alternative to external email addresses, consider creating users with the [[No login]] authentication method.
The same process will be followed when [[Migrations#Report_Builder_audience_and_schedules_prior_to_3.11|importing Report Builder audience and schedules prior to 3.11 using the Migration tool]].

Latest revision as of 10:19, 26 July 2022

workplacelogo.png This feature is part of Moodle Workplace™, which is available through Moodle Partners only.

Moodle Workplace provides a powerful Migration tool that lets you copy data and elements between tenants and sites. It also lets you import and export data from external systems.

Overview

The migration tool can export various parts of your Moodle Workplace instance and import them into the same site or different sites. The following diagram shows a high-level migration overview, visualising the typical workflows of exporting and importing Moodle Workplace data:

The export process contains the following key steps:

  1. Selecting the exporter: Initiating the respective wizard of the selected exporter
  2. Export options: This step covers two parts, namely content (what data must be exported) and instances (which elements should be exported)
  3. Review and export: Executing/scheduling the actual export and creating the export file

The import process contains the following steps:

  1. Selecting a source: Choosing the import file you will be dealing with.
  2. Selecting a tenant: Choosing the tenant where the data must be imported to.
  3. Import options: This step covers two parts: content (what data must be imported) and instances (which elements should be imported).
  4. Conflict resolution: If the import file contains any inconsistencies, manual intervention is required.
  5. Review and importing: Executing/scheduling the actual import.

Migration exporters and importers can be chained together – that way, import and export workflows can be created. For example, the tenant exporter makes use of all other exporters through internal cascading.

Third-party plugins are fully supported, if the activity supports Moodle's backup and restore mechanism.

Migration Export

You can access the migration tool via Site administration > Migration or directly via the Migration icon in the Workplace launcher. On the Exports tab, you will see a list of any exports, which contains the following columns:

  • Date (filter): Date and time the export was created
  • Created by: User account that initiated the export
  • Exporter (filter): Type of exporter
  • Tenant (filter): Name of the tenant that the export was created for
  • Size: Size of the export file
  • Status (filter): Scheduled / In progress / Success / Error
  • Actions:
    • New import from this file: Use the export file as input and start the migration importer. This will take you directly to Step 2 of the import process.
    • Download: Download the export file
    • View export: Show a status report that includes a status or error log
    • Delete: Remove the export file

To create a new migration export, select the + Export button on the Exports tab. You will then be guided through the 3-step export process.

Step 1 - General settings

The first step of the migration process is to choose an exporter.

Some exporters offer a Site / Current tenant switch, which allows you to define whether you want to export entities from the whole site, or only those associated with the current tenant.

Exporter Site / Current tenant switch Description
Certificates Yes Issued certificates and templates
Certifications No Certifications with their associated programs, courses, user allocations and component dynamic rules
Cohorts Yes Cohorts, including cohort members without user data
Course categories Yes Course categories and subcategories
Courses Yes Courses without user data, using default course backup configuration (see details below table)
Custom reports No Custom reports, including audience and schedule data
Dynamic rules No Dynamic rules definition, conditions and actions
Organisation structure frameworks No Frameworks with the whole hierarchy for departments and/or positions
Organisation structure jobs No Jobs with their associated department and position frameworks
Programs No Programs with their courses, user allocations and component dynamic rules
Site No Full site content including all tenants along with all entities contained within them. Course backups will include all content and user data except logs.
Tenants Yes Tenants along with all entities contained within them
Users Yes Site and tenant users

When courses are included in the export, the following default course backup configuration settings are taken into account:

  • Include filters
  • Include groups and groupings


When all course content is exported, for example, when exporting the entire site or a tenant, the following default course backup configuration settings are also taken into account:

  • Include users
  • Include role assignments
  • Include activities and resources
  • Include blocks
  • Include comments
  • Include badges
  • Include calendar events
  • Include user completion information
  • Include histories
  • Include question bank
  • Include competencies
  • Include content bank content

Step 2 - Options

During the second step, you can narrow down your selection and specify which elements you want to export. The exporter options always contain the following two elements (except when exporting the entire site):

  • Content: Which attributes and settings may or may not be included in the export file.
  • Instances: Which elements will be included in the export file.

The content and instances available depend on the chosen exporter. Details about each setting are displayed in the balloon help indicated by a question mark.

The table below lists all Content and Instances settings (locked entries in italics) for each exporter plugin.

Exporter Content Instances
Certificates Certificate template details

Issued certificates

Select all certificate templates

Select categories and subcategories manually…

Select certificate templates manually...

Certifications Settings

Associated programs

Course structure

Include course content

Certification user allocations

Dynamic rules

Select all active certifications

Select active and archived certifications

Select manually…

Include shared entities

Cohorts Cohort details

Cohort members

All cohorts

All system cohorts

Select categories and subcategories manually...

Select manually...

Course categories Course category details

Course structure

Include course content

Certificate templates

Cohort details including members

Select categories and subcategories manually
Courses Course structure

Include course content

Select categories and subcategories manually…

Select courses manually...

Custom reports Report definitions

Audiences

Schedules

Export all custom reports

Export specific custom reports...

Dynamic rules Rule definitions, conditions and actions Select all Dynamic rules (excluding archived)

Select all enabled Dynamic rules

Select all Dynamic rules (including archived)

Organisation structure frameworks Descriptions, hierarchy and permissions Select all department and position frameworks

Select all department frameworks

Select all position frameworks

Select manually...

Include shared entities

Organisation structure jobs Job assignments

Department and position frameworks

Select all active jobs

Select all active and past jobs

Select all jobs in any of the selected frameworks...

Programs Settings

Course structure

Include course content

Program user allocations

Dynamic rules

Select all active programs

Select all active and archived programs

Select manually...

Include shared entities

Site Details

Appearance

Users

Content: courses, categories, programs and certifications

Cohort details including members

Certificate templates

Organisation structure

Dynamic rules

Custom reports

Tenants Details

Appearance

Users

Course categories, with cohorts and course structure

Include course content

Certificate templates

Programs

Certifications

Organisation structure

Dynamic rules

Custom reports

Select all tenants (excluding archived)

Select all tenants (including archived)

Select tenants manually...

Users User profiles

User pictures

Include suspended users

Select all users

Select users manually...

Step 3 - Review

During the third and final step, you need to review and execute the export process. You will be presented with a summary of the selected and unselected options for content and instances. The export can only be started when the resulting file will be non-empty; that is, at least one instance has been selected. For instance, if you choose a course category that does not contain any courses, the Export button will remain deactivated. Finally, you have to confirm the execution of the export.

Once the export has been initiated, its status will be set to Scheduled. The export will commence on the subsequent cron execution. While the export file is being generated, the status will be changed to In progress. This might take some time on larger exports. When the export file is ready for download or import, the status changes again to Success. You will also receive a notification via the usual on-board mechanism. If an Error occurred during the export process, details would be displayed on the status page.

Migration Import

You can access the migration tool via Site administration > Migration or directly via the Migration icon in the Workplace launcher.

On the Imports tab, you will see a list of any imports which contains the following columns:

  • Date (filter): Date and time the import was created
  • Created by: User account that initiated the import
  • Importer (filter): Type of importer
  • Tenant (filter): Name of the tenant that the import was created for
  • Size: Size of the import file
  • Status (filter): Scheduled / In progress / Success / Error
  • Actions:
    • Download: Download the import file
    • View import: Show a status report that includes a status or error log
    • Delete: Remove the import file

To create a new migration export, select the + Import button on the Imports tab. You will then be guided through the 5-step import process.

Step 1 - Select source

In step 1, you must provide an input file: You will need to upload either a migration export file or a CSV file.

By default, the importer automatically detects what export type has been used before proceeding to step 2. In addition to the Detect automatically option, you can also select the File format, choosing between Workplace format and CSV. When the latter is used, make sure the CSV delimiter and Encoding settings are correct. The CSV import format only supports departments and positions.

Step 2 - General settings

The second step differs depending on what type of import file you have uploaded.

Step 2a - General settings (Importing a file in Workplace format)

The following two sections are available when importing a file in Workplace format:

  • About this file: Metadata about the file and its content is displayed
  • Destination: Select the tenant the file has to be imported to. You can either opt for the current tenant or select a tenant from your site. This section is not available when you wish to import a tenant import file. For some import files, the option Available for all tenants is offered; that is, it will be imported into Shared space.

Step 2b - General settings (Importing a file in CSV format)

The following three sections are available when importing a file in CSV format:

  • About this file: A preview of the first three rows is displayed, along with an option to show additional records.
  • Select importer: Moodle Workplace supports a Departments importer and a Positions importer. The formats of those files will be shown in Step 3b - Options.
  • Destination: Select the tenant the file has to be imported to. You can either opt for the current tenant or select a tenant from your site.

Step 3 - Options

During the third step, you can narrow down your selection and specify which elements you want to import.

Step 3a - Options (Importing a file in Workplace format)

When importing a file in Workplace format, the importer options always contains at least the following two elements (except when importing tenants or the entire site):

  • Content: Which attributes and settings may or may not be included when importing the file.
  • Instances: Which elements will be included when importing the file.

The content and instances available depend on the chosen importer. Details about each setting are displayed in the balloon help indicated by a question mark.

The table below lists all Content and Instances settings (locked entries in italics) for each importer plugin.

Importer Content Instances
Certificates Certificate template details

Issued certificates

Select all certificate templates in this file

Select certificate templates manually…

Course category…

Certifications Settings

Associated programs

Course backups excluding user data

Select course category...

Certification user allocations (if available)

Dynamic rules

Select all certifications in this file

Select manually...

Cohorts Cohort details

Cohort members

Select all cohorts in this file

Select manually...

Import all in system context

Import all in the selected category...

Course categories Course category details

Course backups excluding user data

Certificate templates (if available)

Cohort details including members (if available)

Select all course categories in this file

Select categories and subcategories manually…

Parent category...

Courses Course backups excluding user data Select all courses in this file

Select courses manually…

Course category...

Custom reports Report definitions

Audiences

Schedules

Import all custom reports

Import specific custom reports...

Dynamic rules Rule definitions, conditions and actions Select all Dynamic rules in this file

Select manually...

Organisation structure frameworks Descriptions, hierarchy and permissions Select all department and position frameworks

Select all department frameworks

Select all position frameworks

Select manually...

Organisation structure jobs Job assignments

Department and position frameworks

Select all jobs in this file

Select all jobs from specific frameworks...

Programs Settings

Course backups excluding user data

Course category...

Dynamic rules

Select all the programs in this file

Select manually...

Site Details

Appearance

Users

Content: courses, categories, programs and certifications

Cohort details including members

Certificate templates

Organisation structure

Dynamic rules

Custom reports

Tenants Details

Appearance

Users

Course categories, with cohorts and course structure

Certificate templates

Programs

Certifications

Organisation structure

Dynamic rules

Custom reports

Merge into existing tenant...

Create new tenants

Select all tenants

Select tenants manually...

Users User profiles

User pictures

Select all users

Select users manually...

Step 3b - Options (Importing a file in CSV format)

The following three sections exist when you're importing either departments or positions via CSV files:

  • Position/Department framework: Either create a new (generic) framework where a name will be provided automatically (Import <name of import file>) or select an existing framework.
  • Hierarchy: Positions and departments can either be arranged hierarchically or as a list. If the organisation structure is hierarchical, a position identifier must be provided, which will be used to identify the parent.
  • Field mapping: The import file contains the following fields:
    • name: Name of the position or department
    • idnumber: ID number of the position or department
    • description: Textual description of the position or department
    • descriptionformat: 1 = Moodle auto format (default), 2 = Plain text format, 3 = HTML format, 4 = Markdown format
    • parent: ID number of the parent. The mapping for this field cannot be the same as the position identifier provided in the hierarchy section.
    • manager (positions only): Toggle whether the manager role permissions are all available (1) or not (0)
    • departmentlead (positions only): Toggle whether the department lead role permissions are all available (1) or not (0)


When the import file does not follow the notation laid out here, you have the option to select CSV columns manually.

Sample departments import file:

idnumber,parent,name,description
SAL,,Sales,Sales department
SAD,SAL,Sales development,Sales development department
CSC,SAL,Customer Success,Customer Success department
EDI,,Editorial,Editorial department
AND,EDI,Art and design,Art and design department
PRO,EDI,Production,Production department
RNE,EDI,Research and Education,Research and Education department
MED,RNE,Medical advising,Medical advising department
MHR,RNE,Mental Health research,Mental Health research department
ONR,RNE,Oncology research,Oncology research department
MKT,,"Marketing, Promotion and Advertising","Marketing, Promotion and Advertising department"
PUB,MKT,Publicity,Publicity department
MH,MKT,Mental Health,Mental Health department
ONC,MKT,Oncology,Oncology department
IT,,Information and Technology,Information and Technology department
SUP,IT,ICT Support,ICT Support department
ASP,IT,Applications support,Applications support department
ADE,IT,Applications development,Applications development department
INF,IT,Infraestructure,Infraestructure department
HR,,Human Resources,Human Resources department
LND,HR,Learning and Development,Learning and Development department
ADM,,Administration,Administration department
LNC,ADM,Legal and Contracts,Legal and Contracts department
SRI,LNC,Subsidiary Rights,Subsidiary Rights department
FNA,ADM,Finance and Accounting,Finance and Accounting department

Sample positions import file:

idnumber,parent,name,description,manager
CEO,,Chief Executive Officer,Chief Executive Officer position,1
COO,CEO,Chief Operating Officer,Chief Operating Officer position,1
HMD,COO,Head of Marketing and Design,Head of Marketing and Design position,1
DES,HMD,Designer,Designer position,0
MCO,HMD,Marketing Consultant,Marketing Consultant position,0
OM,COO,Office Manager,Office Manager position,0
PM,COO,Project Manager,Project Manager position,1
EDI,PM,Editor,Editor position,0
MEW,PM,Medical Writer,Medical Writer position,0
CTO,CEO,Chief Technology Officer,Chief Technology Officer position,1
LDE,CTO,Lead Developer,Lead Developer position,1
DEV,LDE,Developer,Developer position,0
LDO,CTO,Lead DevOps,Lead DevOps position,1
DEO,LDO,DevOps,DevOps position,0
SA,CTO,Sysadmin,Sysadmin position,1
OPE,SA,Operator,Operator position,0
CFO,CEO,Chief Financial Officer,Chief Financial Officer position,1
AC,CFO,Accountant,Accountant position,0
DPO,CFO,Data Privacy Officer,Data Privacy Officer position,0
LEA,CFO,Legal advisor,Legal advisor position,0
HHR,CFO,HR Manager,HR Manager position,1
ODA,HHR,Organisational Development advisor,Organisational Development advisor position,0
CCO,CEO,Chief Commercial Officer,Chief Commercial Officer position,1
AMG,CCO,Account Manager,Account Manager position,0
SAR,CCO,Sales Representative,Sales Representative position,0

Step 4 - Conflicts

If the import file contains any inconsistencies that the migration tool cannot resolve automatically, the conflict resolution screen will be shown, and action must be taken.

Typical conflicts are that a record already exists (for instance, the username or an organisation ID) or an associated user account does not exist. Depending on the conflict, two types of resolution mechanisms are offered: either a solution is provided, or the record has to be skipped.

Step 5 - Review

During the fifth and final step, you need to review and execute the import process. You will be presented with a summary of the content that will be imported.

Once the import has been initiated, its status will be set to Scheduled. The import will commence on the subsequent cron execution. While the import is being processed, the status will be changed to In progress. This might take some time on larger imports. When the import is completed, the status changes again to Success. You will also receive a notification via the usual on-board mechanism. If an Error occurred during the import process, details would be displayed on the status page.

Command-line interface

The following commands will allow you to perform export and import from CLI:

php admin/tool/wp/cli/export.php

php admin/tool/wp/cli/import.php

Use the --help parameter to display all the available options, alongside a short description of each parameter. Exporter and importer-specific parameters are passed as JSON-encoded strings.

The two migration CLIs support interactive and non-interactive modes.

During interactive mode, you will be asked various questions, such as the program name to be exported, and you have to confirm actions, such as the usage of default values. This is particularly useful during one-off operations and when testing new processes.

In non-interactive mode, no questions are asked, no confirmations are needed, and default values are used. It is intended for usage in scripts to automate regular processes fully.

When run in interactive mode, the non-interactive counterpart, that is, the valid command will be displayed. That way, you can create your command step by step and, once completed, use it in your shell scripts.

Import of legacy data

Moodle Workplace 3.11 introduced some changes in the audience and schedules for Report Builder to support more types of audiences and link schedules to existing audiences. Due to these improvements, audiences and schedules from versions prior to 3.11 need to be upgraded following this process:

  • Audiences: report audiences based on job department/position will be converted to the new "job" audience type.
  • Schedules: New audiences will be automatically created for previously defined report schedule recipients for job department, position and manually added users. Each of these audiences will automatically be added to the new schedule as recipients.

Note recipient emails are no longer supported in schedules. Where previous schedules used these, the person who created the schedule will receive an email notifying them of any changes made. The email will include a list of previous email recipients, with a link to this page. As an alternative to external email addresses, consider creating users with the No login authentication method.

The same process will be followed when importing Report Builder audience and schedules prior to 3.11 using the Migration tool.