Migrations

From MoodleDocs
workplacelogo.png This feature is part of Moodle Workplace™, which is available through Moodle Partners.

Moodle Workplace provides a powerful Migration tool that lets you copy data and elements between tenants and sites. It also lets you import and export data from external systems.

Overview

The migration tool can export various parts of your Moodle Workplace instance and import them into the same site or different sites. The following diagram shows a high-level migration overview, visualising the typical workflows of exporting and importing Moodle Workplace data:

Migration - Overview schema.png

The export process contains the following key steps:

  1. Selecting the exporter: Initiating the respective wizard of the selected exporter
  2. Export options: This step covers two parts, namely content (what data must be exported) and instances (which elements should be exported)
  3. Review and export: Executing/scheduling the actual export and creating the export file

The import process contains the following steps:

  1. Selecting a source: Choosing the import file you will be dealing with.
  2. Selecting a tenant: Choosing the tenant where the data must be imported to.
  3. Import options: This step covers two parts: content (what data must be imported) and instances (which elements should be imported).
  4. Conflict resolution: If the import file contains any inconsistencies, manual intervention is required.
  5. Review and importing: Executing/scheduling the actual import.

Migration exporters and importers can be chained together – that way, import and export workflows can be created. For example, the tenant exporter makes use of all other exporters through internal cascading.

Migration Export

You can access the migration tool via Site administration > Migration or directly via the Migration icon in the Workplace launcher. On the Exports tab, you will see a list of any exports, which contains the following columns:

  • Date (filter): Date and time the export was created
  • Created by: User account that initiated the export
  • Exporter (filter): Type of exporter
  • Tenant (filter): Name of the tenant that the export was created for
  • Size: Size of the export file
  • Status (filter): Scheduled / In progress / Success / Error
  • Actions:
    • New import from this file: Use the export file as input and start the migration importer. This will take you directly to Step 2 of the import process.
    • Download: Download the export file
    • View export: Show a status report that includes a status or error log
    • Delete: Remove the export file
Migration - Exports.png

To create a new migration export, select the + Export button on the Exports tab. You will then be guided through the 3-step export process.

Step 1 - General settings

The first step of the migration process is to choose an exporter.

Migration - Export Step 1.png

Some exporters offer a Site / Current tenant switch, which allows you to define whether you want to export entities from the whole site, or only those associated with the current tenant.

Exporter Site / Current tenant switch Description
Certificates Yes Issued certificates and templates
Certifications No Certifications with their associated programs, courses, user allocations and component dynamic rules
Cohorts Yes Cohorts, including cohort members without user data
Course categories Yes Course categories and subcategories
Courses Yes Courses without user data, using default course backup configuration
Custom reports No Custom reports, including audience and schedule data
Dynamic rules No Dynamic rules definition, conditions and actions
Organisation structure frameworks No Frameworks with the whole hierarchy for departments and/or positions
Organisation structure jobs No Jobs with their associated department and position frameworks
Programs No Programs with their courses, user allocations and component dynamic rules
Site No Full site content including all tenants along with all entities contained within them. Course backups will include all content and user data except logs.
Tenants Yes Tenants along with all entities contained within them
Users Yes Site and tenant users

Step 2 - Options

During the second step, you can narrow down your selection and specify which elements you want to export. The exporter options always contain the following two elements (except when exporting the entire site):

  • Content: Which attributes and settings may or may not be included in the export file.
  • Instances: Which elements will be included in the export file.

The content and instances available depend on the chosen exporter. Details about each setting are displayed in the balloon help indicated by a question mark.

Migration - Export Step 2.png

The table below lists all Content and Instances settings (locked entries in italics) for each exporter plugin.

Exporter Content Instances
Certificates Certificate template details

Issued certificates

Select all certificate templates

Select categories and subcategories manually…

Select certificate templates manually...

Certifications Settings

Associated programs

Course structure

Include course content

Certification user allocations

Dynamic rules

Select all active certifications

Select active and archived certifications

Select manually…

Include shared entities

Cohorts Cohort details

Cohort members

All cohorts

All system cohorts

Select categories and subcategories manually...

Select manually...

Course categories Course category details

Course structure

Include course content

Certificate templates

Cohort details including members

Select categories and subcategories manually
Courses Course structure

Include course content

Select categories and subcategories manually…

Select courses manually...

Custom reports Report definitions

Audiences

Schedules

Export all custom reports

Export specific custom reports...

Dynamic rules Rule definitions, conditions and actions Select all Dynamic rules (excluding archived)

Select all enabled Dynamic rules

Select all Dynamic rules (including archived)

Organisation structure frameworks Descriptions, hierarchy and permissions Select all department and position frameworks

Select all department frameworks

Select all position frameworks

Select manually...

Include shared entities

Organisation structure jobs Job assignments

Department and position frameworks

Select all active jobs

Select all active and past jobs

Select all jobs in any of the selected frameworks...

Programs Settings

Course structure

Include course content

Program user allocations

Dynamic rules

Select all active programs

Select all active and archived programs

Select manually...

Include shared entities

Site Details

Appearance

Users

Content: courses, categories, programs and certifications

Cohort details including members

Certificate templates

Organisation structure

Dynamic rules

Custom reports

Tenants Details

Appearance

Users

Course categories, with cohorts and course structure

Include course content

Certificate templates

Programs

Certifications

Organisation structure

Dynamic rules

Custom reports

Select all tenants (excluding archived)

Select all tenants (including archived)

Select tenants manually...

Users User profiles

User pictures

Include suspended users

Select all users

Select users manually...

Step 3 - Review

During the third and final step, you need to review and execute the export process. You will be presented with a summary of the selected and unselected options for content and instances. The export can only be started when the resulting file will be non-empty; that is, at least one instance has been selected. For instance, if you choose a course category that does not contain any courses, the Export button will remain deactivated. Finally, you have to confirm the execution of the export.

Migration - Export Step 3.png

Once the export has been initiated, its status will be set to Scheduled. The export will commence on the subsequent cron execution. While the export file is being generated, the status will be changed to In progress. This might take some time on larger exports. When the export file is ready for download or import, the status changes again to Success. You will also receive a notification via the usual on-board mechanism. If an Error occurred during the export process, details would be displayed on the status page.

Migration Import

You can access the migration tool via Site administration > Migration or directly via the Migration icon in the Workplace launcher.

On the Imports tab, you will see a list of any imports which contains the following columns:

  • Date (filter): Date and time the import was created
  • Created by: User account that initiated the import
  • Importer (filter): Type of importer
  • Tenant (filter): Name of the tenant that the import was created for
  • Size: Size of the import file
  • Status (filter): Scheduled / In progress / Success / Error
  • Actions:
    • Download: Download the import file
    • View import: Show a status report that includes a status or error log
    • Delete: Remove the import file
Migration - Imports.png

To create a new migration export, select the + Import button on the Imports tab. You will then be guided through the 5-step import process.

Step 1 - Select source

In step 1, you must provide an input file: You will need to upload either a migration export file or a CSV file.

Migration - Import Step 1.png

By default, the importer automatically detects what export type has been used before proceeding to step 2. In addition to the Detect automatically option, you can also select the File format, choosing between Workplace format and CSV. When the latter is used, make sure the CSV delimiter and Encoding settings are correct. The CSV import format only supports departments and positions.

Step 2 - General settings

The second step differs depending on what type of import file you have uploaded.

Step 2a - General settings (Importing a file in Workplace format)

The following two sections are available when importing a file in Workplace format:

  • About this file: Metadata about the file and its content is displayed
  • Destination: Select the tenant the file has to be imported to. You can either opt for the current tenant or select a tenant from your site. This section is not available when you wish to import a tenant import file. For some import files, the option Available for all tenants is offered; that is, it will be imported into Shared space.
Migration - Import Step 2a.png

Step 2b - General settings (Importing a file in CSV format)

The following three sections are available when importing a file in CSV format:

  • About this file: A preview of the first three rows is displayed, along with an option to show additional records.
  • Select importer: Moodle Workplace supports a Departments importer and a Positions importer. The formats of those files will be shown in Step 3b - Options.
  • Destination: Select the tenant the file has to be imported to. You can either opt for the current tenant or select a tenant from your site.
Migration - Import Step 2b.png

Step 3 - Options

During the third step, you can narrow down your selection and specify which elements you want to import.

Step 3a - Options (Importing a file in Workplace format)

When importing a file in Workplace format, the importer options always contains at least the following two elements (except when importing tenants or the entire site):

  • Content: Which attributes and settings may or may not be included when importing the file.
  • Instances: Which elements will be included when importing the file.

The content and instances available depend on the chosen importer. Details about each setting are displayed in the balloon help indicated by a question mark.

Migration - Import Step 3a.png

The table below lists all Content and Instances settings (locked entries in italics) for each importer plugin.

Importer Content Instances
Certificates Certificate template details

Issued certificates

Select all certificate templates in this file

Select certificate templates manually…

Course category…

Certifications Settings

Associated programs

Course backups excluding user data

Select course category...

Certification user allocations (if available)

Dynamic rules

Select all certifications in this file

Select manually...

Cohorts Cohort details

Cohort members

Select all cohorts in this file

Select manually...

Import all in system context

Import all in the selected category...

Course categories Course category details

Course backups excluding user data

Certificate templates (if available)

Cohort details including members (if available)

Select all course categories in this file

Select categories and subcategories manually…

Parent category...

Courses Course backups excluding user data Select all courses in this file

Select courses manually…

Course category...

Custom reports Report definitions

Audiences

Schedules

Import all custom reports

Import specific custom reports...

Dynamic rules Rule definitions, conditions and actions Select all Dynamic rules in this file

Select manually...

Organisation structure frameworks Descriptions, hierarchy and permissions Select all department and position frameworks

Select all department frameworks

Select all position frameworks

Select manually...

Organisation structure jobs Job assignments

Department and position frameworks

Select all jobs in this file

Select all jobs from specific frameworks...

Programs Settings

Course backups excluding user data

Course category...

Dynamic rules

Select all the programs in this file

Select manually...

Site Details

Appearance

Users

Content: courses, categories, programs and certifications

Cohort details including members

Certificate templates

Organisation structure

Dynamic rules

Custom reports

Tenants Details

Appearance

Users

Course categories, with cohorts and course structure

Certificate templates

Programs

Certifications

Organisation structure

Dynamic rules

Custom reports

Merge into existing tenant...

Create new tenants

Select all tenants

Select tenants manually...

Users User profiles

User pictures

Select all users

Select users manually...

Step 3b - Options (Importing a file in CSV format)

The following three sections exist when you're importing either departments or positions via CSV files:

  • Position/Department framework: Either create a new (generic) framework where a name will be provided automatically (Import <name of import file>) or select an existing framework.
  • Hierarchy: Positions and departments can either be arranged hierarchically or as a list. If the organisation structure is hierarchical, a position identifier must be provided, which will be used to identify the parent.
  • Field mapping: The import file contains the following fields:
    • name: Name of the position or department
    • idnumber: ID number of the position or department
    • description: Textual description of the position or department
    • descriptionformat: 1 = Moodle auto format (default), 2 = Plain text format, 3 = HTML format, 4 = Markdown format
    • parent: ID number of the parent. The mapping for this field cannot be the same as the position identifier provided in the hierarchy section.
    • manager (positions only): Toggle whether the manager role permissions are all available (1) or not (0)
    • departmentlead (positions only): Toggle whether the department lead role permissions are all available (1) or not (0)

When the import file does not follow the notation laid out here, you have the option to select CSV columns manually.

Migration - Import Step 3b.png

Sample departments import file:

idnumber,parent,name,description
SAL,,Sales,Sales department
SAD,SAL,Sales development,Sales development department
CSC,SAL,Customer Success,Customer Success department
EDI,,Editorial,Editorial department
AND,EDI,Art and design,Art and design department
PRO,EDI,Production,Production department
RNE,EDI,Research and Education,Research and Education department
MED,RNE,Medical advising,Medical advising department
MHR,RNE,Mental Health research,Mental Health research department
ONR,RNE,Oncology research,Oncology research department
MKT,,"Marketing, Promotion and Advertising","Marketing, Promotion and Advertising department"
PUB,MKT,Publicity,Publicity department
MH,MKT,Mental Health,Mental Health department
ONC,MKT,Oncology,Oncology department
IT,,Information and Technology,Information and Technology department
SUP,IT,ICT Support,ICT Support department
ASP,IT,Applications support,Applications support department
ADE,IT,Applications development,Applications development department
INF,IT,Infraestructure,Infraestructure department
HR,,Human Resources,Human Resources department
LND,HR,Learning and Development,Learning and Development department
ADM,,Administration,Administration department
LNC,ADM,Legal and Contracts,Legal and Contracts department
SRI,LNC,Subsidiary Rights,Subsidiary Rights department
FNA,ADM,Finance and Accounting,Finance and Accounting department

Sample positions import file:

idnumber,parent,name,description,manager
CEO,,Chief Executive Officer,Chief Executive Officer position,1
COO,CEO,Chief Operating Officer,Chief Operating Officer position,1
HMD,COO,Head of Marketing and Design,Head of Marketing and Design position,1
DES,HMD,Designer,Designer position,0
MCO,HMD,Marketing Consultant,Marketing Consultant position,0
OM,COO,Office Manager,Office Manager position,0
PM,COO,Project Manager,Project Manager position,1
EDI,PM,Editor,Editor position,0
MEW,PM,Medical Writer,Medical Writer position,0
CTO,CEO,Chief Technology Officer,Chief Technology Officer position,1
LDE,CTO,Lead Developer,Lead Developer position,1
DEV,LDE,Developer,Developer position,0
LDO,CTO,Lead DevOps,Lead DevOps position,1
DEO,LDO,DevOps,DevOps position,0
SA,CTO,Sysadmin,Sysadmin position,1
OPE,SA,Operator,Operator position,0
CFO,CEO,Chief Financial Officer,Chief Financial Officer position,1
AC,CFO,Accountant,Accountant position,0
DPO,CFO,Data Privacy Officer,Data Privacy Officer position,0
LEA,CFO,Legal advisor,Legal advisor position,0
HHR,CFO,HR Manager,HR Manager position,1
ODA,HHR,Organisational Development advisor,Organisational Development advisor position,0
CCO,CEO,Chief Commercial Officer,Chief Commercial Officer position,1
AMG,CCO,Account Manager,Account Manager position,0
SAR,CCO,Sales Representative,Sales Representative position,0

Step 4 - Conflicts

If the import file contains any inconsistencies that the migration tool cannot resolve automatically, the conflict resolution screen will be shown, and action must be taken.

Migration - Import Step 4.png

Typical conflicts are that a record already exists (for instance, the username or an organisation ID) or an associated user account does not exist. Depending on the conflict, two types of resolution mechanisms are offered: either a solution is provided, or the record has to be skipped.

Step 5 - Review

During the fifth and final step, you need to review and execute the import process. You will be presented with a summary of the content that will be imported.

Migration - Import Step 5.png

Once the import has been initiated, its status will be set to Scheduled. The import will commence on the subsequent cron execution. While the import is being processed, the status will be changed to In progress. This might take some time on larger imports. When the import is completed, the status changes again to Success. You will also receive a notification via the usual on-board mechanism. If an Error occurred during the import process, details would be displayed on the status page.

Command-line interface

The following commands will allow you to perform export and import from CLI:

php admin/tool/wp/cli/export.php

php admin/tool/wp/cli/import.php

Use the --help parameter to display all the available options, alongside a short description of each parameter. Exporter and importer-specific parameters are passed as JSON-encoded strings.

The two migration CLIs support interactive and non-interactive modes.

During interactive mode, you will be asked various questions, such as the program name to be exported, and you have to confirm actions, such as the usage of default values. This is particularly useful during one-off operations and when testing new processes.

In non-interactive mode, no questions are asked, no confirmations are needed, and default values are used. It is intended for usage in scripts to automate regular processes fully.

When run in interactive mode, the non-interactive counterpart, that is, the valid command will be displayed. That way, you can create your command step by step and, once completed, use it in your shell scripts.