MoodleDocs - User contributions [en]

Moodle development environment with Git submodules

2016-03-15T15:37:25Z

Jan-eberhardt:

This page describes how to set up a development environment for a Moodle instance with external plugins.
Over time the number of plugins may increase in such a way, that they can't be managed manually by developers anymore.
Without any precautions this can lead to various problems, for instance:
* Missing plugins in cloned repositories
* Diverged versions in existing repositories (can lead to fatal errors when trying to update the site)
* Messed up Git excludes

You'll learn in this guide how to use Git submodules in order to achieve the following setup:

[[Image:superprojects1.png|frame|center|Scheme of a development environment]]

'''Note: The extension '''mod_mylittleextension''' is a fake plugin, which was created for testing purposes only.'''

By using Git submodules all extensions of a repository can be simply synchronized with other projects.
All repositories are based on the so called '''superproject''' repository and the developers' projects will be sometimes referred as 'local' repository.
Anyone who is interested in maintaining the superproject and core changes by Git can visit [[Moodle Production Server with GIT]].

'''Note: This guide won't cover basic Git commands and how to install/maintain Git submodules.
It is assumed, that you have read [[Git for Administrators]] where these topics are covered.
The superproject will be seen as a Moodle repository where several plugins have been already installed via Git submodules.'''

== Cloning a superproject ==

First a short explanation of events which are triggered, when a superproject is cloned.
There are some things in Git submodules, which aren't obvious and can be quite surprising.
'''Note: Ensure that you've understood the how-to at [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] before continuing.'''

Assume the plugin '''mod_mylittleextension''' (remember: MLE is a fake plugin) was installed in the superproject and these changes have been commited:
<pre>
$ cd /path/to/your/superproject
$ git submodule add /local/repositories/mle/ mod/mylittleextension
$ git commit -a -m "New module MLE installed"
</pre>

Now a developer has to type in the following commands, in order to clone the superproject (maybe from a remote device).
<pre>
$ cd /path/to/local/directory
$ git clone <source> moodle
$ cd moodle
$ git submodule update --init --recursive
Submodule 'mod/mylittleextension' (/local/repositories/mle) registered for path 'mod/mylittleextension'
Cloning into 'mod/mylittleextension'...
done.
Submodule path 'mod/mylittleextension': checked out '89d9eae3d5142474d8452128e8df5720d89012cd'
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
</pre>

The new main repositories will automatically know about the new extension, but it won't be initialized, which means that the submodule's directory will be empty in the beginning.
For initializing the submodule, the developer needs to execute '''git submodule update --init --recursive'''.
The '''--recursive''' option means, that Git will walk through nested submodules as well (not obligatory in this example).

After the intialization the submodule's repository will be in detached HEAD state.
This is due the fact, that Git receives following data from the superproject:
* source url
* local path
* HEAD reference (hash)
See this figure, for a better understanding:

[[Image:superprojects2.png|frame|center|Dotted line: information obtained from the superproject, but already saved locally (by pull)]]

This means, '''git submodule update --init''' will first do a clone of '''source url''' into the '''path''' and then check out the '''reference''', leaving the submodule in detached HEAD state.
You and your developers need to keep in mind, that any change of the HEAD's reference of your submodule will be noticed by the main repository.
This means also, that everyone can switch branches unnoticedly as long as the HEAD's hash reference doesn't change.
E.g. a checkout of the branch '''master''' won't be noticed by the main repository, but a checkout of '''dev''' will:
<pre>
mod/mylittleextension $ git checkout master
mod/mylittleextension $ git branch -av
* master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
$ cd ../..
$ git status
# On branch master
nothing to commit (working directory clean)
$ cd mod/mylittleextension
mod/mylittleextension $ git checkout dev
Branch dev set up to track remote branch dev from origin.
Switched to a new branch 'dev'
mod/mylittleextension $ cd ../..
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

There is a shorter command to initialize the submodules, but the outcome will be the same as above.
Instead of changing the moodle directory and use '''git submodule update --init''' after the clone, you can use '''git clone''' with the '''--recursive''' option as well.
<pre>
$ cd /path/to/new/location
$ git clone --recursive <source> moodle
</pre>

In other words, you may use '''git submodule update --init''' in the case you forget the '''--recursive''' option in your '''git clone''' command.

== Upgrading your submodules via the superproject ==

We now assume, that the external plugin is updated by it's own repository only and no customization is needed.
See [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] for further information on the topic of maintaining submodules of your superproject.

=== Pull and upgrade ===

In case a submodule was updated inside the superproject, your developers have to be careful.
'''git pull''' won't upgrade the submodules at once and an additional command is needed.
This is what '''git pull''' would look like:
<pre>
$ cd /path/to/local/repository
$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /local/repositories/super/
b547f6c..7f0c348 master -> origin/master
Fetching submodule mod/mylittleextension
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From /local/repositories/mle
3a2d487..5e0e66a dev -> origin/dev
89d9eae..5e0e66a master -> origin/master
Updating b547f6c..7f0c348
Fast-forward
mod/mylittleextension | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
dev 3a2d487 [behind 1] First commit
master 89d9eae [behind 2] Initial commit
remotes/origin/master 5e0e66a Second commit
remotes/origin/dev 5e0e66a Second commit
</pre>

As you see, '''git pull'' triggers '''git fetch''' inside the submodules repository, but nothing else.
Therefore, your developers have to update the submodules afterwards:
<pre>
$ cd /path/to/local/repository
$ git submodule update
Submodule path 'mod/mylittleextension': checked out '5e0e66aa787f4ebe9f61a940969a6e94abf01a1e'
</pre>

=== Figure: Updates without customization ===

[[File:superprojects3.png|frame|center|A workflow for an upgrade from the source repository]]

=== Known issues ===

Since '''git submodule upgrade''' checks out the reference, given by your main repository's commit, your submodule's repository will turn into detached HEAD state again.
The command is equivalent to '''git checkout''' of the reference inside each submodule's path.
That means, that conflicts will abort the checkout, like entering the directory and trying to do the checkout manually.
So the submodule's repository should be clean in order to prevent problems with the submodule update.

A special case of this problem happens, if you try to clone a superproject, whose submodule's reference is not known by the origin.
Your submodule's directory would be left blank (since there's nothing to check out) and you would have to resolve this issue manually (e.g. add your superprojects submodule repository as a remote).
A situation, you don't want to get, for sure.

== Upgrading customized plugins ==

=== Figure: Suggested workflow ===

In a normal workflow you don't touch external plugins and therefore there is no need to alter the submodules in any way.
But what, if you '''want to alter''' them.
In some cases the submodule is your own plugin or the extension is somehow insufficient but the original contributor doesn't want to merge your pull requests.

As you know from the previous section '''git submodule update''' tries to checkout a specific reference.
Therefore, this reference must exist in the origin or you will get a problem while updating them.
If you're not the contributer of the extension it is necessary to fork these repositiories.
A suggested workflow would be like this:

[[File:superprojects4.png|frame|center|Suggested workflow with customized plugins]]

Your developers are responsible for updating your forked plugin.
Therefore they must merge the original changes into your forked code and resolve conflicts, if necessary.
Your superproject pulls from the fork only.
If you're setting up your first superproject, it doesn't have to be connected to the original repositiory at all, since you'll get all updates from your fork.
So instead of adding the extension's original remote, you may add the fork only (after adding you can use '''git remote rename origin fork''' before your initial commit of the plugin).

=== Setting up a new submodule ===

In case you haven't installed the extension yet, you can add and rename the repository afterwards (to avoid misunderstanding).
On your superproject's machine do the following commands:
<pre>
$ cd /path/to/moodle
$ git submodule add <source:fork> mod/mylittleextension
Cloning into 'mod/mylittleextension'...
done.
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git branch -avv
* master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
mod/mylittleextension $ cd ../..
$ git commit -a -m "New extension 'MLE' installed"
[master adecc96] New extension 'MLE' installed
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 mod/mylittleextension
</pre>

Your developers will get your new submodule using '''git clone''' or '''git pull''' and can rename the remote likewise.
In addition, they have to fetch updates from the original remote and have to add a remote repository manually.
On your developers machine type in (output messages from Git are omitted):
<pre>
cd /path/to/local/repository
$ git pull
$ git submodule update --init
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Now your developers can add branches, edit code, do commits and contribute to your fork.
Details about that topic will follow after the next section.

=== Changes for an existing submodule ===

If the submodule is already installed and your superproject cloned, you must edit the '''.gitmodules''' file.
Since the '''path''' and '''url''' of your submodule are saved ('''reference''' is saved in your commit), you may edit the file manually.
But you can also use a Git command for that purpose.
On the superproject's machine type in (output messages from Git are omitted):
<pre>
cd /path/to/moodle/
$ git config --file=.gitmodules submodule."mod/mylittleextension".url "<source:fork>"
</pre>
'''or''' edit the file '''.gitmodules''' respectively.

Afterwards type in:
<pre>
$ git submodule sync
Synchronizing submodule url for 'mod/mylittleextension'
$ git commit -a -m "Changed source url of 'MLE'"
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
</pre>

The '''sync''' command will alter the remotes of your submodule's repository to match the settings of your '''.gitmodules'''.
It is to mention, that changes in the remotes (like '''rename''' or '''add''') of a submodule wont be noticed by the Moodle repository.
In fact, unless you edit a file or change the submodule's HEAD reference, you can do whatever you want with your branches.

At last your developer has to update his project as well.
On the developers machine type in (output messages are omitted again):
<pre>
$ cd /path/to/local/repository
$ git pull
$ git submodule update
$ git submodule sync
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Again, the '''sync''' command will change the remote settings of the submodule to match them with the '''.gitmodules'''.

=== Contributing to the superproject ===

This section will explain how to contribute patches for your superproject.
Assume your developer has changed an extension and commit his changes in the submodule's repository.
<pre>
$ cd /path/to/local/repository
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -avv
* dev 9701a0c [fork/dev: ahead 1] Third commit
master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
</pre>

Push the changes into the fork.
<pre>
mod/mylittleextension $ git push fork dev:master
</pre>

You may organize your forked repository in another way, e.g. let the developers push into a seperate branch and merge them only, when it is satisfying.
For now, we assume the developers can push into your '''master''' (or, in general, the branch, which is tracked by your superproject).
Your developer's Moodle repository will notice the changes from their commit:
<pre>
$ cd /path/to/moodle
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

'''Do not commit the changes.'''
We presume, that every change in your main repository is done through a commit from your superproject.
The status is just saying, that your submodule is on another state than your superproject.
But this is not a problem at all. We will fix this later.
'''Note: When your developers need to commit changes, which are not related to the submodule, they have to avoid to stage the submodule until the superproject has been updated.'''

Update the superproject with the following commands on the superproject's machine:
<pre>
(On the superproject's machine)
$ cd /path/to/moodle
$ git submodule foreach git pull
$ git commit -a -m "Plugin updates"
</pre>

Your developers can now fetch the updated superproject.
<pre>
(On the developer's machine)
$ cd /path/to/local/repository
$ git pull
$ git status
# On branch master
nothing to commit (working directory clean)
</pre>

As you can see, your developer don't need to commit the changed status of the submodule.
Your developer's repository will be clean again after a fetch.

=== Figure: Workflow of an update of your submodules ===

[[Image:superprojects5.png|frame|center|The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.]]

== See also ==

; Moodle Docs
* [[Git for Administrators]]
* [[:dev:Git for developers|Git for developers guide]]
* [[Windows installation using Git]]
* [[Git for Mac]]
* [[Moodle_Production_Server_with_GIT|Case study Git + Moodle from Technical University Berlin]]

; External resources
* [https://git-scm.com/book/en/v2/Git-Tools-Submodules Submodules at the Git book]

Moodle development environment with Git submodules

2016-03-15T15:32:13Z

Jan-eberhardt:

This page describes how to set up a development environment for a Moodle instance with external plugins.
Over time the number of plugins may increase in such a way, that they can't be managed manually by developers anymore.
Without any precautions this can lead to various problems, for instance:
* Missing plugins in cloned repositories
* Diverged versions in existing repositories (can lead to fatal errors when trying to update the site)
* Messed up Git excludes

You'll learn in this guide how to use Git submodules in order to achieve the following setup:

[[Image:superprojects1.png|thumbs|center|Scheme of a development environment]]

'''Note: The extension '''mod_mylittleextension''' is a fake plugin, which was created for testing purposes only.'''

By using Git submodules all extensions of a repository can be simply synchronized with other projects.
All repositories are based on the so called '''superproject''' repository and the developers' projects will be sometimes referred as 'local' repository.
Anyone who is interested in maintaining the superproject and core changes by Git can visit [[Moodle Production Server with GIT]].

'''Note: This guide won't cover basic Git commands and how to install/maintain Git submodules.
It is assumed, that you have read [[Git for Administrators]] where these topics are covered.
The superproject will be seen as a Moodle repository where several plugins have been already installed via Git submodules.'''

== Cloning a superproject ==

First a short explanation of events which are triggered, when a superproject is cloned.
There are some things in Git submodules, which aren't obvious and can be quite surprising.
'''Note: Ensure that you've understood the how-to at [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] before continuing.'''

Assume the plugin '''mod_mylittleextension''' (remember: MLE is a fake plugin) was installed in the superproject and these changes have been commited:
<pre>
$ cd /path/to/your/superproject
$ git submodule add /local/repositories/mle/ mod/mylittleextension
$ git commit -a -m "New module MLE installed"
</pre>

Now a developer has to type in the following commands, in order to clone the superproject (maybe from a remote device).
<pre>
$ cd /path/to/local/directory
$ git clone <source> moodle
$ cd moodle
$ git submodule update --init --recursive
Submodule 'mod/mylittleextension' (/local/repositories/mle) registered for path 'mod/mylittleextension'
Cloning into 'mod/mylittleextension'...
done.
Submodule path 'mod/mylittleextension': checked out '89d9eae3d5142474d8452128e8df5720d89012cd'
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
</pre>

The new main repositories will automatically know about the new extension, but it won't be initialized, which means that the submodule's directory will be empty in the beginning.
For initializing the submodule, the developer needs to execute '''git submodule update --init --recursive'''.
The '''--recursive''' option means, that Git will walk through nested submodules as well (not obligatory in this example).

After the intialization the submodule's repository will be in detached HEAD state.
This is due the fact, that Git receives following data from the superproject:
* source url
* local path
* HEAD reference (hash)
See this figure, for a better understanding:

[[Image:superprojects2.png| Dotted line: information obtained from the superproject, but already saved locally (by pull)]]

This means, '''git submodule update --init''' will first do a clone of '''source url''' into the '''path''' and then check out the '''reference''', leaving the submodule in detached HEAD state.
You and your developers need to keep in mind, that any change of the HEAD's reference of your submodule will be noticed by the main repository.
This means also, that everyone can switch branches unnoticedly as long as the HEAD's hash reference doesn't change.
E.g. a checkout of the branch '''master''' won't be noticed by the main repository, but a checkout of '''dev''' will:
<pre>
mod/mylittleextension $ git checkout master
mod/mylittleextension $ git branch -av
* master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
$ cd ../..
$ git status
# On branch master
nothing to commit (working directory clean)
$ cd mod/mylittleextension
mod/mylittleextension $ git checkout dev
Branch dev set up to track remote branch dev from origin.
Switched to a new branch 'dev'
mod/mylittleextension $ cd ../..
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

There is a shorter command to initialize the submodules, but the outcome will be the same as above.
Instead of changing the moodle directory and use '''git submodule update --init''' after the clone, you can use '''git clone''' with the '''--recursive''' option as well.
<pre>
$ cd /path/to/new/location
$ git clone --recursive <source> moodle
</pre>

In other words, you may use '''git submodule update --init''' in the case you forget the '''--recursive''' option in your '''git clone''' command.

== Upgrading your submodules via the superproject ==

We now assume, that the external plugin is updated by it's own repository only and no customization is needed.
See [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] for further information on the topic of maintaining submodules of your superproject.

=== Pull and upgrade ===

In case a submodule was updated inside the superproject, your developers have to be careful.
'''git pull''' won't upgrade the submodules at once and an additional command is needed.
This is what '''git pull''' would look like:
<pre>
$ cd /path/to/local/repository
$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /local/repositories/super/
b547f6c..7f0c348 master -> origin/master
Fetching submodule mod/mylittleextension
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From /local/repositories/mle
3a2d487..5e0e66a dev -> origin/dev
89d9eae..5e0e66a master -> origin/master
Updating b547f6c..7f0c348
Fast-forward
mod/mylittleextension | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
dev 3a2d487 [behind 1] First commit
master 89d9eae [behind 2] Initial commit
remotes/origin/master 5e0e66a Second commit
remotes/origin/dev 5e0e66a Second commit
</pre>

As you see, '''git pull'' triggers '''git fetch''' inside the submodules repository, but nothing else.
Therefore, your developers have to update the submodules afterwards:
<pre>
$ cd /path/to/local/repository
$ git submodule update
Submodule path 'mod/mylittleextension': checked out '5e0e66aa787f4ebe9f61a940969a6e94abf01a1e'
</pre>

=== Figure: Updates without customization ===

[[File:superprojects3.png|center|A workflow for an upgrade from the source repository]]

=== Known issues ===

Since '''git submodule upgrade''' checks out the reference, given by your main repository's commit, your submodule's repository will turn into detached HEAD state again.
The command is equivalent to '''git checkout''' of the reference inside each submodule's path.
That means, that conflicts will abort the checkout, like entering the directory and trying to do the checkout manually.
So the submodule's repository should be clean in order to prevent problems with the submodule update.

A special case of this problem happens, if you try to clone a superproject, whose submodule's reference is not known by the origin.
Your submodule's directory would be left blank (since there's nothing to check out) and you would have to resolve this issue manually (e.g. add your superprojects submodule repository as a remote).
A situation, you don't want to get, for sure.

== Upgrading customized plugins ==

=== Figure: Suggested workflow ===

In a normal workflow you don't touch external plugins and therefore there is no need to alter the submodules in any way.
But what, if you '''want to alter''' them.
In some cases the submodule is your own plugin or the extension is somehow insufficient but the original contributor doesn't want to merge your pull requests.

As you know from the previous section '''git submodule update''' tries to checkout a specific reference.
Therefore, this reference must exist in the origin or you will get a problem while updating them.
If you're not the contributer of the extension it is necessary to fork these repositiories.
A suggested workflow would be like this:

[[File:superprojects4.png|center|Suggested workflow with customized plugins]]

Your developers are responsible for updating your forked plugin.
Therefore they must merge the original changes into your forked code and resolve conflicts, if necessary.
Your superproject pulls from the fork only.
If you're setting up your first superproject, it doesn't have to be connected to the original repositiory at all, since you'll get all updates from your fork.
So instead of adding the extension's original remote, you may add the fork only (after adding you can use '''git remote rename origin fork''' before your initial commit of the plugin).

=== Setting up a new submodule ===

In case you haven't installed the extension yet, you can add and rename the repository afterwards (to avoid misunderstanding).
On your superproject's machine do the following commands:
<pre>
$ cd /path/to/moodle
$ git submodule add <source:fork> mod/mylittleextension
Cloning into 'mod/mylittleextension'...
done.
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git branch -avv
* master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
mod/mylittleextension $ cd ../..
$ git commit -a -m "New extension 'MLE' installed"
[master adecc96] New extension 'MLE' installed
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 mod/mylittleextension
</pre>

Your developers will get your new submodule using '''git clone''' or '''git pull''' and can rename the remote likewise.
In addition, they have to fetch updates from the original remote and have to add a remote repository manually.
On your developers machine type in (output messages from Git are omitted):
<pre>
cd /path/to/local/repository
$ git pull
$ git submodule update --init
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Now your developers can add branches, edit code, do commits and contribute to your fork.
Details about that topic will follow after the next section.

=== Changes for an existing submodule ===

If the submodule is already installed and your superproject cloned, you must edit the '''.gitmodules''' file.
Since the '''path''' and '''url''' of your submodule are saved ('''reference''' is saved in your commit), you may edit the file manually.
But you can also use a Git command for that purpose.
On the superproject's machine type in (output messages from Git are omitted):
<pre>
cd /path/to/moodle/
$ git config --file=.gitmodules submodule."mod/mylittleextension".url "<source:fork>"
</pre>
'''or''' edit the file '''.gitmodules''' respectively.

Afterwards type in:
<pre>
$ git submodule sync
Synchronizing submodule url for 'mod/mylittleextension'
$ git commit -a -m "Changed source url of 'MLE'"
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
</pre>

The '''sync''' command will alter the remotes of your submodule's repository to match the settings of your '''.gitmodules'''.
It is to mention, that changes in the remotes (like '''rename''' or '''add''') of a submodule wont be noticed by the Moodle repository.
In fact, unless you edit a file or change the submodule's HEAD reference, you can do whatever you want with your branches.

At last your developer has to update his project as well.
On the developers machine type in (output messages are omitted again):
<pre>
$ cd /path/to/local/repository
$ git pull
$ git submodule update
$ git submodule sync
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Again, the '''sync''' command will change the remote settings of the submodule to match them with the '''.gitmodules'''.

=== Contributing to the superproject ===

This section will explain how to contribute patches for your superproject.
Assume your developer has changed an extension and commit his changes in the submodule's repository.
<pre>
$ cd /path/to/local/repository
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -avv
* dev 9701a0c [fork/dev: ahead 1] Third commit
master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
</pre>

Push the changes into the fork.
<pre>
mod/mylittleextension $ git push fork dev:master
</pre>

You may organize your forked repository in another way, e.g. let the developers push into a seperate branch and merge them only, when it is satisfying.
For now, we assume the developers can push into your '''master''' (or, in general, the branch, which is tracked by your superproject).
Your developer's Moodle repository will notice the changes from their commit:
<pre>
$ cd /path/to/moodle
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

'''Do not commit the changes.'''
We presume, that every change in your main repository is done through a commit from your superproject.
The status is just saying, that your submodule is on another state than your superproject.
But this is not a problem at all. We will fix this later.
'''Note: When your developers need to commit changes, which are not related to the submodule, they have to avoid to stage the submodule until the superproject has been updated.'''

Update the superproject with the following commands on the superproject's machine:
<pre>
(On the superproject's machine)
$ cd /path/to/moodle
$ git submodule foreach git pull
$ git commit -a -m "Plugin updates"
</pre>

Your developers can now fetch the updated superproject.
<pre>
(On the developer's machine)
$ cd /path/to/local/repository
$ git pull
$ git status
# On branch master
nothing to commit (working directory clean)
</pre>

As you can see, your developer don't need to commit the changed status of the submodule.
Your developer's repository will be clean again after a fetch.

=== Figure: Workflow of an update of your submodules ===

[[Image:superprojects5.png|The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.]]

== See also ==

; Moodle Docs
* [[Git for Administrators]]
* [[:dev:Git for developers|Git for developers guide]]
* [[Windows installation using Git]]
* [[Git for Mac]]
* [[Moodle_Production_Server_with_GIT|Case study Git + Moodle from Technical University Berlin]]

; External resources
* [https://git-scm.com/book/en/v2/Git-Tools-Submodules Submodules at the Git book]

Moodle development environment with Git submodules

2016-03-15T15:31:07Z

Jan-eberhardt:

This page describes how to set up a development environment for a Moodle instance with external plugins.
Over time the number of plugins may increase in such a way, that they can't be managed manually by developers anymore.
Without any precautions this can lead to various problems, for instance:
* Missing plugins in cloned repositories
* Diverged versions in existing repositories (can lead to fatal errors when trying to update the site)
* Messed up Git excludes

You'll learn in this guide how to use Git submodules in order to achieve the following setup:

[[Image:superprojects1.png|thumbs|center|Scheme of a development environment]]

'''Note: The extension '''mod_mylittleextension''' is a fake plugin, which was created for testing purposes only.'''

By using Git submodules all extensions of a repository can be simply synchronized with other projects.
All repositories are based on the so called '''superproject''' repository and the developers' projects will be sometimes referred as 'local' repository.
Anyone who is interested in maintaining the superproject and core changes by Git can visit [[Moodle Production Server with GIT]].

'''Note: This guide won't cover basic Git commands and how to install/maintain Git submodules.
It is assumed, that you have read [[Git for Administrators]] where these topics are covered.
The superproject will be seen as a Moodle repository where several plugins have been already installed via Git submodules.'''

== Cloning a superproject ==

First a short explanation of events which are triggered, when a superproject is cloned.
There are some things in Git submodules, which aren't obvious and can be quite surprising.
'''Note: Ensure that you've understood the how-to at [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] before continuing.'''

Assume the plugin '''mod_mylittleextension''' (remember: MLE is a fake plugin) was installed in the superproject and these changes have been commited:
<pre>
$ cd /path/to/your/superproject
$ git submodule add /local/repositories/mle/ mod/mylittleextension
$ git commit -a -m "New module MLE installed"
</pre>

Now a developer has to type in the following commands, in order to clone the superproject (maybe from a remote device).
<pre>
$ cd /path/to/local/directory
$ git clone <source> moodle
$ cd moodle
$ git submodule update --init --recursive
Submodule 'mod/mylittleextension' (/local/repositories/mle) registered for path 'mod/mylittleextension'
Cloning into 'mod/mylittleextension'...
done.
Submodule path 'mod/mylittleextension': checked out '89d9eae3d5142474d8452128e8df5720d89012cd'
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
</pre>

The new main repositories will automatically know about the new extension, but it won't be initialized, which means that the submodule's directory will be empty in the beginning.
For initializing the submodule, the developer needs to execute '''git submodule update --init --recursive'''.
The '''--recursive''' option means, that Git will walk through nested submodules as well (not obligatory in this example).

After the intialization the submodule's repository will be in detached HEAD state.
This is due the fact, that Git receives following data from the superproject:
* source url
* local path
* HEAD reference (hash)
See this figure, for a better understanding:

[[Image:superprojects2.png| Dotted line: information obtained from the superproject, but already saved locally (by pull)]]

This means, '''git submodule update --init''' will first do a clone of '''source url''' into the '''path''' and then check out the '''reference''', leaving the submodule in detached HEAD state.
You and your developers need to keep in mind, that any change of the HEAD's reference of your submodule will be noticed by the main repository.
This means also, that everyone can switch branches unnoticedly as long as the HEAD's hash reference doesn't change.
E.g. a checkout of the branch '''master''' won't be noticed by the main repository, but a checkout of '''dev''' will:
<pre>
mod/mylittleextension $ git checkout master
mod/mylittleextension $ git branch -av
* master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
$ cd ../..
$ git status
# On branch master
nothing to commit (working directory clean)
$ cd mod/mylittleextension
mod/mylittleextension $ git checkout dev
Branch dev set up to track remote branch dev from origin.
Switched to a new branch 'dev'
mod/mylittleextension $ cd ../..
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

There is a shorter command to initialize the submodules, but the outcome will be the same as above.
Instead of changing the moodle directory and use '''git submodule update --init''' after the clone, you can use '''git clone''' with the '''--recursive''' option as well.
<pre>
$ cd /path/to/new/location
$ git clone --recursive <source> moodle
</pre>

In other words, you may use '''git submodule update --init''' in the case you forget the '''--recursive''' option in your '''git clone''' command.

== Upgrading your submodules via the superproject ==

We now assume, that the external plugin is updated by it's own repository only and no customization is needed.
See [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] for further information on the topic of maintaining submodules of your superproject.

=== Pull and upgrade ===

In case a submodule was updated inside the superproject, your developers have to be careful.
'''git pull''' won't upgrade the submodules at once and an additional command is needed.
This is what '''git pull''' would look like:
<pre>
$ cd /path/to/local/repository
$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /local/repositories/super/
b547f6c..7f0c348 master -> origin/master
Fetching submodule mod/mylittleextension
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From /local/repositories/mle
3a2d487..5e0e66a dev -> origin/dev
89d9eae..5e0e66a master -> origin/master
Updating b547f6c..7f0c348
Fast-forward
mod/mylittleextension | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
dev 3a2d487 [behind 1] First commit
master 89d9eae [behind 2] Initial commit
remotes/origin/master 5e0e66a Second commit
remotes/origin/dev 5e0e66a Second commit
</pre>

As you see, '''git pull'' triggers '''git fetch''' inside the submodules repository, but nothing else.
Therefore, your developers have to update the submodules afterwards:
<pre>
$ cd /path/to/local/repository
$ git submodule update
Submodule path 'mod/mylittleextension': checked out '5e0e66aa787f4ebe9f61a940969a6e94abf01a1e'
</pre>

=== Figure: Updates without customization ===

[[File:superprojects3.png|center|A workflow for an upgrade from the source repository]]

=== Known issues ===

Since '''git submodule upgrade''' checks out the reference, given by your main repository's commit, your submodule's repository will turn into detached HEAD state again.
The command is equivalent to '''git checkout''' of the reference inside each submodule's path.
That means, that conflicts will abort the checkout, like entering the directory and trying to do the checkout manually.
So the submodule's repository should be clean in order to prevent problems with the submodule update.

A special case of this problem happens, if you try to clone a superproject, whose submodule's reference is not known by the origin.
Your submodule's directory would be left blank (since there's nothing to check out) and you would have to resolve this issue manually (e.g. add your superprojects submodule repository as a remote).
A situation, you don't want to get, for sure.

== Upgrading customized plugins ==

=== Figure: Suggested workflow ===

In a normal workflow you don't touch external plugins and therefore there is no need to alter the submodules in any way.
But what, if you '''want to alter''' them.
In some cases the submodule is your own plugin or the extension is somehow insufficient but the original contributor doesn't want to merge your pull requests.

As you know from the previous section '''git submodule update''' tries to checkout a specific reference.
Therefore, this reference must exist in the origin or you will get a problem while updating them.
If you're not the contributer of the extension it is necessary to fork these repositiories.
A suggested workflow would be like this:

[[File:superprojects4.png|center|Suggested workflow with customized plugins]]

Your developers are responsible for updating your forked plugin.
Therefore they must merge the original changes into your forked code and resolve conflicts, if necessary.
Your superproject pulls from the fork only.
If you're setting up your first superproject, it doesn't have to be connected to the original repositiory at all, since you'll get all updates from your fork.
So instead of adding the extension's original remote, you may add the fork only (after adding you can use '''git remote rename origin fork''' before your initial commit of the plugin).

=== Setting up a new submodule ===

In case you haven't installed the extension yet, you can add and rename the repository afterwards (to avoid misunderstanding).
On your superproject's machine do the following commands:
<pre>
$ cd /path/to/moodle
$ git submodule add <source:fork> mod/mylittleextension
Cloning into 'mod/mylittleextension'...
done.
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git branch -avv
* master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
mod/mylittleextension $ cd ../..
$ git commit -a -m "New extension 'MLE' installed"
[master adecc96] New extension 'MLE' installed
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 mod/mylittleextension
</pre>

Your developers will get your new submodule using '''git clone''' or '''git pull''' and can rename the remote likewise.
In addition, they have to fetch updates from the original remote and have to add a remote repository manually.
On your developers machine type in (output messages from Git are omitted):
<pre>
cd /path/to/local/repository
$ git pull
$ git submodule update --init
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Now your developers can add branches, edit code, do commits and contribute to your fork.
Details about that topic will follow after the next section.

=== Changes for an existing submodule ===

If the submodule is already installed and your superproject cloned, you must edit the '''.gitmodules''' file.
Since the '''path''' and '''url''' of your submodule are saved ('''reference''' is saved in your commit), you may edit the file manually.
But you can also use a Git command for that purpose.
On the superproject's machine type in (output messages from Git are omitted):
<pre>
cd /path/to/moodle/
$ git config --file=.gitmodules submodule."mod/mylittleextension".url "<source:fork>"
</pre>
'''or''' edit the file '''.gitmodules''' respectively.

Afterwards type in:
<pre>
$ git submodule sync
Synchronizing submodule url for 'mod/mylittleextension'
$ git commit -a -m "Changed source url of 'MLE'"
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
</pre>

The '''sync''' command will alter the remotes of your submodule's repository to match the settings of your '''.gitmodules'''.
It is to mention, that changes in the remotes (like '''rename''' or '''add''') of a submodule wont be noticed by the Moodle repository.
In fact, unless you edit a file or change the submodule's HEAD reference, you can do whatever you want with your branches.

At last your developer has to update his project as well.
On the developers machine type in (output messages are omitted again):
<pre>
$ cd /path/to/local/repository
$ git pull
$ git submodule update
$ git submodule sync
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Again, the '''sync''' command will change the remote settings of the submodule to match them with the '''.gitmodules'''.

=== Contributing to the superproject ===

This section will explain how to contribute patches for your superproject.
Assume your developer has changed an extension and commit his changes in the submodule's repository.
<pre>
$ cd /path/to/local/repository
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -avv
* dev 9701a0c [fork/dev: ahead 1] Third commit
master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
</pre>

Push the changes into the fork.
<pre>
mod/mylittleextension $ git push fork dev:master
</pre>

You may organize your forked repository in another way, e.g. let the developers push into a seperate branch and merge them only, when it is satisfying.
For now, we assume the developers can push into your '''master''' (or, in general, the branch, which is tracked by your superproject).
Your developer's Moodle repository will notice the changes from their commit:
<pre>
$ cd /path/to/moodle
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

'''Do not commit the changes.'''
We presume, that every change in your main repository is done through a commit from your superproject.
The status is just saying, that your submodule is on another state than your superproject.
But this is not a problem at all. We will fix this later.
'''Note: When your developers need to commit changes, which are not related to the submodule, they have to avoid to stage the submodule until the superproject has been updated.'''

Update the superproject with the following commands on the superproject's machine:
<pre>
(On the superproject's machine)
$ cd /path/to/moodle
$ git submodule foreach git pull
$ git commit -a -m "Plugin updates"
</pre>

Your developers can now fetch the updated superproject.
<pre>
(On the developer's machine)
$ cd /path/to/local/repository
$ git pull
$ git status
# On branch master
nothing to commit (working directory clean)
</pre>

As you can see, your developer don't need to commit the changed status of the submodule.
Your developer's repository will be clean again after a fetch.

=== Figure: Workflow of an update of your submodules ===

[[Image:superprojects5.png|The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.]]

== See also ==

; Moodle Docs
* [[Git for Administrators]]
* [[:dev:Git for developers|Git for developers guide]]
* [[Windows installation using Git]]
* [[Git for Mac]]
* [[Moodle_Production_Server_with_GIT|Case study Git + Moodle from Technical University Berlin]]

; External resources
* [https://git-scm.com/book/en/v2/Git-Tools-Submodules Submodules at the Git book]

Moodle development environment with Git submodules

2016-03-15T15:30:47Z

Jan-eberhardt:

This page describes how to set up a development environment for a Moodle instance with external plugins.
Over time the number of plugins may increase in such a way, that they can't be managed manually by developers anymore.
Without any precautions this can lead to various problems, for instance:
* Missing plugins in cloned repositories
* Diverged versions in existing repositories (can lead to fatal errors when trying to update the site)
* Messed up Git excludes

You'll learn in this guide how to use Git submodules in order to achieve the following setup:

[[Image:superprojects1.png|thumbs|center|Scheme of a development environment]]

'''Note: The extension '''mod_mylittleextension''' is a fake plugin, which was created for testing purposes only.'''

By using Git submodules all extensions of a repository can be simply synchronized with other projects.
All repositories are based on the so called '''superproject''' repository and the developers' projects will be sometimes referred as 'local' repository.
Anyone who is interested in maintaining the superproject and core changes by Git can visit [[Moodle Production Server with Git]].

'''Note: This guide won't cover basic Git commands and how to install/maintain Git submodules.
It is assumed, that you have read [[Git for Administrators]] where these topics are covered.
The superproject will be seen as a Moodle repository where several plugins have been already installed via Git submodules.'''

== Cloning a superproject ==

First a short explanation of events which are triggered, when a superproject is cloned.
There are some things in Git submodules, which aren't obvious and can be quite surprising.
'''Note: Ensure that you've understood the how-to at [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] before continuing.'''

Assume the plugin '''mod_mylittleextension''' (remember: MLE is a fake plugin) was installed in the superproject and these changes have been commited:
<pre>
$ cd /path/to/your/superproject
$ git submodule add /local/repositories/mle/ mod/mylittleextension
$ git commit -a -m "New module MLE installed"
</pre>

Now a developer has to type in the following commands, in order to clone the superproject (maybe from a remote device).
<pre>
$ cd /path/to/local/directory
$ git clone <source> moodle
$ cd moodle
$ git submodule update --init --recursive
Submodule 'mod/mylittleextension' (/local/repositories/mle) registered for path 'mod/mylittleextension'
Cloning into 'mod/mylittleextension'...
done.
Submodule path 'mod/mylittleextension': checked out '89d9eae3d5142474d8452128e8df5720d89012cd'
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
</pre>

The new main repositories will automatically know about the new extension, but it won't be initialized, which means that the submodule's directory will be empty in the beginning.
For initializing the submodule, the developer needs to execute '''git submodule update --init --recursive'''.
The '''--recursive''' option means, that Git will walk through nested submodules as well (not obligatory in this example).

After the intialization the submodule's repository will be in detached HEAD state.
This is due the fact, that Git receives following data from the superproject:
* source url
* local path
* HEAD reference (hash)
See this figure, for a better understanding:

[[Image:superprojects2.png| Dotted line: information obtained from the superproject, but already saved locally (by pull)]]

This means, '''git submodule update --init''' will first do a clone of '''source url''' into the '''path''' and then check out the '''reference''', leaving the submodule in detached HEAD state.
You and your developers need to keep in mind, that any change of the HEAD's reference of your submodule will be noticed by the main repository.
This means also, that everyone can switch branches unnoticedly as long as the HEAD's hash reference doesn't change.
E.g. a checkout of the branch '''master''' won't be noticed by the main repository, but a checkout of '''dev''' will:
<pre>
mod/mylittleextension $ git checkout master
mod/mylittleextension $ git branch -av
* master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
$ cd ../..
$ git status
# On branch master
nothing to commit (working directory clean)
$ cd mod/mylittleextension
mod/mylittleextension $ git checkout dev
Branch dev set up to track remote branch dev from origin.
Switched to a new branch 'dev'
mod/mylittleextension $ cd ../..
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

There is a shorter command to initialize the submodules, but the outcome will be the same as above.
Instead of changing the moodle directory and use '''git submodule update --init''' after the clone, you can use '''git clone''' with the '''--recursive''' option as well.
<pre>
$ cd /path/to/new/location
$ git clone --recursive <source> moodle
</pre>

In other words, you may use '''git submodule update --init''' in the case you forget the '''--recursive''' option in your '''git clone''' command.

== Upgrading your submodules via the superproject ==

We now assume, that the external plugin is updated by it's own repository only and no customization is needed.
See [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] for further information on the topic of maintaining submodules of your superproject.

=== Pull and upgrade ===

In case a submodule was updated inside the superproject, your developers have to be careful.
'''git pull''' won't upgrade the submodules at once and an additional command is needed.
This is what '''git pull''' would look like:
<pre>
$ cd /path/to/local/repository
$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /local/repositories/super/
b547f6c..7f0c348 master -> origin/master
Fetching submodule mod/mylittleextension
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From /local/repositories/mle
3a2d487..5e0e66a dev -> origin/dev
89d9eae..5e0e66a master -> origin/master
Updating b547f6c..7f0c348
Fast-forward
mod/mylittleextension | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
dev 3a2d487 [behind 1] First commit
master 89d9eae [behind 2] Initial commit
remotes/origin/master 5e0e66a Second commit
remotes/origin/dev 5e0e66a Second commit
</pre>

As you see, '''git pull'' triggers '''git fetch''' inside the submodules repository, but nothing else.
Therefore, your developers have to update the submodules afterwards:
<pre>
$ cd /path/to/local/repository
$ git submodule update
Submodule path 'mod/mylittleextension': checked out '5e0e66aa787f4ebe9f61a940969a6e94abf01a1e'
</pre>

=== Figure: Updates without customization ===

[[File:superprojects3.png|center|A workflow for an upgrade from the source repository]]

=== Known issues ===

Since '''git submodule upgrade''' checks out the reference, given by your main repository's commit, your submodule's repository will turn into detached HEAD state again.
The command is equivalent to '''git checkout''' of the reference inside each submodule's path.
That means, that conflicts will abort the checkout, like entering the directory and trying to do the checkout manually.
So the submodule's repository should be clean in order to prevent problems with the submodule update.

A special case of this problem happens, if you try to clone a superproject, whose submodule's reference is not known by the origin.
Your submodule's directory would be left blank (since there's nothing to check out) and you would have to resolve this issue manually (e.g. add your superprojects submodule repository as a remote).
A situation, you don't want to get, for sure.

== Upgrading customized plugins ==

=== Figure: Suggested workflow ===

In a normal workflow you don't touch external plugins and therefore there is no need to alter the submodules in any way.
But what, if you '''want to alter''' them.
In some cases the submodule is your own plugin or the extension is somehow insufficient but the original contributor doesn't want to merge your pull requests.

As you know from the previous section '''git submodule update''' tries to checkout a specific reference.
Therefore, this reference must exist in the origin or you will get a problem while updating them.
If you're not the contributer of the extension it is necessary to fork these repositiories.
A suggested workflow would be like this:

[[File:superprojects4.png|center|Suggested workflow with customized plugins]]

Your developers are responsible for updating your forked plugin.
Therefore they must merge the original changes into your forked code and resolve conflicts, if necessary.
Your superproject pulls from the fork only.
If you're setting up your first superproject, it doesn't have to be connected to the original repositiory at all, since you'll get all updates from your fork.
So instead of adding the extension's original remote, you may add the fork only (after adding you can use '''git remote rename origin fork''' before your initial commit of the plugin).

=== Setting up a new submodule ===

In case you haven't installed the extension yet, you can add and rename the repository afterwards (to avoid misunderstanding).
On your superproject's machine do the following commands:
<pre>
$ cd /path/to/moodle
$ git submodule add <source:fork> mod/mylittleextension
Cloning into 'mod/mylittleextension'...
done.
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git branch -avv
* master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
mod/mylittleextension $ cd ../..
$ git commit -a -m "New extension 'MLE' installed"
[master adecc96] New extension 'MLE' installed
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 mod/mylittleextension
</pre>

Your developers will get your new submodule using '''git clone''' or '''git pull''' and can rename the remote likewise.
In addition, they have to fetch updates from the original remote and have to add a remote repository manually.
On your developers machine type in (output messages from Git are omitted):
<pre>
cd /path/to/local/repository
$ git pull
$ git submodule update --init
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Now your developers can add branches, edit code, do commits and contribute to your fork.
Details about that topic will follow after the next section.

=== Changes for an existing submodule ===

If the submodule is already installed and your superproject cloned, you must edit the '''.gitmodules''' file.
Since the '''path''' and '''url''' of your submodule are saved ('''reference''' is saved in your commit), you may edit the file manually.
But you can also use a Git command for that purpose.
On the superproject's machine type in (output messages from Git are omitted):
<pre>
cd /path/to/moodle/
$ git config --file=.gitmodules submodule."mod/mylittleextension".url "<source:fork>"
</pre>
'''or''' edit the file '''.gitmodules''' respectively.

Afterwards type in:
<pre>
$ git submodule sync
Synchronizing submodule url for 'mod/mylittleextension'
$ git commit -a -m "Changed source url of 'MLE'"
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
</pre>

The '''sync''' command will alter the remotes of your submodule's repository to match the settings of your '''.gitmodules'''.
It is to mention, that changes in the remotes (like '''rename''' or '''add''') of a submodule wont be noticed by the Moodle repository.
In fact, unless you edit a file or change the submodule's HEAD reference, you can do whatever you want with your branches.

At last your developer has to update his project as well.
On the developers machine type in (output messages are omitted again):
<pre>
$ cd /path/to/local/repository
$ git pull
$ git submodule update
$ git submodule sync
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Again, the '''sync''' command will change the remote settings of the submodule to match them with the '''.gitmodules'''.

=== Contributing to the superproject ===

This section will explain how to contribute patches for your superproject.
Assume your developer has changed an extension and commit his changes in the submodule's repository.
<pre>
$ cd /path/to/local/repository
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -avv
* dev 9701a0c [fork/dev: ahead 1] Third commit
master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
</pre>

Push the changes into the fork.
<pre>
mod/mylittleextension $ git push fork dev:master
</pre>

You may organize your forked repository in another way, e.g. let the developers push into a seperate branch and merge them only, when it is satisfying.
For now, we assume the developers can push into your '''master''' (or, in general, the branch, which is tracked by your superproject).
Your developer's Moodle repository will notice the changes from their commit:
<pre>
$ cd /path/to/moodle
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

'''Do not commit the changes.'''
We presume, that every change in your main repository is done through a commit from your superproject.
The status is just saying, that your submodule is on another state than your superproject.
But this is not a problem at all. We will fix this later.
'''Note: When your developers need to commit changes, which are not related to the submodule, they have to avoid to stage the submodule until the superproject has been updated.'''

Update the superproject with the following commands on the superproject's machine:
<pre>
(On the superproject's machine)
$ cd /path/to/moodle
$ git submodule foreach git pull
$ git commit -a -m "Plugin updates"
</pre>

Your developers can now fetch the updated superproject.
<pre>
(On the developer's machine)
$ cd /path/to/local/repository
$ git pull
$ git status
# On branch master
nothing to commit (working directory clean)
</pre>

As you can see, your developer don't need to commit the changed status of the submodule.
Your developer's repository will be clean again after a fetch.

=== Figure: Workflow of an update of your submodules ===

[[Image:superprojects5.png|The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.]]

== See also ==

; Moodle Docs
* [[Git for Administrators]]
* [[:dev:Git for developers|Git for developers guide]]
* [[Windows installation using Git]]
* [[Git for Mac]]
* [[Moodle_Production_Server_with_GIT|Case study Git + Moodle from Technical University Berlin]]

; External resources
* [https://git-scm.com/book/en/v2/Git-Tools-Submodules Submodules at the Git book]

Moodle development environment with Git submodules

2016-03-15T15:28:34Z

Jan-eberhardt: /* See also */

This page describes how to set up a development environment for a Moodle instance with external plugins.
Over time the number of plugins may increase in such a way, that they can't be managed manually by developers anymore.
Without any precautions this can lead to various problems, for instance:
* Missing plugins in cloned repositories
* Diverged versions in existing repositories (can lead to fatal errors when trying to update the site)
* Messed up Git excludes

You'll learn in this guide how to use Git submodules in order to achieve the following setup:

[[Image:superprojects1.png|center|Scheme of a development environment]]

'''Note: The extension '''mod_mylittleextension''' is a fake plugin, which was created for testing purposes only.'''

By using Git submodules all extensions of a repository can be simply synchronized with other projects.
All repositories are based on the so called '''superproject''' repository and the developers' projects will be sometimes referred as 'local' repository.
Anyone who is interested in maintaining the superproject and core changes by Git can visit [[Moodle Production Server with Git]].

'''Note: This guide won't cover basic Git commands and how to install/maintain Git submodules.
It is assumed, that you have read [[Git for Administrators]] where these topics are covered.
The superproject will be seen as a Moodle repository where several plugins have been already installed via Git submodules.'''

== Cloning a superproject ==

First a short explanation of events which are triggered, when a superproject is cloned.
There are some things in Git submodules, which aren't obvious and can be quite surprising.
'''Note: Ensure that you've understood the how-to at [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] before continuing.'''

Assume the plugin '''mod_mylittleextension''' (remember: MLE is a fake plugin) was installed in the superproject and these changes have been commited:
<pre>
$ cd /path/to/your/superproject
$ git submodule add /local/repositories/mle/ mod/mylittleextension
$ git commit -a -m "New module MLE installed"
</pre>

Now a developer has to type in the following commands, in order to clone the superproject (maybe from a remote device).
<pre>
$ cd /path/to/local/directory
$ git clone <source> moodle
$ cd moodle
$ git submodule update --init --recursive
Submodule 'mod/mylittleextension' (/local/repositories/mle) registered for path 'mod/mylittleextension'
Cloning into 'mod/mylittleextension'...
done.
Submodule path 'mod/mylittleextension': checked out '89d9eae3d5142474d8452128e8df5720d89012cd'
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
</pre>

The new main repositories will automatically know about the new extension, but it won't be initialized, which means that the submodule's directory will be empty in the beginning.
For initializing the submodule, the developer needs to execute '''git submodule update --init --recursive'''.
The '''--recursive''' option means, that Git will walk through nested submodules as well (not obligatory in this example).

After the intialization the submodule's repository will be in detached HEAD state.
This is due the fact, that Git receives following data from the superproject:
* source url
* local path
* HEAD reference (hash)
See this figure, for a better understanding:

[[Image:superprojects2.png| Dotted line: information obtained from the superproject, but already saved locally (by pull)]]

This means, '''git submodule update --init''' will first do a clone of '''source url''' into the '''path''' and then check out the '''reference''', leaving the submodule in detached HEAD state.
You and your developers need to keep in mind, that any change of the HEAD's reference of your submodule will be noticed by the main repository.
This means also, that everyone can switch branches unnoticedly as long as the HEAD's hash reference doesn't change.
E.g. a checkout of the branch '''master''' won't be noticed by the main repository, but a checkout of '''dev''' will:
<pre>
mod/mylittleextension $ git checkout master
mod/mylittleextension $ git branch -av
* master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
$ cd ../..
$ git status
# On branch master
nothing to commit (working directory clean)
$ cd mod/mylittleextension
mod/mylittleextension $ git checkout dev
Branch dev set up to track remote branch dev from origin.
Switched to a new branch 'dev'
mod/mylittleextension $ cd ../..
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

There is a shorter command to initialize the submodules, but the outcome will be the same as above.
Instead of changing the moodle directory and use '''git submodule update --init''' after the clone, you can use '''git clone''' with the '''--recursive''' option as well.
<pre>
$ cd /path/to/new/location
$ git clone --recursive <source> moodle
</pre>

In other words, you may use '''git submodule update --init''' in the case you forget the '''--recursive''' option in your '''git clone''' command.

== Upgrading your submodules via the superproject ==

We now assume, that the external plugin is updated by it's own repository only and no customization is needed.
See [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] for further information on the topic of maintaining submodules of your superproject.

=== Pull and upgrade ===

In case a submodule was updated inside the superproject, your developers have to be careful.
'''git pull''' won't upgrade the submodules at once and an additional command is needed.
This is what '''git pull''' would look like:
<pre>
$ cd /path/to/local/repository
$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /local/repositories/super/
b547f6c..7f0c348 master -> origin/master
Fetching submodule mod/mylittleextension
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From /local/repositories/mle
3a2d487..5e0e66a dev -> origin/dev
89d9eae..5e0e66a master -> origin/master
Updating b547f6c..7f0c348
Fast-forward
mod/mylittleextension | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
dev 3a2d487 [behind 1] First commit
master 89d9eae [behind 2] Initial commit
remotes/origin/master 5e0e66a Second commit
remotes/origin/dev 5e0e66a Second commit
</pre>

As you see, '''git pull'' triggers '''git fetch''' inside the submodules repository, but nothing else.
Therefore, your developers have to update the submodules afterwards:
<pre>
$ cd /path/to/local/repository
$ git submodule update
Submodule path 'mod/mylittleextension': checked out '5e0e66aa787f4ebe9f61a940969a6e94abf01a1e'
</pre>

=== Figure: Updates without customization ===

[[File:superprojects3.png|center|A workflow for an upgrade from the source repository]]

=== Known issues ===

Since '''git submodule upgrade''' checks out the reference, given by your main repository's commit, your submodule's repository will turn into detached HEAD state again.
The command is equivalent to '''git checkout''' of the reference inside each submodule's path.
That means, that conflicts will abort the checkout, like entering the directory and trying to do the checkout manually.
So the submodule's repository should be clean in order to prevent problems with the submodule update.

A special case of this problem happens, if you try to clone a superproject, whose submodule's reference is not known by the origin.
Your submodule's directory would be left blank (since there's nothing to check out) and you would have to resolve this issue manually (e.g. add your superprojects submodule repository as a remote).
A situation, you don't want to get, for sure.

== Upgrading customized plugins ==

=== Figure: Suggested workflow ===

In a normal workflow you don't touch external plugins and therefore there is no need to alter the submodules in any way.
But what, if you '''want to alter''' them.
In some cases the submodule is your own plugin or the extension is somehow insufficient but the original contributor doesn't want to merge your pull requests.

As you know from the previous section '''git submodule update''' tries to checkout a specific reference.
Therefore, this reference must exist in the origin or you will get a problem while updating them.
If you're not the contributer of the extension it is necessary to fork these repositiories.
A suggested workflow would be like this:

[[File:superprojects4.png|center|Suggested workflow with customized plugins]]

Your developers are responsible for updating your forked plugin.
Therefore they must merge the original changes into your forked code and resolve conflicts, if necessary.
Your superproject pulls from the fork only.
If you're setting up your first superproject, it doesn't have to be connected to the original repositiory at all, since you'll get all updates from your fork.
So instead of adding the extension's original remote, you may add the fork only (after adding you can use '''git remote rename origin fork''' before your initial commit of the plugin).

=== Setting up a new submodule ===

In case you haven't installed the extension yet, you can add and rename the repository afterwards (to avoid misunderstanding).
On your superproject's machine do the following commands:
<pre>
$ cd /path/to/moodle
$ git submodule add <source:fork> mod/mylittleextension
Cloning into 'mod/mylittleextension'...
done.
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git branch -avv
* master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
mod/mylittleextension $ cd ../..
$ git commit -a -m "New extension 'MLE' installed"
[master adecc96] New extension 'MLE' installed
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 mod/mylittleextension
</pre>

Your developers will get your new submodule using '''git clone''' or '''git pull''' and can rename the remote likewise.
In addition, they have to fetch updates from the original remote and have to add a remote repository manually.
On your developers machine type in (output messages from Git are omitted):
<pre>
cd /path/to/local/repository
$ git pull
$ git submodule update --init
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Now your developers can add branches, edit code, do commits and contribute to your fork.
Details about that topic will follow after the next section.

=== Changes for an existing submodule ===

If the submodule is already installed and your superproject cloned, you must edit the '''.gitmodules''' file.
Since the '''path''' and '''url''' of your submodule are saved ('''reference''' is saved in your commit), you may edit the file manually.
But you can also use a Git command for that purpose.
On the superproject's machine type in (output messages from Git are omitted):
<pre>
cd /path/to/moodle/
$ git config --file=.gitmodules submodule."mod/mylittleextension".url "<source:fork>"
</pre>
'''or''' edit the file '''.gitmodules''' respectively.

Afterwards type in:
<pre>
$ git submodule sync
Synchronizing submodule url for 'mod/mylittleextension'
$ git commit -a -m "Changed source url of 'MLE'"
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
</pre>

The '''sync''' command will alter the remotes of your submodule's repository to match the settings of your '''.gitmodules'''.
It is to mention, that changes in the remotes (like '''rename''' or '''add''') of a submodule wont be noticed by the Moodle repository.
In fact, unless you edit a file or change the submodule's HEAD reference, you can do whatever you want with your branches.

At last your developer has to update his project as well.
On the developers machine type in (output messages are omitted again):
<pre>
$ cd /path/to/local/repository
$ git pull
$ git submodule update
$ git submodule sync
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Again, the '''sync''' command will change the remote settings of the submodule to match them with the '''.gitmodules'''.

=== Contributing to the superproject ===

This section will explain how to contribute patches for your superproject.
Assume your developer has changed an extension and commit his changes in the submodule's repository.
<pre>
$ cd /path/to/local/repository
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -avv
* dev 9701a0c [fork/dev: ahead 1] Third commit
master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
</pre>

Push the changes into the fork.
<pre>
mod/mylittleextension $ git push fork dev:master
</pre>

You may organize your forked repository in another way, e.g. let the developers push into a seperate branch and merge them only, when it is satisfying.
For now, we assume the developers can push into your '''master''' (or, in general, the branch, which is tracked by your superproject).
Your developer's Moodle repository will notice the changes from their commit:
<pre>
$ cd /path/to/moodle
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

'''Do not commit the changes.'''
We presume, that every change in your main repository is done through a commit from your superproject.
The status is just saying, that your submodule is on another state than your superproject.
But this is not a problem at all. We will fix this later.
'''Note: When your developers need to commit changes, which are not related to the submodule, they have to avoid to stage the submodule until the superproject has been updated.'''

Update the superproject with the following commands on the superproject's machine:
<pre>
(On the superproject's machine)
$ cd /path/to/moodle
$ git submodule foreach git pull
$ git commit -a -m "Plugin updates"
</pre>

Your developers can now fetch the updated superproject.
<pre>
(On the developer's machine)
$ cd /path/to/local/repository
$ git pull
$ git status
# On branch master
nothing to commit (working directory clean)
</pre>

As you can see, your developer don't need to commit the changed status of the submodule.
Your developer's repository will be clean again after a fetch.

=== Figure: Workflow of an update of your submodules ===

[[Image:superprojects5.png|The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.]]

== See also ==

; Moodle Docs
* [[Git for Administrators]]
* [[:dev:Git for developers|Git for developers guide]]
* [[Windows installation using Git]]
* [[Git for Mac]]
* [[Moodle_Production_Server_with_GIT|Case study Git + Moodle from Technical University Berlin]]

; External resources
* [https://git-scm.com/book/en/v2/Git-Tools-Submodules Submodules at the Git book]

File:superprojects5.png

2016-03-15T15:26:09Z

Jan-eberhardt: The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.

The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.

Moodle development environment with Git submodules

2016-03-15T15:25:32Z

Jan-eberhardt: /* Figure: Suggested workflow */

This page describes how to set up a development environment for a Moodle instance with external plugins.
Over time the number of plugins may increase in such a way, that they can't be managed manually by developers anymore.
Without any precautions this can lead to various problems, for instance:
* Missing plugins in cloned repositories
* Diverged versions in existing repositories (can lead to fatal errors when trying to update the site)
* Messed up Git excludes

You'll learn in this guide how to use Git submodules in order to achieve the following setup:

[[Image:superprojects1.png|center|Scheme of a development environment]]

'''Note: The extension '''mod_mylittleextension''' is a fake plugin, which was created for testing purposes only.'''

By using Git submodules all extensions of a repository can be simply synchronized with other projects.
All repositories are based on the so called '''superproject''' repository and the developers' projects will be sometimes referred as 'local' repository.
Anyone who is interested in maintaining the superproject and core changes by Git can visit [[Moodle Production Server with Git]].

'''Note: This guide won't cover basic Git commands and how to install/maintain Git submodules.
It is assumed, that you have read [[Git for Administrators]] where these topics are covered.
The superproject will be seen as a Moodle repository where several plugins have been already installed via Git submodules.'''

== Cloning a superproject ==

First a short explanation of events which are triggered, when a superproject is cloned.
There are some things in Git submodules, which aren't obvious and can be quite surprising.
'''Note: Ensure that you've understood the how-to at [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] before continuing.'''

Assume the plugin '''mod_mylittleextension''' (remember: MLE is a fake plugin) was installed in the superproject and these changes have been commited:
<pre>
$ cd /path/to/your/superproject
$ git submodule add /local/repositories/mle/ mod/mylittleextension
$ git commit -a -m "New module MLE installed"
</pre>

Now a developer has to type in the following commands, in order to clone the superproject (maybe from a remote device).
<pre>
$ cd /path/to/local/directory
$ git clone <source> moodle
$ cd moodle
$ git submodule update --init --recursive
Submodule 'mod/mylittleextension' (/local/repositories/mle) registered for path 'mod/mylittleextension'
Cloning into 'mod/mylittleextension'...
done.
Submodule path 'mod/mylittleextension': checked out '89d9eae3d5142474d8452128e8df5720d89012cd'
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
</pre>

The new main repositories will automatically know about the new extension, but it won't be initialized, which means that the submodule's directory will be empty in the beginning.
For initializing the submodule, the developer needs to execute '''git submodule update --init --recursive'''.
The '''--recursive''' option means, that Git will walk through nested submodules as well (not obligatory in this example).

After the intialization the submodule's repository will be in detached HEAD state.
This is due the fact, that Git receives following data from the superproject:
* source url
* local path
* HEAD reference (hash)
See this figure, for a better understanding:

[[Image:superprojects2.png| Dotted line: information obtained from the superproject, but already saved locally (by pull)]]

This means, '''git submodule update --init''' will first do a clone of '''source url''' into the '''path''' and then check out the '''reference''', leaving the submodule in detached HEAD state.
You and your developers need to keep in mind, that any change of the HEAD's reference of your submodule will be noticed by the main repository.
This means also, that everyone can switch branches unnoticedly as long as the HEAD's hash reference doesn't change.
E.g. a checkout of the branch '''master''' won't be noticed by the main repository, but a checkout of '''dev''' will:
<pre>
mod/mylittleextension $ git checkout master
mod/mylittleextension $ git branch -av
* master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
$ cd ../..
$ git status
# On branch master
nothing to commit (working directory clean)
$ cd mod/mylittleextension
mod/mylittleextension $ git checkout dev
Branch dev set up to track remote branch dev from origin.
Switched to a new branch 'dev'
mod/mylittleextension $ cd ../..
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

There is a shorter command to initialize the submodules, but the outcome will be the same as above.
Instead of changing the moodle directory and use '''git submodule update --init''' after the clone, you can use '''git clone''' with the '''--recursive''' option as well.
<pre>
$ cd /path/to/new/location
$ git clone --recursive <source> moodle
</pre>

In other words, you may use '''git submodule update --init''' in the case you forget the '''--recursive''' option in your '''git clone''' command.

== Upgrading your submodules via the superproject ==

We now assume, that the external plugin is updated by it's own repository only and no customization is needed.
See [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] for further information on the topic of maintaining submodules of your superproject.

=== Pull and upgrade ===

In case a submodule was updated inside the superproject, your developers have to be careful.
'''git pull''' won't upgrade the submodules at once and an additional command is needed.
This is what '''git pull''' would look like:
<pre>
$ cd /path/to/local/repository
$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /local/repositories/super/
b547f6c..7f0c348 master -> origin/master
Fetching submodule mod/mylittleextension
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From /local/repositories/mle
3a2d487..5e0e66a dev -> origin/dev
89d9eae..5e0e66a master -> origin/master
Updating b547f6c..7f0c348
Fast-forward
mod/mylittleextension | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
dev 3a2d487 [behind 1] First commit
master 89d9eae [behind 2] Initial commit
remotes/origin/master 5e0e66a Second commit
remotes/origin/dev 5e0e66a Second commit
</pre>

As you see, '''git pull'' triggers '''git fetch''' inside the submodules repository, but nothing else.
Therefore, your developers have to update the submodules afterwards:
<pre>
$ cd /path/to/local/repository
$ git submodule update
Submodule path 'mod/mylittleextension': checked out '5e0e66aa787f4ebe9f61a940969a6e94abf01a1e'
</pre>

=== Figure: Updates without customization ===

[[File:superprojects3.png|center|A workflow for an upgrade from the source repository]]

=== Known issues ===

Since '''git submodule upgrade''' checks out the reference, given by your main repository's commit, your submodule's repository will turn into detached HEAD state again.
The command is equivalent to '''git checkout''' of the reference inside each submodule's path.
That means, that conflicts will abort the checkout, like entering the directory and trying to do the checkout manually.
So the submodule's repository should be clean in order to prevent problems with the submodule update.

A special case of this problem happens, if you try to clone a superproject, whose submodule's reference is not known by the origin.
Your submodule's directory would be left blank (since there's nothing to check out) and you would have to resolve this issue manually (e.g. add your superprojects submodule repository as a remote).
A situation, you don't want to get, for sure.

== Upgrading customized plugins ==

=== Figure: Suggested workflow ===

In a normal workflow you don't touch external plugins and therefore there is no need to alter the submodules in any way.
But what, if you '''want to alter''' them.
In some cases the submodule is your own plugin or the extension is somehow insufficient but the original contributor doesn't want to merge your pull requests.

As you know from the previous section '''git submodule update''' tries to checkout a specific reference.
Therefore, this reference must exist in the origin or you will get a problem while updating them.
If you're not the contributer of the extension it is necessary to fork these repositiories.
A suggested workflow would be like this:

[[File:superprojects4.png|center|Suggested workflow with customized plugins]]

Your developers are responsible for updating your forked plugin.
Therefore they must merge the original changes into your forked code and resolve conflicts, if necessary.
Your superproject pulls from the fork only.
If you're setting up your first superproject, it doesn't have to be connected to the original repositiory at all, since you'll get all updates from your fork.
So instead of adding the extension's original remote, you may add the fork only (after adding you can use '''git remote rename origin fork''' before your initial commit of the plugin).

=== Setting up a new submodule ===

In case you haven't installed the extension yet, you can add and rename the repository afterwards (to avoid misunderstanding).
On your superproject's machine do the following commands:
<pre>
$ cd /path/to/moodle
$ git submodule add <source:fork> mod/mylittleextension
Cloning into 'mod/mylittleextension'...
done.
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git branch -avv
* master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
mod/mylittleextension $ cd ../..
$ git commit -a -m "New extension 'MLE' installed"
[master adecc96] New extension 'MLE' installed
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 mod/mylittleextension
</pre>

Your developers will get your new submodule using '''git clone''' or '''git pull''' and can rename the remote likewise.
In addition, they have to fetch updates from the original remote and have to add a remote repository manually.
On your developers machine type in (output messages from Git are omitted):
<pre>
cd /path/to/local/repository
$ git pull
$ git submodule update --init
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Now your developers can add branches, edit code, do commits and contribute to your fork.
Details about that topic will follow after the next section.

=== Changes for an existing submodule ===

If the submodule is already installed and your superproject cloned, you must edit the '''.gitmodules''' file.
Since the '''path''' and '''url''' of your submodule are saved ('''reference''' is saved in your commit), you may edit the file manually.
But you can also use a Git command for that purpose.
On the superproject's machine type in (output messages from Git are omitted):
<pre>
cd /path/to/moodle/
$ git config --file=.gitmodules submodule."mod/mylittleextension".url "<source:fork>"
</pre>
'''or''' edit the file '''.gitmodules''' respectively.

Afterwards type in:
<pre>
$ git submodule sync
Synchronizing submodule url for 'mod/mylittleextension'
$ git commit -a -m "Changed source url of 'MLE'"
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
</pre>

The '''sync''' command will alter the remotes of your submodule's repository to match the settings of your '''.gitmodules'''.
It is to mention, that changes in the remotes (like '''rename''' or '''add''') of a submodule wont be noticed by the Moodle repository.
In fact, unless you edit a file or change the submodule's HEAD reference, you can do whatever you want with your branches.

At last your developer has to update his project as well.
On the developers machine type in (output messages are omitted again):
<pre>
$ cd /path/to/local/repository
$ git pull
$ git submodule update
$ git submodule sync
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Again, the '''sync''' command will change the remote settings of the submodule to match them with the '''.gitmodules'''.

=== Contributing to the superproject ===

This section will explain how to contribute patches for your superproject.
Assume your developer has changed an extension and commit his changes in the submodule's repository.
<pre>
$ cd /path/to/local/repository
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -avv
* dev 9701a0c [fork/dev: ahead 1] Third commit
master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
</pre>

Push the changes into the fork.
<pre>
mod/mylittleextension $ git push fork dev:master
</pre>

You may organize your forked repository in another way, e.g. let the developers push into a seperate branch and merge them only, when it is satisfying.
For now, we assume the developers can push into your '''master''' (or, in general, the branch, which is tracked by your superproject).
Your developer's Moodle repository will notice the changes from their commit:
<pre>
$ cd /path/to/moodle
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

'''Do not commit the changes.'''
We presume, that every change in your main repository is done through a commit from your superproject.
The status is just saying, that your submodule is on another state than your superproject.
But this is not a problem at all. We will fix this later.
'''Note: When your developers need to commit changes, which are not related to the submodule, they have to avoid to stage the submodule until the superproject has been updated.'''

Update the superproject with the following commands on the superproject's machine:
<pre>
(On the superproject's machine)
$ cd /path/to/moodle
$ git submodule foreach git pull
$ git commit -a -m "Plugin updates"
</pre>

Your developers can now fetch the updated superproject.
<pre>
(On the developer's machine)
$ cd /path/to/local/repository
$ git pull
$ git status
# On branch master
nothing to commit (working directory clean)
</pre>

As you can see, your developer don't need to commit the changed status of the submodule.
Your developer's repository will be clean again after a fetch.

=== Figure: Workflow of an update of your submodules ===

[[Image:superprojects5.png|The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.]]

== See also ==

; Moodle Docs
* [[Git for Administrators]]
* [[Git for developers guide]]
* [[Git FAQ]]
* [[Windows installation using Git]]
* [[Git for Mac]]
* [[Moodle_Production_Server_with_GIT|Case study Git + Moodle from Technical University Berlin]]

; External resources
* [https://git-scm.com/book/en/v2/Git-Tools-Submodules Submodules at the Git book]

File:superprojects4.png

2016-03-15T15:24:14Z

Jan-eberhardt: Suggested workflow with customized plugins

Suggested workflow with customized plugins

Moodle development environment with Git submodules

2016-03-15T15:23:48Z

Jan-eberhardt: /* Figure: Updates without customization */

This page describes how to set up a development environment for a Moodle instance with external plugins.
Over time the number of plugins may increase in such a way, that they can't be managed manually by developers anymore.
Without any precautions this can lead to various problems, for instance:
* Missing plugins in cloned repositories
* Diverged versions in existing repositories (can lead to fatal errors when trying to update the site)
* Messed up Git excludes

You'll learn in this guide how to use Git submodules in order to achieve the following setup:

[[Image:superprojects1.png|center|Scheme of a development environment]]

'''Note: The extension '''mod_mylittleextension''' is a fake plugin, which was created for testing purposes only.'''

By using Git submodules all extensions of a repository can be simply synchronized with other projects.
All repositories are based on the so called '''superproject''' repository and the developers' projects will be sometimes referred as 'local' repository.
Anyone who is interested in maintaining the superproject and core changes by Git can visit [[Moodle Production Server with Git]].

'''Note: This guide won't cover basic Git commands and how to install/maintain Git submodules.
It is assumed, that you have read [[Git for Administrators]] where these topics are covered.
The superproject will be seen as a Moodle repository where several plugins have been already installed via Git submodules.'''

== Cloning a superproject ==

First a short explanation of events which are triggered, when a superproject is cloned.
There are some things in Git submodules, which aren't obvious and can be quite surprising.
'''Note: Ensure that you've understood the how-to at [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] before continuing.'''

Assume the plugin '''mod_mylittleextension''' (remember: MLE is a fake plugin) was installed in the superproject and these changes have been commited:
<pre>
$ cd /path/to/your/superproject
$ git submodule add /local/repositories/mle/ mod/mylittleextension
$ git commit -a -m "New module MLE installed"
</pre>

Now a developer has to type in the following commands, in order to clone the superproject (maybe from a remote device).
<pre>
$ cd /path/to/local/directory
$ git clone <source> moodle
$ cd moodle
$ git submodule update --init --recursive
Submodule 'mod/mylittleextension' (/local/repositories/mle) registered for path 'mod/mylittleextension'
Cloning into 'mod/mylittleextension'...
done.
Submodule path 'mod/mylittleextension': checked out '89d9eae3d5142474d8452128e8df5720d89012cd'
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
</pre>

The new main repositories will automatically know about the new extension, but it won't be initialized, which means that the submodule's directory will be empty in the beginning.
For initializing the submodule, the developer needs to execute '''git submodule update --init --recursive'''.
The '''--recursive''' option means, that Git will walk through nested submodules as well (not obligatory in this example).

After the intialization the submodule's repository will be in detached HEAD state.
This is due the fact, that Git receives following data from the superproject:
* source url
* local path
* HEAD reference (hash)
See this figure, for a better understanding:

[[Image:superprojects2.png| Dotted line: information obtained from the superproject, but already saved locally (by pull)]]

This means, '''git submodule update --init''' will first do a clone of '''source url''' into the '''path''' and then check out the '''reference''', leaving the submodule in detached HEAD state.
You and your developers need to keep in mind, that any change of the HEAD's reference of your submodule will be noticed by the main repository.
This means also, that everyone can switch branches unnoticedly as long as the HEAD's hash reference doesn't change.
E.g. a checkout of the branch '''master''' won't be noticed by the main repository, but a checkout of '''dev''' will:
<pre>
mod/mylittleextension $ git checkout master
mod/mylittleextension $ git branch -av
* master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
$ cd ../..
$ git status
# On branch master
nothing to commit (working directory clean)
$ cd mod/mylittleextension
mod/mylittleextension $ git checkout dev
Branch dev set up to track remote branch dev from origin.
Switched to a new branch 'dev'
mod/mylittleextension $ cd ../..
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

There is a shorter command to initialize the submodules, but the outcome will be the same as above.
Instead of changing the moodle directory and use '''git submodule update --init''' after the clone, you can use '''git clone''' with the '''--recursive''' option as well.
<pre>
$ cd /path/to/new/location
$ git clone --recursive <source> moodle
</pre>

In other words, you may use '''git submodule update --init''' in the case you forget the '''--recursive''' option in your '''git clone''' command.

== Upgrading your submodules via the superproject ==

We now assume, that the external plugin is updated by it's own repository only and no customization is needed.
See [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] for further information on the topic of maintaining submodules of your superproject.

=== Pull and upgrade ===

In case a submodule was updated inside the superproject, your developers have to be careful.
'''git pull''' won't upgrade the submodules at once and an additional command is needed.
This is what '''git pull''' would look like:
<pre>
$ cd /path/to/local/repository
$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /local/repositories/super/
b547f6c..7f0c348 master -> origin/master
Fetching submodule mod/mylittleextension
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From /local/repositories/mle
3a2d487..5e0e66a dev -> origin/dev
89d9eae..5e0e66a master -> origin/master
Updating b547f6c..7f0c348
Fast-forward
mod/mylittleextension | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
dev 3a2d487 [behind 1] First commit
master 89d9eae [behind 2] Initial commit
remotes/origin/master 5e0e66a Second commit
remotes/origin/dev 5e0e66a Second commit
</pre>

As you see, '''git pull'' triggers '''git fetch''' inside the submodules repository, but nothing else.
Therefore, your developers have to update the submodules afterwards:
<pre>
$ cd /path/to/local/repository
$ git submodule update
Submodule path 'mod/mylittleextension': checked out '5e0e66aa787f4ebe9f61a940969a6e94abf01a1e'
</pre>

=== Figure: Updates without customization ===

[[File:superprojects3.png|center|A workflow for an upgrade from the source repository]]

=== Known issues ===

Since '''git submodule upgrade''' checks out the reference, given by your main repository's commit, your submodule's repository will turn into detached HEAD state again.
The command is equivalent to '''git checkout''' of the reference inside each submodule's path.
That means, that conflicts will abort the checkout, like entering the directory and trying to do the checkout manually.
So the submodule's repository should be clean in order to prevent problems with the submodule update.

A special case of this problem happens, if you try to clone a superproject, whose submodule's reference is not known by the origin.
Your submodule's directory would be left blank (since there's nothing to check out) and you would have to resolve this issue manually (e.g. add your superprojects submodule repository as a remote).
A situation, you don't want to get, for sure.

== Upgrading customized plugins ==

=== Figure: Suggested workflow ===

In a normal workflow you don't touch external plugins and therefore there is no need to alter the submodules in any way.
But what, if you '''want to alter''' them.
In some cases the submodule is your own plugin or the extension is somehow insufficient but the original contributor doesn't want to merge your pull requests.

As you know from the previous section '''git submodule update''' tries to checkout a specific reference.
Therefore, this reference must exist in the origin or you will get a problem while updating them.
If you're not the contributer of the extension it is necessary to fork these repositiories.
A suggested workflow would be like this:

[[Image:superprojects4.png|Suggested workflow with customized plugins]]

Your developers are responsible for updating your forked plugin.
Therefore they must merge the original changes into your forked code and resolve conflicts, if necessary.
Your superproject pulls from the fork only.
If you're setting up your first superproject, it doesn't have to be connected to the original repositiory at all, since you'll get all updates from your fork.
So instead of adding the extension's original remote, you may add the fork only (after adding you can use '''git remote rename origin fork''' before your initial commit of the plugin).

=== Setting up a new submodule ===

In case you haven't installed the extension yet, you can add and rename the repository afterwards (to avoid misunderstanding).
On your superproject's machine do the following commands:
<pre>
$ cd /path/to/moodle
$ git submodule add <source:fork> mod/mylittleextension
Cloning into 'mod/mylittleextension'...
done.
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git branch -avv
* master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
mod/mylittleextension $ cd ../..
$ git commit -a -m "New extension 'MLE' installed"
[master adecc96] New extension 'MLE' installed
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 mod/mylittleextension
</pre>

Your developers will get your new submodule using '''git clone''' or '''git pull''' and can rename the remote likewise.
In addition, they have to fetch updates from the original remote and have to add a remote repository manually.
On your developers machine type in (output messages from Git are omitted):
<pre>
cd /path/to/local/repository
$ git pull
$ git submodule update --init
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Now your developers can add branches, edit code, do commits and contribute to your fork.
Details about that topic will follow after the next section.

=== Changes for an existing submodule ===

If the submodule is already installed and your superproject cloned, you must edit the '''.gitmodules''' file.
Since the '''path''' and '''url''' of your submodule are saved ('''reference''' is saved in your commit), you may edit the file manually.
But you can also use a Git command for that purpose.
On the superproject's machine type in (output messages from Git are omitted):
<pre>
cd /path/to/moodle/
$ git config --file=.gitmodules submodule."mod/mylittleextension".url "<source:fork>"
</pre>
'''or''' edit the file '''.gitmodules''' respectively.

Afterwards type in:
<pre>
$ git submodule sync
Synchronizing submodule url for 'mod/mylittleextension'
$ git commit -a -m "Changed source url of 'MLE'"
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
</pre>

The '''sync''' command will alter the remotes of your submodule's repository to match the settings of your '''.gitmodules'''.
It is to mention, that changes in the remotes (like '''rename''' or '''add''') of a submodule wont be noticed by the Moodle repository.
In fact, unless you edit a file or change the submodule's HEAD reference, you can do whatever you want with your branches.

At last your developer has to update his project as well.
On the developers machine type in (output messages are omitted again):
<pre>
$ cd /path/to/local/repository
$ git pull
$ git submodule update
$ git submodule sync
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Again, the '''sync''' command will change the remote settings of the submodule to match them with the '''.gitmodules'''.

=== Contributing to the superproject ===

This section will explain how to contribute patches for your superproject.
Assume your developer has changed an extension and commit his changes in the submodule's repository.
<pre>
$ cd /path/to/local/repository
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -avv
* dev 9701a0c [fork/dev: ahead 1] Third commit
master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
</pre>

Push the changes into the fork.
<pre>
mod/mylittleextension $ git push fork dev:master
</pre>

You may organize your forked repository in another way, e.g. let the developers push into a seperate branch and merge them only, when it is satisfying.
For now, we assume the developers can push into your '''master''' (or, in general, the branch, which is tracked by your superproject).
Your developer's Moodle repository will notice the changes from their commit:
<pre>
$ cd /path/to/moodle
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

'''Do not commit the changes.'''
We presume, that every change in your main repository is done through a commit from your superproject.
The status is just saying, that your submodule is on another state than your superproject.
But this is not a problem at all. We will fix this later.
'''Note: When your developers need to commit changes, which are not related to the submodule, they have to avoid to stage the submodule until the superproject has been updated.'''

Update the superproject with the following commands on the superproject's machine:
<pre>
(On the superproject's machine)
$ cd /path/to/moodle
$ git submodule foreach git pull
$ git commit -a -m "Plugin updates"
</pre>

Your developers can now fetch the updated superproject.
<pre>
(On the developer's machine)
$ cd /path/to/local/repository
$ git pull
$ git status
# On branch master
nothing to commit (working directory clean)
</pre>

As you can see, your developer don't need to commit the changed status of the submodule.
Your developer's repository will be clean again after a fetch.

=== Figure: Workflow of an update of your submodules ===

[[Image:superprojects5.png|The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.]]

== See also ==

; Moodle Docs
* [[Git for Administrators]]
* [[Git for developers guide]]
* [[Git FAQ]]
* [[Windows installation using Git]]
* [[Git for Mac]]
* [[Moodle_Production_Server_with_GIT|Case study Git + Moodle from Technical University Berlin]]

; External resources
* [https://git-scm.com/book/en/v2/Git-Tools-Submodules Submodules at the Git book]

File:superprojects3.png

2016-03-15T15:23:18Z

Jan-eberhardt: A workflow for an upgrade from the source repository

A workflow for an upgrade from the source repository

Moodle development environment with Git submodules

2016-03-15T15:22:42Z

Jan-eberhardt:

This page describes how to set up a development environment for a Moodle instance with external plugins.
Over time the number of plugins may increase in such a way, that they can't be managed manually by developers anymore.
Without any precautions this can lead to various problems, for instance:
* Missing plugins in cloned repositories
* Diverged versions in existing repositories (can lead to fatal errors when trying to update the site)
* Messed up Git excludes

You'll learn in this guide how to use Git submodules in order to achieve the following setup:

[[Image:superprojects1.png|center|Scheme of a development environment]]

'''Note: The extension '''mod_mylittleextension''' is a fake plugin, which was created for testing purposes only.'''

By using Git submodules all extensions of a repository can be simply synchronized with other projects.
All repositories are based on the so called '''superproject''' repository and the developers' projects will be sometimes referred as 'local' repository.
Anyone who is interested in maintaining the superproject and core changes by Git can visit [[Moodle Production Server with Git]].

'''Note: This guide won't cover basic Git commands and how to install/maintain Git submodules.
It is assumed, that you have read [[Git for Administrators]] where these topics are covered.
The superproject will be seen as a Moodle repository where several plugins have been already installed via Git submodules.'''

== Cloning a superproject ==

First a short explanation of events which are triggered, when a superproject is cloned.
There are some things in Git submodules, which aren't obvious and can be quite surprising.
'''Note: Ensure that you've understood the how-to at [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] before continuing.'''

Assume the plugin '''mod_mylittleextension''' (remember: MLE is a fake plugin) was installed in the superproject and these changes have been commited:
<pre>
$ cd /path/to/your/superproject
$ git submodule add /local/repositories/mle/ mod/mylittleextension
$ git commit -a -m "New module MLE installed"
</pre>

Now a developer has to type in the following commands, in order to clone the superproject (maybe from a remote device).
<pre>
$ cd /path/to/local/directory
$ git clone <source> moodle
$ cd moodle
$ git submodule update --init --recursive
Submodule 'mod/mylittleextension' (/local/repositories/mle) registered for path 'mod/mylittleextension'
Cloning into 'mod/mylittleextension'...
done.
Submodule path 'mod/mylittleextension': checked out '89d9eae3d5142474d8452128e8df5720d89012cd'
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
</pre>

The new main repositories will automatically know about the new extension, but it won't be initialized, which means that the submodule's directory will be empty in the beginning.
For initializing the submodule, the developer needs to execute '''git submodule update --init --recursive'''.
The '''--recursive''' option means, that Git will walk through nested submodules as well (not obligatory in this example).

After the intialization the submodule's repository will be in detached HEAD state.
This is due the fact, that Git receives following data from the superproject:
* source url
* local path
* HEAD reference (hash)
See this figure, for a better understanding:

[[Image:superprojects2.png| Dotted line: information obtained from the superproject, but already saved locally (by pull)]]

This means, '''git submodule update --init''' will first do a clone of '''source url''' into the '''path''' and then check out the '''reference''', leaving the submodule in detached HEAD state.
You and your developers need to keep in mind, that any change of the HEAD's reference of your submodule will be noticed by the main repository.
This means also, that everyone can switch branches unnoticedly as long as the HEAD's hash reference doesn't change.
E.g. a checkout of the branch '''master''' won't be noticed by the main repository, but a checkout of '''dev''' will:
<pre>
mod/mylittleextension $ git checkout master
mod/mylittleextension $ git branch -av
* master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
$ cd ../..
$ git status
# On branch master
nothing to commit (working directory clean)
$ cd mod/mylittleextension
mod/mylittleextension $ git checkout dev
Branch dev set up to track remote branch dev from origin.
Switched to a new branch 'dev'
mod/mylittleextension $ cd ../..
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

There is a shorter command to initialize the submodules, but the outcome will be the same as above.
Instead of changing the moodle directory and use '''git submodule update --init''' after the clone, you can use '''git clone''' with the '''--recursive''' option as well.
<pre>
$ cd /path/to/new/location
$ git clone --recursive <source> moodle
</pre>

In other words, you may use '''git submodule update --init''' in the case you forget the '''--recursive''' option in your '''git clone''' command.

== Upgrading your submodules via the superproject ==

We now assume, that the external plugin is updated by it's own repository only and no customization is needed.
See [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] for further information on the topic of maintaining submodules of your superproject.

=== Pull and upgrade ===

In case a submodule was updated inside the superproject, your developers have to be careful.
'''git pull''' won't upgrade the submodules at once and an additional command is needed.
This is what '''git pull''' would look like:
<pre>
$ cd /path/to/local/repository
$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /local/repositories/super/
b547f6c..7f0c348 master -> origin/master
Fetching submodule mod/mylittleextension
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From /local/repositories/mle
3a2d487..5e0e66a dev -> origin/dev
89d9eae..5e0e66a master -> origin/master
Updating b547f6c..7f0c348
Fast-forward
mod/mylittleextension | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
dev 3a2d487 [behind 1] First commit
master 89d9eae [behind 2] Initial commit
remotes/origin/master 5e0e66a Second commit
remotes/origin/dev 5e0e66a Second commit
</pre>

As you see, '''git pull'' triggers '''git fetch''' inside the submodules repository, but nothing else.
Therefore, your developers have to update the submodules afterwards:
<pre>
$ cd /path/to/local/repository
$ git submodule update
Submodule path 'mod/mylittleextension': checked out '5e0e66aa787f4ebe9f61a940969a6e94abf01a1e'
</pre>

=== Figure: Updates without customization ===

[[Image:superprojects3.png|A workflow for an upgrade from the source repository]]

=== Known issues ===

Since '''git submodule upgrade''' checks out the reference, given by your main repository's commit, your submodule's repository will turn into detached HEAD state again.
The command is equivalent to '''git checkout''' of the reference inside each submodule's path.
That means, that conflicts will abort the checkout, like entering the directory and trying to do the checkout manually.
So the submodule's repository should be clean in order to prevent problems with the submodule update.

A special case of this problem happens, if you try to clone a superproject, whose submodule's reference is not known by the origin.
Your submodule's directory would be left blank (since there's nothing to check out) and you would have to resolve this issue manually (e.g. add your superprojects submodule repository as a remote).
A situation, you don't want to get, for sure.

== Upgrading customized plugins ==

=== Figure: Suggested workflow ===

In a normal workflow you don't touch external plugins and therefore there is no need to alter the submodules in any way.
But what, if you '''want to alter''' them.
In some cases the submodule is your own plugin or the extension is somehow insufficient but the original contributor doesn't want to merge your pull requests.

As you know from the previous section '''git submodule update''' tries to checkout a specific reference.
Therefore, this reference must exist in the origin or you will get a problem while updating them.
If you're not the contributer of the extension it is necessary to fork these repositiories.
A suggested workflow would be like this:

[[Image:superprojects4.png|Suggested workflow with customized plugins]]

Your developers are responsible for updating your forked plugin.
Therefore they must merge the original changes into your forked code and resolve conflicts, if necessary.
Your superproject pulls from the fork only.
If you're setting up your first superproject, it doesn't have to be connected to the original repositiory at all, since you'll get all updates from your fork.
So instead of adding the extension's original remote, you may add the fork only (after adding you can use '''git remote rename origin fork''' before your initial commit of the plugin).

=== Setting up a new submodule ===

In case you haven't installed the extension yet, you can add and rename the repository afterwards (to avoid misunderstanding).
On your superproject's machine do the following commands:
<pre>
$ cd /path/to/moodle
$ git submodule add <source:fork> mod/mylittleextension
Cloning into 'mod/mylittleextension'...
done.
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git branch -avv
* master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
mod/mylittleextension $ cd ../..
$ git commit -a -m "New extension 'MLE' installed"
[master adecc96] New extension 'MLE' installed
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 mod/mylittleextension
</pre>

Your developers will get your new submodule using '''git clone''' or '''git pull''' and can rename the remote likewise.
In addition, they have to fetch updates from the original remote and have to add a remote repository manually.
On your developers machine type in (output messages from Git are omitted):
<pre>
cd /path/to/local/repository
$ git pull
$ git submodule update --init
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Now your developers can add branches, edit code, do commits and contribute to your fork.
Details about that topic will follow after the next section.

=== Changes for an existing submodule ===

If the submodule is already installed and your superproject cloned, you must edit the '''.gitmodules''' file.
Since the '''path''' and '''url''' of your submodule are saved ('''reference''' is saved in your commit), you may edit the file manually.
But you can also use a Git command for that purpose.
On the superproject's machine type in (output messages from Git are omitted):
<pre>
cd /path/to/moodle/
$ git config --file=.gitmodules submodule."mod/mylittleextension".url "<source:fork>"
</pre>
'''or''' edit the file '''.gitmodules''' respectively.

Afterwards type in:
<pre>
$ git submodule sync
Synchronizing submodule url for 'mod/mylittleextension'
$ git commit -a -m "Changed source url of 'MLE'"
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
</pre>

The '''sync''' command will alter the remotes of your submodule's repository to match the settings of your '''.gitmodules'''.
It is to mention, that changes in the remotes (like '''rename''' or '''add''') of a submodule wont be noticed by the Moodle repository.
In fact, unless you edit a file or change the submodule's HEAD reference, you can do whatever you want with your branches.

At last your developer has to update his project as well.
On the developers machine type in (output messages are omitted again):
<pre>
$ cd /path/to/local/repository
$ git pull
$ git submodule update
$ git submodule sync
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Again, the '''sync''' command will change the remote settings of the submodule to match them with the '''.gitmodules'''.

=== Contributing to the superproject ===

This section will explain how to contribute patches for your superproject.
Assume your developer has changed an extension and commit his changes in the submodule's repository.
<pre>
$ cd /path/to/local/repository
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -avv
* dev 9701a0c [fork/dev: ahead 1] Third commit
master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
</pre>

Push the changes into the fork.
<pre>
mod/mylittleextension $ git push fork dev:master
</pre>

You may organize your forked repository in another way, e.g. let the developers push into a seperate branch and merge them only, when it is satisfying.
For now, we assume the developers can push into your '''master''' (or, in general, the branch, which is tracked by your superproject).
Your developer's Moodle repository will notice the changes from their commit:
<pre>
$ cd /path/to/moodle
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

'''Do not commit the changes.'''
We presume, that every change in your main repository is done through a commit from your superproject.
The status is just saying, that your submodule is on another state than your superproject.
But this is not a problem at all. We will fix this later.
'''Note: When your developers need to commit changes, which are not related to the submodule, they have to avoid to stage the submodule until the superproject has been updated.'''

Update the superproject with the following commands on the superproject's machine:
<pre>
(On the superproject's machine)
$ cd /path/to/moodle
$ git submodule foreach git pull
$ git commit -a -m "Plugin updates"
</pre>

Your developers can now fetch the updated superproject.
<pre>
(On the developer's machine)
$ cd /path/to/local/repository
$ git pull
$ git status
# On branch master
nothing to commit (working directory clean)
</pre>

As you can see, your developer don't need to commit the changed status of the submodule.
Your developer's repository will be clean again after a fetch.

=== Figure: Workflow of an update of your submodules ===

[[Image:superprojects5.png|The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.]]

== See also ==

; Moodle Docs
* [[Git for Administrators]]
* [[Git for developers guide]]
* [[Git FAQ]]
* [[Windows installation using Git]]
* [[Git for Mac]]
* [[Moodle_Production_Server_with_GIT|Case study Git + Moodle from Technical University Berlin]]

; External resources
* [https://git-scm.com/book/en/v2/Git-Tools-Submodules Submodules at the Git book]

File:superprojects2.png

2016-03-15T15:21:27Z

Jan-eberhardt: Dotted line: information obtained from the superproject, but already saved locally (by pull)

Dotted line: information obtained from the superproject, but already saved locally (by pull)

Moodle development environment with Git submodules

2016-03-15T15:18:53Z

Jan-eberhardt:

This page describes how to set up a development environment for a Moodle instance with external plugins.
Over time the number of plugins may increase in such a way, that they can't be managed manually by developers anymore.
Without any precautions this can lead to various problems, for instance:
* Missing plugins in cloned repositories
* Diverged versions in existing repositories (can lead to fatal errors when trying to update the site)
* Messed up Git excludes

You'll learn in this guide how to use Git submodules in order to achieve the following setup:

[[Image:superprojects1.png|center|Scheme of a development environment]]

'''Note: The extension '''mod_mylittleextension''' is a fake plugin, which was created for testing purposes only.'''

By using Git submodules all extensions of a repository can be simply synchronized with other projects.
All repositories are based on the so called '''superproject''' repository and the developers' projects will be sometimes referred as 'local' repository.
Anyone who is interested in maintaining the superproject and core changes by Git can visit [[Moodle Production Server with Git]].

'''Note: This guide won't cover basic Git commands and how to install/maintain Git submodules.
It is assumed, that you have read [[Git for Administrators]] where these topics are covered.
The superproject will be seen as a Moodle repository where several plugins have been already installed via Git submodules.'''

== Cloning a superproject ==

First a short explanation of events which are triggered, when a superproject is cloned.
There are some things in Git submodules, which aren't obvious and can be quite surprising.
'''Note: Ensure that you've understood the how-to at [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] before continuing.'''

Assume the plugin '''mod_mylittleextension''' (remember: MLE is a fake plugin) was installed in the superproject and these changes have been commited:
<pre>
$ cd /path/to/your/superproject
$ git submodule add /local/repositories/mle/ mod/mylittleextension
$ git commit -a -m "New module MLE installed"
</pre>

Now a developer has to type in the following commands, in order to clone the superproject (maybe from a remote device).
<pre>
$ cd /path/to/local/directory
$ git clone <source> moodle
$ cd moodle
$ git submodule update --init --recursive
Submodule 'mod/mylittleextension' (/local/repositories/mle) registered for path 'mod/mylittleextension'
Cloning into 'mod/mylittleextension'...
done.
Submodule path 'mod/mylittleextension': checked out '89d9eae3d5142474d8452128e8df5720d89012cd'
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
</pre>

The new main repositories will automatically know about the new extension, but it won't be initialized, which means that the submodule's directory will be empty in the beginning.
For initializing the submodule, the developer needs to execute '''git submodule update --init --recursive'''.
The '''--recursive''' option means, that Git will walk through nested submodules as well (not obligatory in this example).

After the intialization the submodule's repository will be in detached HEAD state.
This is due the fact, that Git receives following data from the superproject:
* source url
* local path
* HEAD reference (hash)
See this figure, for a better understanding:

[[File:superprojects2.png| Dotted line: information obtained from the superproject, but already saved locally (by pull)]]

This means, '''git submodule update --init''' will first do a clone of '''source url''' into the '''path''' and then check out the '''reference''', leaving the submodule in detached HEAD state.
You and your developers need to keep in mind, that any change of the HEAD's reference of your submodule will be noticed by the main repository.
This means also, that everyone can switch branches unnoticedly as long as the HEAD's hash reference doesn't change.
E.g. a checkout of the branch '''master''' won't be noticed by the main repository, but a checkout of '''dev''' will:
<pre>
mod/mylittleextension $ git checkout master
mod/mylittleextension $ git branch -av
* master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
$ cd ../..
$ git status
# On branch master
nothing to commit (working directory clean)
$ cd mod/mylittleextension
mod/mylittleextension $ git checkout dev
Branch dev set up to track remote branch dev from origin.
Switched to a new branch 'dev'
mod/mylittleextension $ cd ../..
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

There is a shorter command to initialize the submodules, but the outcome will be the same as above.
Instead of changing the moodle directory and use '''git submodule update --init''' after the clone, you can use '''git clone''' with the '''--recursive''' option as well.
<pre>
$ cd /path/to/new/location
$ git clone --recursive <source> moodle
</pre>

In other words, you may use '''git submodule update --init''' in the case you forget the '''--recursive''' option in your '''git clone''' command.

== Upgrading your submodules via the superproject ==

We now assume, that the external plugin is updated by it's own repository only and no customization is needed.
See [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] for further information on the topic of maintaining submodules of your superproject.

=== Pull and upgrade ===

In case a submodule was updated inside the superproject, your developers have to be careful.
'''git pull''' won't upgrade the submodules at once and an additional command is needed.
This is what '''git pull''' would look like:
<pre>
$ cd /path/to/local/repository
$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /local/repositories/super/
b547f6c..7f0c348 master -> origin/master
Fetching submodule mod/mylittleextension
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From /local/repositories/mle
3a2d487..5e0e66a dev -> origin/dev
89d9eae..5e0e66a master -> origin/master
Updating b547f6c..7f0c348
Fast-forward
mod/mylittleextension | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
dev 3a2d487 [behind 1] First commit
master 89d9eae [behind 2] Initial commit
remotes/origin/master 5e0e66a Second commit
remotes/origin/dev 5e0e66a Second commit
</pre>

As you see, '''git pull'' triggers '''git fetch''' inside the submodules repository, but nothing else.
Therefore, your developers have to update the submodules afterwards:
<pre>
$ cd /path/to/local/repository
$ git submodule update
Submodule path 'mod/mylittleextension': checked out '5e0e66aa787f4ebe9f61a940969a6e94abf01a1e'
</pre>

=== Figure: Updates without customization ===

[[File:superprojects3.png|A workflow for an upgrade from the source repository]]

=== Known issues ===

Since '''git submodule upgrade''' checks out the reference, given by your main repository's commit, your submodule's repository will turn into detached HEAD state again.
The command is equivalent to '''git checkout''' of the reference inside each submodule's path.
That means, that conflicts will abort the checkout, like entering the directory and trying to do the checkout manually.
So the submodule's repository should be clean in order to prevent problems with the submodule update.

A special case of this problem happens, if you try to clone a superproject, whose submodule's reference is not known by the origin.
Your submodule's directory would be left blank (since there's nothing to check out) and you would have to resolve this issue manually (e.g. add your superprojects submodule repository as a remote).
A situation, you don't want to get, for sure.

== Upgrading customized plugins ==

=== Figure: Suggested workflow ===

In a normal workflow you don't touch external plugins and therefore there is no need to alter the submodules in any way.
But what, if you '''want to alter''' them.
In some cases the submodule is your own plugin or the extension is somehow insufficient but the original contributor doesn't want to merge your pull requests.

As you know from the previous section '''git submodule update''' tries to checkout a specific reference.
Therefore, this reference must exist in the origin or you will get a problem while updating them.
If you're not the contributer of the extension it is necessary to fork these repositiories.
A suggested workflow would be like this:

[[File:superprojects4.png|Suggested workflow with customized plugins]]

Your developers are responsible for updating your forked plugin.
Therefore they must merge the original changes into your forked code and resolve conflicts, if necessary.
Your superproject pulls from the fork only.
If you're setting up your first superproject, it doesn't have to be connected to the original repositiory at all, since you'll get all updates from your fork.
So instead of adding the extension's original remote, you may add the fork only (after adding you can use '''git remote rename origin fork''' before your initial commit of the plugin).

=== Setting up a new submodule ===

In case you haven't installed the extension yet, you can add and rename the repository afterwards (to avoid misunderstanding).
On your superproject's machine do the following commands:
<pre>
$ cd /path/to/moodle
$ git submodule add <source:fork> mod/mylittleextension
Cloning into 'mod/mylittleextension'...
done.
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git branch -avv
* master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
mod/mylittleextension $ cd ../..
$ git commit -a -m "New extension 'MLE' installed"
[master adecc96] New extension 'MLE' installed
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 mod/mylittleextension
</pre>

Your developers will get your new submodule using '''git clone''' or '''git pull''' and can rename the remote likewise.
In addition, they have to fetch updates from the original remote and have to add a remote repository manually.
On your developers machine type in (output messages from Git are omitted):
<pre>
cd /path/to/local/repository
$ git pull
$ git submodule update --init
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Now your developers can add branches, edit code, do commits and contribute to your fork.
Details about that topic will follow after the next section.

=== Changes for an existing submodule ===

If the submodule is already installed and your superproject cloned, you must edit the '''.gitmodules''' file.
Since the '''path''' and '''url''' of your submodule are saved ('''reference''' is saved in your commit), you may edit the file manually.
But you can also use a Git command for that purpose.
On the superproject's machine type in (output messages from Git are omitted):
<pre>
cd /path/to/moodle/
$ git config --file=.gitmodules submodule."mod/mylittleextension".url "<source:fork>"
</pre>
'''or''' edit the file '''.gitmodules''' respectively.

Afterwards type in:
<pre>
$ git submodule sync
Synchronizing submodule url for 'mod/mylittleextension'
$ git commit -a -m "Changed source url of 'MLE'"
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
</pre>

The '''sync''' command will alter the remotes of your submodule's repository to match the settings of your '''.gitmodules'''.
It is to mention, that changes in the remotes (like '''rename''' or '''add''') of a submodule wont be noticed by the Moodle repository.
In fact, unless you edit a file or change the submodule's HEAD reference, you can do whatever you want with your branches.

At last your developer has to update his project as well.
On the developers machine type in (output messages are omitted again):
<pre>
$ cd /path/to/local/repository
$ git pull
$ git submodule update
$ git submodule sync
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Again, the '''sync''' command will change the remote settings of the submodule to match them with the '''.gitmodules'''.

=== Contributing to the superproject ===

This section will explain how to contribute patches for your superproject.
Assume your developer has changed an extension and commit his changes in the submodule's repository.
<pre>
$ cd /path/to/local/repository
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -avv
* dev 9701a0c [fork/dev: ahead 1] Third commit
master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
</pre>

Push the changes into the fork.
<pre>
mod/mylittleextension $ git push fork dev:master
</pre>

You may organize your forked repository in another way, e.g. let the developers push into a seperate branch and merge them only, when it is satisfying.
For now, we assume the developers can push into your '''master''' (or, in general, the branch, which is tracked by your superproject).
Your developer's Moodle repository will notice the changes from their commit:
<pre>
$ cd /path/to/moodle
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

'''Do not commit the changes.'''
We presume, that every change in your main repository is done through a commit from your superproject.
The status is just saying, that your submodule is on another state than your superproject.
But this is not a problem at all. We will fix this later.
'''Note: When your developers need to commit changes, which are not related to the submodule, they have to avoid to stage the submodule until the superproject has been updated.'''

Update the superproject with the following commands on the superproject's machine:
<pre>
(On the superproject's machine)
$ cd /path/to/moodle
$ git submodule foreach git pull
$ git commit -a -m "Plugin updates"
</pre>

Your developers can now fetch the updated superproject.
<pre>
(On the developer's machine)
$ cd /path/to/local/repository
$ git pull
$ git status
# On branch master
nothing to commit (working directory clean)
</pre>

As you can see, your developer don't need to commit the changed status of the submodule.
Your developer's repository will be clean again after a fetch.

=== Figure: Workflow of an update of your submodules ===

[[File:superprojects5.png|The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.]]

== See also ==

; Moodle Docs
* [[Git for Administrators]]
* [[Git for developers guide]]
* [[Git FAQ]]
* [[Windows installation using Git]]
* [[Git for Mac]]
* [[Moodle_Production_Server_with_GIT|Case study Git + Moodle from Technical University Berlin]]

; External resources
* [https://git-scm.com/book/en/v2/Git-Tools-Submodules Submodules at the Git book]

File:superprojects1.png

2016-03-15T15:17:13Z

Jan-eberhardt: Scheme of a development environment

Scheme of a development environment

Moodle development environment with Git submodules

2016-03-15T15:15:00Z

Jan-eberhardt: Created page with "This page describes how to set up a development environment for a Moodle instance with external plugins. Over time the number of plugins may increase in such a way, that they..."

This page describes how to set up a development environment for a Moodle instance with external plugins.
Over time the number of plugins may increase in such a way, that they can't be managed manually by developers anymore.
Without any precautions this can lead to various problems, for instance:
* Missing plugins in cloned repositories
* Diverged versions in existing repositories (can lead to fatal errors when trying to update the site)
* Messed up Git excludes

You'll learn in this guide how to use Git submodules in order to achieve the following setup:

[[File:superprojects1.png|center|Scheme of a development environment]]

'''Note: The extension '''mod_mylittleextension''' is a fake plugin, which was created for testing purposes only.'''

By using Git submodules all extensions of a repository can be simply synchronized with other projects.
All repositories are based on the so called '''superproject''' repository and the developers' projects will be sometimes referred as 'local' repository.
Anyone who is interested in maintaining the superproject and core changes by Git can visit [[Moodle Production Server with Git]].

'''Note: This guide won't cover basic Git commands and how to install/maintain Git submodules.
It is assumed, that you have read [[Git for Administrators]] where these topics are covered.
The superproject will be seen as a Moodle repository where several plugins have been already installed via Git submodules.'''

== Cloning a superproject ==

First a short explanation of events which are triggered, when a superproject is cloned.
There are some things in Git submodules, which aren't obvious and can be quite surprising.
'''Note: Ensure that you've understood the how-to at [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] before continuing.'''

Assume the plugin '''mod_mylittleextension''' (remember: MLE is a fake plugin) was installed in the superproject and these changes have been commited:
<pre>
$ cd /path/to/your/superproject
$ git submodule add /local/repositories/mle/ mod/mylittleextension
$ git commit -a -m "New module MLE installed"
</pre>

Now a developer has to type in the following commands, in order to clone the superproject (maybe from a remote device).
<pre>
$ cd /path/to/local/directory
$ git clone <source> moodle
$ cd moodle
$ git submodule update --init --recursive
Submodule 'mod/mylittleextension' (/local/repositories/mle) registered for path 'mod/mylittleextension'
Cloning into 'mod/mylittleextension'...
done.
Submodule path 'mod/mylittleextension': checked out '89d9eae3d5142474d8452128e8df5720d89012cd'
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
</pre>

The new main repositories will automatically know about the new extension, but it won't be initialized, which means that the submodule's directory will be empty in the beginning.
For initializing the submodule, the developer needs to execute '''git submodule update --init --recursive'''.
The '''--recursive''' option means, that Git will walk through nested submodules as well (not obligatory in this example).

After the intialization the submodule's repository will be in detached HEAD state.
This is due the fact, that Git receives following data from the superproject:
* source url
* local path
* HEAD reference (hash)
See this figure, for a better understanding:

[[File:superprojects2.png| Dotted line: information obtained from the superproject, but already saved locally (by pull)]]

This means, '''git submodule update --init''' will first do a clone of '''source url''' into the '''path''' and then check out the '''reference''', leaving the submodule in detached HEAD state.
You and your developers need to keep in mind, that any change of the HEAD's reference of your submodule will be noticed by the main repository.
This means also, that everyone can switch branches unnoticedly as long as the HEAD's hash reference doesn't change.
E.g. a checkout of the branch '''master''' won't be noticed by the main repository, but a checkout of '''dev''' will:
<pre>
mod/mylittleextension $ git checkout master
mod/mylittleextension $ git branch -av
* master 89d9eae Initial commit
remotes/origin/HEAD -> origin/master
remotes/origin/dev 3a2d487 First commit
remotes/origin/master 89d9eae Initial commit
$ cd ../..
$ git status
# On branch master
nothing to commit (working directory clean)
$ cd mod/mylittleextension
mod/mylittleextension $ git checkout dev
Branch dev set up to track remote branch dev from origin.
Switched to a new branch 'dev'
mod/mylittleextension $ cd ../..
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

There is a shorter command to initialize the submodules, but the outcome will be the same as above.
Instead of changing the moodle directory and use '''git submodule update --init''' after the clone, you can use '''git clone''' with the '''--recursive''' option as well.
<pre>
$ cd /path/to/new/location
$ git clone --recursive <source> moodle
</pre>

In other words, you may use '''git submodule update --init''' in the case you forget the '''--recursive''' option in your '''git clone''' command.

== Upgrading your submodules via the superproject ==

We now assume, that the external plugin is updated by it's own repository only and no customization is needed.
See [[Git for Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules|Installing and maintaining contributed extensions using Git submodules]] for further information on the topic of maintaining submodules of your superproject.

=== Pull and upgrade ===

In case a submodule was updated inside the superproject, your developers have to be careful.
'''git pull''' won't upgrade the submodules at once and an additional command is needed.
This is what '''git pull''' would look like:
<pre>
$ cd /path/to/local/repository
$ git pull
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /local/repositories/super/
b547f6c..7f0c348 master -> origin/master
Fetching submodule mod/mylittleextension
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From /local/repositories/mle
3a2d487..5e0e66a dev -> origin/dev
89d9eae..5e0e66a master -> origin/master
Updating b547f6c..7f0c348
Fast-forward
mod/mylittleextension | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -av
* (no branch) 89d9eae Initial commit
dev 3a2d487 [behind 1] First commit
master 89d9eae [behind 2] Initial commit
remotes/origin/master 5e0e66a Second commit
remotes/origin/dev 5e0e66a Second commit
</pre>

As you see, '''git pull'' triggers '''git fetch''' inside the submodules repository, but nothing else.
Therefore, your developers have to update the submodules afterwards:
<pre>
$ cd /path/to/local/repository
$ git submodule update
Submodule path 'mod/mylittleextension': checked out '5e0e66aa787f4ebe9f61a940969a6e94abf01a1e'
</pre>

=== Figure: Updates without customization ===

[[File:superprojects3.png|A workflow for an upgrade from the source repository]]

=== Known issues ===

Since '''git submodule upgrade''' checks out the reference, given by your main repository's commit, your submodule's repository will turn into detached HEAD state again.
The command is equivalent to '''git checkout''' of the reference inside each submodule's path.
That means, that conflicts will abort the checkout, like entering the directory and trying to do the checkout manually.
So the submodule's repository should be clean in order to prevent problems with the submodule update.

A special case of this problem happens, if you try to clone a superproject, whose submodule's reference is not known by the origin.
Your submodule's directory would be left blank (since there's nothing to check out) and you would have to resolve this issue manually (e.g. add your superprojects submodule repository as a remote).
A situation, you don't want to get, for sure.

== Upgrading customized plugins ==

=== Figure: Suggested workflow ===

In a normal workflow you don't touch external plugins and therefore there is no need to alter the submodules in any way.
But what, if you '''want to alter''' them.
In some cases the submodule is your own plugin or the extension is somehow insufficient but the original contributor doesn't want to merge your pull requests.

As you know from the previous section '''git submodule update''' tries to checkout a specific reference.
Therefore, this reference must exist in the origin or you will get a problem while updating them.
If you're not the contributer of the extension it is necessary to fork these repositiories.
A suggested workflow would be like this:

[[File:superprojects4.png|Suggested workflow with customized plugins]]

Your developers are responsible for updating your forked plugin.
Therefore they must merge the original changes into your forked code and resolve conflicts, if necessary.
Your superproject pulls from the fork only.
If you're setting up your first superproject, it doesn't have to be connected to the original repositiory at all, since you'll get all updates from your fork.
So instead of adding the extension's original remote, you may add the fork only (after adding you can use '''git remote rename origin fork''' before your initial commit of the plugin).

=== Setting up a new submodule ===

In case you haven't installed the extension yet, you can add and rename the repository afterwards (to avoid misunderstanding).
On your superproject's machine do the following commands:
<pre>
$ cd /path/to/moodle
$ git submodule add <source:fork> mod/mylittleextension
Cloning into 'mod/mylittleextension'...
done.
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git branch -avv
* master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
mod/mylittleextension $ cd ../..
$ git commit -a -m "New extension 'MLE' installed"
[master adecc96] New extension 'MLE' installed
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 mod/mylittleextension
</pre>

Your developers will get your new submodule using '''git clone''' or '''git pull''' and can rename the remote likewise.
In addition, they have to fetch updates from the original remote and have to add a remote repository manually.
On your developers machine type in (output messages from Git are omitted):
<pre>
cd /path/to/local/repository
$ git pull
$ git submodule update --init
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Now your developers can add branches, edit code, do commits and contribute to your fork.
Details about that topic will follow after the next section.

=== Changes for an existing submodule ===

If the submodule is already installed and your superproject cloned, you must edit the '''.gitmodules''' file.
Since the '''path''' and '''url''' of your submodule are saved ('''reference''' is saved in your commit), you may edit the file manually.
But you can also use a Git command for that purpose.
On the superproject's machine type in (output messages from Git are omitted):
<pre>
cd /path/to/moodle/
$ git config --file=.gitmodules submodule."mod/mylittleextension".url "<source:fork>"
</pre>
'''or''' edit the file '''.gitmodules''' respectively.

Afterwards type in:
<pre>
$ git submodule sync
Synchronizing submodule url for 'mod/mylittleextension'
$ git commit -a -m "Changed source url of 'MLE'"
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
</pre>

The '''sync''' command will alter the remotes of your submodule's repository to match the settings of your '''.gitmodules'''.
It is to mention, that changes in the remotes (like '''rename''' or '''add''') of a submodule wont be noticed by the Moodle repository.
In fact, unless you edit a file or change the submodule's HEAD reference, you can do whatever you want with your branches.

At last your developer has to update his project as well.
On the developers machine type in (output messages are omitted again):
<pre>
$ cd /path/to/local/repository
$ git pull
$ git submodule update
$ git submodule sync
$ cd mod/mylittleextension
mod/mylittleextension $ git remote rename origin fork
mod/mylittleextension $ git remote add origin <source:origin>
mod/mylittleextension $ git fetch origin
</pre>

Again, the '''sync''' command will change the remote settings of the submodule to match them with the '''.gitmodules'''.

=== Contributing to the superproject ===

This section will explain how to contribute patches for your superproject.
Assume your developer has changed an extension and commit his changes in the submodule's repository.
<pre>
$ cd /path/to/local/repository
$ cd mod/mylittleextension
mod/mylittleextension $ git branch -avv
* dev 9701a0c [fork/dev: ahead 1] Third commit
master 5e0e66a [fork/master] Second commit
remotes/fork/HEAD -> fork/master
remotes/fork/dev 5e0e66a Second commit
remotes/fork/master 5e0e66a Second commit
</pre>

Push the changes into the fork.
<pre>
mod/mylittleextension $ git push fork dev:master
</pre>

You may organize your forked repository in another way, e.g. let the developers push into a seperate branch and merge them only, when it is satisfying.
For now, we assume the developers can push into your '''master''' (or, in general, the branch, which is tracked by your superproject).
Your developer's Moodle repository will notice the changes from their commit:
<pre>
$ cd /path/to/moodle
$ git status
# On branch master
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
no changes added to commit (use "git add" and/or "git commit -a")
</pre>

'''Do not commit the changes.'''
We presume, that every change in your main repository is done through a commit from your superproject.
The status is just saying, that your submodule is on another state than your superproject.
But this is not a problem at all. We will fix this later.
'''Note: When your developers need to commit changes, which are not related to the submodule, they have to avoid to stage the submodule until the superproject has been updated.'''

Update the superproject with the following commands on the superproject's machine:
<pre>
(On the superproject's machine)
$ cd /path/to/moodle
$ git submodule foreach git pull
$ git commit -a -m "Plugin updates"
</pre>

Your developers can now fetch the updated superproject.
<pre>
(On the developer's machine)
$ cd /path/to/local/repository
$ git pull
$ git status
# On branch master
nothing to commit (working directory clean)
</pre>

As you can see, your developer don't need to commit the changed status of the submodule.
Your developer's repository will be clean again after a fetch.

=== Figure: Workflow of an update of your submodules ===

[[File:superprojects5.png|The 5th command ('''git submodule update [--init]''') is only necessary, when other submodules are added or updated.]]

== See also ==

; Moodle Docs
* [[Git for Administrators]]
* [[Git for developers guide]]
* [[Git FAQ]]
* [[Windows installation using Git]]
* [[Git for Mac]]
* [[Moodle_Production_Server_with_GIT|Case study Git + Moodle from Technical University Berlin]]

; External resources
* [https://git-scm.com/book/en/v2/Git-Tools-Submodules Submodules at the Git book]

Git for Administrators

2016-03-15T15:11:56Z

Jan-eberhardt:

{{Installing Moodle}}
This page describes how to maintain a copy of Moodle on your production server which can easily be upgraded using Git. If you have customisations of Moodle core code, you are advised to follow the instructions in the [[:dev:Git for developers|Git for developers guide]].

To get the most of Git it is worth making the effort to understand its basic concepts - see the section below. It can be a bit of a steep learning curve, especially if you are used to CVS or Subversion.

== Getting hold of Git (Windows, OSX, Linux and others) ==

Support for Git was, up until recently, mostly confined to Linux but builds are now available for most popular operating systems:

* List of downloads from Git site - http://git-scm.com/download

Once you have downloaded and installed your OS relevant git installation, the git commands in this document should work with your operating system.

== Obtaining the code from Git ==

The command line version of Git is discussed here. Graphical clients are little more than wrappers around the command line version, so you should be able to deduce the correct parameters quite easily.

You can find the official Moodle git repository at git://git.moodle.org/moodle.git (with an official clone at git://github.com/moodle/moodle.git). To initialize your local checkout, use
<pre>
$ cd /path/to/your/webroot
$ git clone git://git.moodle.org/moodle.git (1)
$ cd moodle
$ git branch -a (2)
$ git branch --track MOODLE_30_STABLE origin/MOODLE_30_STABLE (3)
$ git checkout MOODLE_30_STABLE (4)
</pre>
* The command (1) initializes the new local repository as a clone of the 'upstream' (i.e. the remote server based) moodle.git repository. The upstream repository is called 'origin' by default. It creates a new directory named ''moodle'', where it downloads all the files. This operation can take a while as it is actually getting the entire history of all Moodle versions
* The command (2) lists all available branches.
* Use the command (3) to create a new local branch called MOODLE_30_STABLE and set it to track the remote branch MOODLE_30_STABLE from the upstream repository.
* The command (4) actually switches to the newly created local branch.

Note that Git has a huge number of options for each command and it's actually possible to do the above process with a single command (left as an exercise!!).

==Git from behind a firewall==

Git uses a read-only protocol that may be blocked by your firewall (port 9418). If this is a problem, you can use Github's http version <nowiki>https://github.com/moodle/moodle.git</nowiki>. It's a bit slower, so use the Git protocol if you can.

== Updating your installation ==

The Moodle development team performs integration and testing of fixed bugs every Monday and Tuesday. On Wednesday you can install all patches by updating your code. Check the [http://git.moodle.org/gw?p=moodle.git;a=summary shortlog] to see if the official repository has been already updated or not.

To update your code to the latest version (on the MOODLE_30_STABLE branch) '''all''' you have to do is:
<pre>
$ cd /path/to/your/moodle/
$ git pull
</pre>
If this is a production site you should still consider the [[Upgrade]] instructions (e.g. take backups).

== Installing a contributed extension from its Git repository ==

This is one way to handle adding plugins from other Git repositories into your Moodle repository. Another way is to use Git Submodules. However, at the time of writing, this is one of Git's rougher features and should be regarded as an advanced option.

For example, let us say we want to install the [[Certificate module]] from its Git repository into our Moodle 3.0.
<pre>
$ cd /path/to/your/moodle/
$ cd mod (1)
$ git clone https://github.com/markn86/moodle-mod_certificate.git certificate (2)
$ cd certificate
$ git checkout -b MOODLE_30_STABLE origin/MOODLE_30_STABLE (3)
$ git branch -d master (4)
</pre>
The command (1) changes the current directory into the ''mod'' folder of your local Moodle clone. The command (2) creates a new subdirectory ''certificate'' and makes a local clone of vanilla Certificate repository. The command (3) creates a new local branch that will track the remote branch with a Certificate version for Moodle 3.0. The command (4) deletes the ''master'' that was created automatically by git-clone in (2) as we do not want it in this production checkout.

Note: you should check first the compatibility of a module with your Moodle branch by asking directly to the Maintainer before cloning the repo or - if you want to guess it - by issuing the command below before running the command (3), in order to verify what is available among the branches:
<pre>
$ git branch -a
* master
remotes/origin/HEAD -> origin/master
remotes/origin/MOODLE_20_STABLE
remotes/origin/MOODLE_21_STABLE
remotes/origin/MOODLE_22_STABLE
remotes/origin/MOODLE_23_STABLE
remotes/origin/MOODLE_24_STABLE
remotes/origin/MOODLE_25_STABLE
remotes/origin/MOODLE_26_STABLE
remotes/origin/MOODLE_27_STABLE
remotes/origin/MOODLE_28_STABLE
remotes/origin/MOODLE_29_STABLE
remotes/origin/MOODLE_30_STABLE
remotes/origin/master
</pre>
This will avoid an error message when you issue the command (3) against a nonexistent branch, e.g.:
<pre>
§ git checkout -b MOODLE_30_STABLE origin/MOODLE_30_STABLE
fatal: git checkout: updating paths is incompatible with switching branches.
Did you intend to checkout 'origin/MOODLE_30_STABLE' which can not be resolved as commit?
</pre>

Now it is wise to add the new directory mod/certificate/ to the list of ignored files of the main Moodle clone, otherwise a status of the main clone will keep reminding you that the new code has not been checked in.
<pre>
$ cd /path/to/your/moodle/
$ echo /mod/certificate/ >> .git/info/exclude
</pre>
To update your Moodle installation now, you must visit both Git repositories and pull changes from upstream.
<pre>
$ cd /path/to/your/moodle/
$ git pull
$ cd mod/certificate
$ git pull
</pre>
Writing a shell script with these lines in the root of Moodle installation is a very good idea. Otherwise it is easy to forget what Git repositories are there within the main Moodle repository.

== Installing and maintaining contributed extensions using Git submodules ==

As it was said in the previous section, this is for advanced users only.
Therefore it is necessary, that you have some experience with Git and its commands.
A step-by-step explanation will be provided, but in order to follow these steps it is helpful to understand, what these commands do.

Advanced options and commands can be found at [[https://git-scm.com/book/en/v2/Git-Tools-Submodules|the Git book]].
If you have any questions about Git submodules, please visit the site above first.

=== Installing a new extension into an existing Moodle ===

As an example we use the [[Certificate module]] from the previous section.
<pre>
$ cd /path/to/your/moodle
$ git submodule add https://github.com/markn86/moodle-mod_certificate.git mod/certificate
</pre>

Note, that Git is reporting two new files in the repository:
<pre>
$ git status
# On branch MOODLE_29_STABLE
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# new file: .gitmodules
# new file: mod/certificate
#
</pre>

The file '''.gitmodules''' contains the local path and url of all your submodules.
It has to be commited, if you intend to clone the repository later (see the page [[Moodle development environment with Git submodules]]).
Before commiting, make sure to check the configuration of the plugin's Git repository, since the automatically generated settings may be not sufficient.
For future updates it is helpful, to track the remote branch, which corresponds to the Moodle version of your repository.
<pre>
$ cd mod/certificate
$ git branch -avv
* master 345f5b1 [origin/master] Replaced deprecated function
remotes/origin/HEAD -> origin/master
remotes/origin/MOODLE_20_STABLE 1aa1040 Added option to print 'grade category' grade
remotes/origin/MOODLE_21_STABLE 1aa1040 Added option to print 'grade category' grade
remotes/origin/MOODLE_22_STABLE 1aa1040 Added option to print 'grade category' grade
remotes/origin/MOODLE_23_STABLE fe047de Check that the function exists rather than relying on the Moodle version
remotes/origin/MOODLE_24_STABLE 1051f7d CONTRIB-4892 Fixed the email to others functionality
remotes/origin/MOODLE_25_STABLE cdb221a CONTRIB-4946: Removed character from language file breaking AMOS
remotes/origin/MOODLE_26_STABLE 696802a Increased version
remotes/origin/MOODLE_27_STABLE d3c0379 Increased version
remotes/origin/MOODLE_28_STABLE fa8df83 Increased version
remotes/origin/MOODLE_29_STABLE 3f03740 Replaced deprecated function
remotes/origin/master 345f5b1 Replaced deprecated function
</pre>

Git created the branch '''master''' which tracks '''origin/master''' automatically, because the remote repository has checked out '''master'''.
Therefore, create a new branch, which tracks the appropriate remote branch.
Of course, this is only possible, if the remote repository offers those branches.
<pre>
$ git checkout -b MOODLE_29_STABLE origin/MOODLE_29_STABLE
Branch MOODLE_29_STABLE set up to track remote branch MOODLE_29_STABLE from origin.
Switched to a new branch 'MOODLE_29_STABLE'
$ git branch -D master
Deleted branch master (was 345f5b1).
</pre>
It is not necessary to delete the '''master''' branch, but it's useless to keep it.
In fact, these settings don't need to be touched afterwards.

The final step is to commit the changes to the main repository.
<pre>
$ cd /path/to/your/moodle
$ git commit -a -m "New extension mod_certificate installed"
</pre>

It has to be ensured, that the commit includes only the changes for the new Git submodule (since '''-a''' commits all non-staged changes).

=== Maintaining Git submodules ===

Maintaining a set of submodules is extremly easy.
Consider a Moodle repository with several submodules installed.
Keep in mind, that the extension '''mod_mylittleextension''' is a fake plugin, created for a test scenario in this example.
It is not an official Moodle module. For updating all your submodules at once, type in:
<pre>
$ cd /path/to/your/moodle
$ git submodule foreach git pull
Entering 'block/coursefeedback'
Already up-to-date.
Entering 'mod/certificate'
Already up-to-date.
Entering 'mod/mylittleextension'
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 4 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (4/4), done.
From /local/repositories/mle
89d9eae..64c122d master -> origin/master
Updating 89d9eae..64c122d
Fast-forward
index.html | 9 +++++++++
version.php | 6 +++---
2 files changed, 12 insertions(+), 3 deletions(-)
create mode 100644 index.html
$ git status
# On branch MOODLE_29_STABLE
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: mod/mylittleextension (new commits)
#
</pre>

The command '''git submodule foreach [another command]''' walks through all submodule repositiories and executes what is specified by '''[another command]'''.
In this case it is '''git pull'''.
Therefore the module mylittleextension was updated and the main repository isn't clean anymore until changes are committed:
<pre>
$ git commit -a -m "Plugin updates"
</pre>

Maintaining plugins with Git submodules has also another application than simplyfying the update process.
In a greater scale it can be used to maintain a Moodle project, where multiple developers need to have an exact copy of your moodle without organizing external plugins manually.
You can read more about this topic at the page [[Moodle development environment with Git submodules]].

== See also ==

* [[Windows installation using Git]]
* [[Git for Mac]]
* [[:dev:Moodle versions]]
* For fixing a Tracker Issue (MDL) / Forking Moodle / CONTRIButing code [[:dev:User:Sam_Hemelryk/My_Moodle_Git_workflow|User:Sam_Hemelryk/My_Moodle_Git_workflow]]
* [[Moodle_Production_Server_with_GIT|Case study Git + Moodle from Technical University Berlin]]

; Moodle forum discussions
* [https://moodle.org/mod/forum/discuss.php?d=255175 Github and Moodle deployment for production]
* [http://moodle.org/mod/forum/discuss.php?d=168094 GIT help needed]
* [https://moodle.org/mod/forum/discuss.php?d=231046 Clear git guide for Admins (not developers)]

; External resources
* [http://thamblings.blogspot.com.au/2013/07/upgrading-moodle-from-git.html Deploying Moodle from git - Blog post from a production experience]
* [http://gitref.org/ Git Reference]
* [http://progit.org/book/ Pro Git book]

[[ja:管理者用Git]]
[[fr:Git_pour_administrateurs]]
[[es:Git para Administradores]]