Development:CVS for developers
NOTE: Moodle has now adopted Git for core development along with a new and evolving workflow. This document may no longer apply
CVS is the Concurrent Versioning System, a commonly-used way of managing source code for large software projects. CVS keeps all versions of all files so that nothing is ever lost, and usage by different people is tracked. It also provides ways to merge code if two or more people are working on the same file. All code and all versions are stored on a central server (in the case of Moodle, at cvs.moodle.org). The CVS book holds more information about CVS than you need.
If you just want to download Moodle using CVS to run a site, then you probably don't need this page - please see CVS for Administrators instead.
Joining the project as a developer
So, you've been offered CVS write access to help us develop and maintain Moodle! Welcome aboard!
To be able to write changes into Moodle's CVS archive, you first need to have an account on the server. Only trusted developers get these accounts. To request access, go to the "Apply for CVS Access" tab on the CVS page on Moodle.org (http://moodle.org/cvs). Tell us what modules you'd like to access (e.g. a language module), and why. You'll receive notification as soon as one of the core admins can authorise it. (Please be patient, but if you feel your application is languishing, message Martin D on moodle.org :) )
With that done, you should have all the permissions you need, so you just need to set up your machine and download the current source code so you can start working on it.
For the examples on this page, let's assume your username is myusername and your password is mypassword. Things are a lot easier if you use an SSH key to access the server, as you won't have to keep typing these in. Check here for instructions on how to make one.
Within CVS, the word "modules" refers to separate collections of code. In Moodle we have the following modules within our repository:
- moodle - the main Moodle source code
- lang - all the language packs
- contrib - user contributions and other assorted code in development
- mysql - a customised phpMyAdmin to plug into Moodle for database admin
- windows-cron - a small package that makes cron possible on Windows systems
- docs - various extra user-contributed documentation
Most people are working on the existing features in the moodle module, but many are also contributing new ideas in the contrib modules. Once code reaches a certain level of maturity in the contrib area, it can be migrated over into the main moodle tree.
Basic CVS commands
CVS on Unix
Moodle CVS uses ssh as a transport layer for security, so you will have to set a CVS_RSH environment variable in your Unix shell. It's best to put these commands in your .bashrc or .cshrc so you don't have to type it all the time:
setenv CVS_RSH ssh (for csh, tcsh etc) export CVS_RSH=ssh (for sh, bash etc)
Next, you can check out the latest development version of Moodle using this (all one line). NOTE: Don't try to do run this first CVS command over an existing moodle installation: start fresh with a new directory!:
cvs -z3 -d:ext:firstname.lastname@example.org:/cvsroot/moodle co -P moodle
The command is similar for other CVS modules:
cvs -z3 -d:ext:email@example.com:/cvsroot/moodle co -P contrib
If you want to checkout a single plugin (attendance block as an example here) from contrib into the current directory, you can use:
cvs -z3 -d:ext:firstname.lastname@example.org:/cvsroot/moodle co -d attendance contrib/plugins/blocks/attendance
Note that you will be prompted for mypassword for each command unless you set up authorized keys. Read SSH Keys for more details on how to set those up.
Now, you should have a new 'moodle' directory. You can rename it and move it around if you like. Go into it:
All the latest Moodle files should be in there. You can now change files in your copy. To compare your files and directories against the main CVS copy on the server use cvs diff, e.g.:
cvs diff -c config-dist.php cvs diff -c lang
To fetch the latest updates from the server use:
cvs update -dP
To copy your new files back to the server you would do something like:
cd lang/ca cvs commit
You will be prompted to add some comments (depends on your default text editor). Please write a meaningful, descriptive comment and always include the name of any tracker issues that talk about the issue you are fixing (eg MDL-XXXX).
After that your changes will be sent to the CVS server and stored in the repository. Done!
To save more time you can put default arguments into a file called .cvsrc in your home directory. For example, mine contains:
diff -c update -dP
Try 'cvs help' for more details ...
CVS on Mac OSX
You can follow the same instructions as for Unix (above) in a terminal window. However, the cvs command is not installed by default in an OSX. You first need to install the Xcode Tools. You should find this on your original installation disk. Failing that it can be downloaded (a fairly hefty download) from the Apple developer web site.
CVS on Windows
First, you need to download a completely fresh copy of Moodle using your developer account. (If you only want a private copy for local development, anonymous login will work too.)
1. Get TortoiseCVS and install it, then reboot. TortoiseCVS works under Windows 95, 98, ME, NT, 2000, XP, and 2003. Vista is also supported, although some people report problems with UAC. (In the latter case you might resort to SmartCVS.)
2. Find or create a new folder somewhere where you want Moodle to be downloaded to.
3. Right-mouse-click that folder and choose "CVS Checkout" from the menu. You should see a dialog box.
4. Copy this text into the CVSROOT field (using your own username!):
5. Under the "Module" field, type "moodle" to get the latest development version of Moodle, "contrib" to get the contributions directory, or "mysql" to get the MySQL Admin module.
6. Press the button: "OK" and everything should be downloaded.
A dialog box should show all the files being downloaded, and after a while you should have a complete copy of Moodle. After this first checkout, you can fetch the latest updated files from the CVS server:
- Right-mouse-click on your Moodle folder (or any file) and select "CVS Update".
- Sit back and watch the logs scroll by. Take note of conflicts that may occur if your local code has changes that conflict with the incoming versions - you will need to edit these files and resolve the conflicts manually.
After modifying files (you will notice their icons change from green to red!), you can commit them back to the CVS server like this:
- Right-mouse-click on your Moodle folder (or any file) and select "CVS Commit...".
- In the dialog box, type a clear description of the changes you are committing. Always include the name of any tracker issues related to what you are fixing (eg MDL-XXXX).
- Click "OK". Your changes will be sent to the server.
- If you create a folder, BE CAREFUL about using the "CVS Add" option as it will add the folder to CVS without requiring a commit. Once added, the folder cannot be removed from CVS even though it will be pruned so long as it is empty.
N.B. I had enormous headaches with the above settings until I changed from ext to ssh. This was using an ssh certificate on windows 2003 to both check out and commit code. Your mileage may vary, of course. Matt Gibson 12:52, 18 August 2008 (CDT)
CVS through your IDE
Will naturally depend on the IDE you choose to use so we do not give specific instructions here. For some IDEs, there are instructions elsewhere here. (E.g. for Eclipse.)
However, be warned, we have noticed that when adding a new file to CVS using some IDEs (for example Eclipse), then by default they forcibly set the CVS substitution mode to -kk for ASCII files. In Moodle we like all ASCII files to be stored with the CVS default of -kkv, so if you use an IDE like this, please adjust its default.
If you are never likely to add a new file to CVS, you don't need to worry too much about the preceding paragraph. If you know something version control, you might think that -kk is better than -kkv, and in many ways you would be right. (For example it makes merging easier.) However, for the Moodle project, we use -kkv because it makes it easier to generate the Unmerged files page, and because expanded $Id$ tags in the download packages makes it easier for people to make more specific bug reports.
CVS commit messages
Every commit you make to CVS should have a commit message. These can be invaluable. They are included in the automatic emails sent out whenever a change is made to Moodle, so they help other developers follow and review what is being changed; and they are very helpful when tyring to understand a piece of code. CVS will tell you who last changed each line of code, and when they did it. The commit comment (hopefully) tells you why.
A commit message should contain, in order:
- A couple of words indicating which part of moodle this change affects. For example 'forum backup', 'quiz attempting' or 'weblib'.
- A tracker issue id like MDL-12345. (If necessary, create an issue before committing, the only exception is for very minor typos.)
- A brief description of the problem you are fixing. (If you are feeling lazy, you may be able to get away with copying and pasting the issue summary from the tracker.)
- If it is not immediately obvious, a brief summary of why this change fixes the problem. If a longer description is necessary, you may also want to add more information to the tracker issue, or Moodle Docs, but the code changes + commit message should provide enough clues to how the fix works on their own.
Here is an example of a commit message from a simple change:
user selection: MDL-17072 Polishing the role assign page: Do processing before print_header in line with best practice.
Actually, the ideal commit message would be a bit shorter than that. It is good if the first three items (area of Moodle, tracker id and brief description) will fit in the subject line of an email message.
Here is an example from a bigger commit (would it have been better if this could have been done as several smaller commits?):
role overrides: MDL-17070 Improve override roles page to match the recent usability improvements on the assign page. Including: MDL-11529 When assigning/overriding roles, the dropdown for switching to another role should have a number in brackets MDL-16549 Should not be able to edit the permission associated with moodle/site:doanything on any role.
Here is an example of a commit (from a while ago, which is why the issue number is in the wrong place) with some explanation of the change:
accesslib: get_user_by_capability() - Handle complex rolecap resolution With this patch, get_user_by_capability() can handle the cases where users have multiple role assignments to the same course, and PREVENTs and PROHIBITs affect the rolecaps of this course. Without stored procedures we cannot resolve this entirely on the server side - so in the complex cases we do as much as we can on SQL, and post-process the data on the PHP side, including SQL-style pagination. MDL-12452
The CVS commits email list
You can subscribe at http://lists.moodle.org/info/commits to this email list. You will then get an email every time someone commits a change to Moodle. This is a good way to keep track of what is going on, and also a good way to learn best-practice for CVS commit messages.
You probably want to set your mail client to automatically move messages from this list into a separate mail folder. You may not want to remain subscribed all the time, but when you are new to the project, it is a great way to do a bit of social-constructionist peer learning about our use of CVS.
Working with branches
This diagram shows how the main moodle module branches into different versions over time.
To see all the current tags and branches that are available, use this command on any old file (such as index.php in the top moodle directory):
cvs status -v index.php
Some tagging guidelines:
- Tag and branch names should always be all upper-case.
- Tags and branches should ALWAYS be applied to the entire module (all of Moodle). Don't tag individual files or directories.
- We don't allow renaming of tags because people may be relying on them, so get them right the first time!
The Trunk of CVS is the main development version of Moodle. In CVS it is also known as the HEAD, or default branch.
Moodle developers try to keep this stable as possible, but as it usually contains new code it probably has bugs and small instabilities.
Every now and then we decide the product has enough features to make a release. At this time, the trunk is tagged with a MOODLE_XX_BETA tag (in case we ever want to roll back to that point) and a new branch is formed for the release, called MOODLE_XX_STABLE.
A Beta package is also released at this point - it's for testers who don't use CVS but want to test the latest features and report bugs.
Stable branches for each release
As soon as the stable branch MOODLE_XX_STABLE is created, development efforts will fork into two streams for a while. Some people may continue working on new features in the trunk for the next release, but most developers should be concentrating on using the current STABLE branch and fixing bugs that are found in it.
You can switch your local copy of Moodle to the STABLE version using the following command in Unix from the root directory:
cvs update -dP -r MOODLE_XX_STABLE
After that, all the commands described above will apply to that stable version. To return to the trunk version just issue:
cvs update -dPA
On Windows clients you should have a menu from which you can choose the branch.
Once the new STABLE branch really stabilises, a release can be declared. Packages are created for distribution and the branch will be tagged (by Martin Dougiamas) with a tag named: MOODLE_XXX
All bug fixes in any STABLE branch should be merged back into the trunk so that they become available in future versions of Moodle. A floating tag called MOODLE_XX_MERGED needs to be maintained to keep track of the last merge. The procedure for such a merge is as follows (I'm using CVS on the Unix command line but the steps are the same for any CVS client):
1. I highly recommend keeping one checked out copy of each stable branch and the trunk/head version locally to make it easier to jump between them. Set them all up as virtual web sites on your development machine. I use directories like /moodle/18, /moodle/19 /moodle/dev etc.
2. Update to the latest stable version (I'll use XX in the example but it could be 18, 19 etc) for the relevant files, and do a cvs diff just to double-check exactly what you are checking in. Note you only need to update the files/directories you are working in, but sometimes it help to update everything anyway just to do a final check on the functionality using the web.
cd /moodle/XX/user cvs update -dPA cvs diff -c file1.php file2.php
3. If all looks OK, check in the fix to the stable branch. Make sure that the commit message contains the bug tracker ID (eg MDL-1111) and a decent description of your thoughts on the fix. If you don't specify a message in the command line, you'll be put into an editor to type one.
cvs commit -m "MDL-1234 Corrected a small typo in the user name field" file1.php file2.php
4. Go to the very latest trunk version and make sure it's up-to-date.
cd /moodle/dev/user cvs update -dPA
5. Merge everything into your local copy of the trunk from your stable branch since the last merge. You can use the same sequence of (4) and (5) to merge the changes into other stable branches too (to backport the fix to the previous stable versions).
cvs update -kk -j MOODLE_XX_MERGED -j MOODLE_XX_STABLE file1.php file2.php
6. Carefully watch the update logs for conflicts, and fix every file that you see with a conflict. Afterwards, it may help to just run a diff to make sure the result is what you expected:
cvs diff -c file1.php file2.php
7. Check your merged local copy back into CVS trunk version
cvs commit -m "MDL-1234 Corrected a small typo in the user name field, merged from XX" file1.php file2.php
8. Go back to your branch version
9. Update the floating merge tag for the affected files (so that it matches MOODLE_XX_STABLE) so that this process can be repeated next time
cvs tag -F MOODLE_XX_MERGED file1.php file2.php
Finally, the values for $version in all the Moodle version.php files within the stable branch should not be updated at all if possible (except the last digit if absolutely necessary). The reason is that someone updating from a very stable version to the next very stable version could miss database upgrades that happened on the trunk.
In other words, if you have changes to the database schema to commit to a stable branch, please check with Martin Dougiamas, or one of the other core Moodle developers.
Sometimes you fix something in a branch that needs to be "backported" to an earlier branch. This is often the case with security fixes. The procedure is exactly the same as described above, except you apply your changes backward instead of forward (to previous versions instead of HEAD). Here is a summary, using the steps described above. The scenario is that you have a fix in MOODLE_19_STABLE that needs backporting to MOODLE_18_STABLE, but NOT to HEAD (the problem you are fixing doesn't exist in HEAD):
cd /moodle/19/lib cvs update -dPA cvs diff -c file1.php file2.php
cvs commit -m "MDL-1234 Fixed major security hole" file1.php file2.php
cd /moodle/18/lib cvs update -dPA
cvs update -kk -j MOODLE_19_MERGED -j MOODLE_19_STABLE file1.php file2.php cvs diff -c file1.php file2.php
cvs commit -m "MDL-1234 Fixed major security hole, backported from 19" file1.php file2.php
When it comes to tagging, things change a little bit. You've now got changes in 18 and 19, both of which are stable releases, so need to be tagged as MERGED. You start with 18 (assuming you are still in that directory)
cvs tag -F MOODLE_18_MERGED file1.php file2.php
Then, you go and do the same for 19, regardless of whether or not the changes were merged into HEAD. Of course, if the changes need to be merged into HEAD, you do that first. But if they don't, you still need to tag the modified files of each stable branch once your fixes are complete and tested:
cd /moodle/18/lib cvs tag -F MOODLE_19_MERGED file1.php file2.php
Merging fixes with TortoiseCVS
Situation: we did a fix on MOODLE_19_STABLE. We modified one file for this fix. This fix needs to be done on Moodle 1.8 and trunk(HEAD). All following CVS instructions will be applied on the file that we changed.
1. Update to the latest stable versions (HEAD included)
Right click on the moodle folder CVS update OK
2. On your MOODLE19 branch: commit your fix
Right Click on the file CVS commit Enter a description: "MDL-XXXX bug fix description" OK
3. On your HEAD repository: merge the changes
Right Click on the file CVS> Merge... Start: MOODLE_19_MERGED End : MOODLE_19_STABLE OK
4. If the resulting file have some conflicts, TortoiseCVS displays a red square on the file icon. Edit the file with your text editor and resolve the conflicts.
5. Run a diff to make sure the result is what you expected
Right Click on the file CVS Diff OK
6. Commit the fix
Right Click on the file CVS commit Enter a description: "MDL-XXXX bug fix description, merged from 19" OK
We've commit MOODLE_19_STABLE and trunk. It's now time to backport the change on MOODLE_18_STABLE. On your MOODLE18 branch: reproduce step 3 to 6.
7. On your MOODLE19 branch: update the floating merge tag for the affected file
Right Click on the file CVS> Tag... Tag: MOODLE_19_MERGED Select 'Move existing tag' OK
On your MOODLE18 branch: reproduce step 7 (with Tag: MOODLE_18_MERGED).
Feature branches for large changes
Occasionally, there may be a very large feature that needs to be checked in so several people can work on it, but it is too unstable to be included in the main development trunk.
In these cases a short-term branch can be created to work on the feature, and then merged back into the main trunk as soon as possible. An example called MOODLE_19_WIDGET branch can be seen in the above diagram.
If you need to do this for your new WIDGET feature, follow these steps:
1. Discuss with other developers to make sure it's necessary!
2. Make a new tag on the trunk (for all of moodle) called MOODLE_XX_WIDGET_PRE
cvs tag -R MOODLE_XX_WIDGET_PRE
3. Create your branch called MOODLE_XX_WIDGET
cvs tag -Rb MOODLE_XX_WIDGET
4. Work in that branch until the feature is reasonably stable. Commit as necessary.
5. When ready, merge the whole branch into the trunk, fix conflicts, commit it to the trunk and then abandon the branch.
cvs update -dPA cvs update -kk -j MOODLE_XX_WIDGET cvs commit
Good luck, be careful and have fun!
Who on earth is ...
Some of the people committing to the Moodle codebase have non-boring account names on the CVS server. If you are ever looking at a CVS commit, and wondering "who on earth is this purpleblob person?" This information is now available at: Moodle developers with write access.
Tips and Hints
- Change your password or SSH key (click on "Update my developer information")
- Tracking Moodle CVS with git
- Merging Custom Moodle Code With Stable Releases
- Unmerged files: To see if you've forgotten to merge any file.
- CVS Manual
- Introduction to the contrib area of CVS