Workshop 2.0 specification
This page tracks and summarizes the progress of my attempt to try and rewrite the Workshop module for Moodle 2.0 (yes, yet another attempt).
PROJECT STATE: Planning
- 1 Introduction
- 2 Current implementation problems
- 3 Project analysis
- 4 Project schedule
- 5 User interaface mockups
- 6 Implementation plan
- 7 Out of scope
- 8 To be discussed/decided
- 9 Other links and resources
- 10 Credits
Workshop module is an advanced Moodle activity designed for peer-assessments within a structured review/feedback/grading framework. It is generally agreed the Workshop module has a huge pedagogical potential and there is quite a big demand of the community (including some Moodle Partners) to have such module. It was originally written by Ray Kingdon, and was the very first third-party module written for Moodle. For a long time it's been largely un-maintained except for emergency fixes by various developers to keep it operational. During last years, several attempts were done by various volunteers to rewrite the module or to replace it with an alternative. At the moment it is removed from the HEAD branch and lies in the CONTRIB area. This document specifies the proposed way of revitalizing the Workshop module so it becomes the standard core module again in Moodle 2.0.
There are two key features making the Workshop module unique among other Moodle activies
- Advanced grading methods - contrary to the standard Assignment module, Workshop supports more structured forms of assessment/evaluation/grading. Teacher defines several aspects of the work, they are assessed separately and then aggregated. Examples include multi-criterion evaluation and rubrics.
- Peer-assessment - according to some psychology/education theories (for example the Bloom's taxonomy), the evaluation is one of the highest level of cognitive operations. It implicitly requires paying attention, understanding the concepts, critical thinking, knowledge applying and analysing of a subject being evaluated. Peer-assessment fits very well into the social constructionist model of learning. In Workshop, students not only create (which is the other non-trivial cognitive operation) and submit their own work. They also participate on the assessment of others' submissions, give them feedback and suggest a grade for them. The Workshop module provides some (semi)automatic ways how to measure the quality of the peer-assessment and calculates a "grading grade", ie. the grade for how well a student assessed her peers.
Basic scenario of the module usage
The following scenario includes all supported workshop features and phases. Some of them are optional, eg. teacher may decide to disable the peer-assessment completely so the Workshop behaves similarly to the Assignment module with the benefit of multi-criteria evaluation.
- Workshop setup - teacher prepares workshop assignment (eg. an essay, research paper etc.) and sets up various aspects of the activity instance. Especially, the grading strategy (see below) has to be chosen and the assessment elements (criteria) defined.
- Examples from teacher (optional)
- teacher publishes example submissions - eg. examples of a good work and a poor work
- teacher publishes example assessments of the example submissions. Again, there may (and should, in fact) be an example of very well done evaluation/assessment and an example of a poor one.
- students can try and train the process of assessment of example submissions (this part to be discussed: how would the module compute the grading grade?)
- Students work on their submissions - the typical result is a file (several files?) that can be submitted into the workshop (together with a comment maybe?). Students may upload draft version of their work before they decide it is a final submission (or there is a deadline for submissions). Other types of work are possible using the new Repository API - eg. students can prepare a view in Mahara, GoogleDoc, publish video at YouTube etc.
- Self-assessment (optional) - after the work is submitted, the student is asked to assess her own work using the selected grading strategy evaluation form
- Peer-assessment (optional) - the module randomly selects a given number of submissions to be reviewed/commented/evaluated/assessed by student.
- Assessment by teacher (optional)
- teacher manually evaluates submissions using the selected grading strategy evaluation form
- teacher manually evaluates the quality of peer-assessments
- Final grade aggregation - TODO: more research of the current code and behaviour is still needed in this area
- generally, the final grade consists of a grade for submission and grade for assessments
- every grading strategy defines how the final grade is computed
Current implementation problems
- Very old and unmaintained code - the vast majority of the code is the same as it was in Moodle 1.1. mforms are not used (accessibility). Missing modularity. No unit testing.
- Does not talk with the Gradebook - there are some patches around in forums fixing partial problems.
- User interface and usability
- Grades calculation is kind of dark magic - see this discussion. It is not easy to understand the way how Workshop calculates the grading grade even if you read the code that actually does the calculation. More than that, the whole algorithm depends on a set of strange constants without any explanation or reasoning of the value. The grade calculation must be clear and easy to understand to both teacher and student. Teachers must be always able to explain why students got their final grades. Therefore the plan is to get rid of the fog above the grades calculation, even if it breaks backward compatibility.
- Lack of custom scales support - see this discussion
- Phases overlapping, balancing, overall allocation issues - see this discussion
- Date: 15/05/2009
- Goal: The functional specification is in docs wiki and is reviewed and agreed by the community and Moodle HQ. The implementation plan is transferred into sub-tasks in the tracker.
- Date: 15/07/2009
- Goal: All features implemented. The community is asked for the testing.
- Date: 15/08/2009
- Goal: Major bugs fixed, upgrading from pre-2.0 versions works. The community is asked for the QA testing.
- Date: 15/09/2009
- Goal: The module is moved from contrib back to the core.
User interaface mockups
The structure of the code
Files, libraries, interfaces, classes, unit tests
File API integration
How Workshop will use the new File API.
Repository/portfolio API integration
If and how will be implemented.
We are currently working on this part of specification, see - the Google doc
Pluggable submodules in mod/workshop/grading/xxxx/. Implement grading_strategy interface. Basically provide grading form and way of grade calculation. During the current phase, four strategies will be implemented (the names are to discuss)
See MDL-18912 for UI mockups of grading forms.
Number of errors
Currently called Error banded
Merges the current Rubrics and Criterion strategy into a single one (basically, current Criterion is an one-dimensional Rubrics)
Out of scope
What are not goals of this project (may be discussed later for 2.1 etc.)
- No new/other grading strategies are to implement in the current phase
- No new Workshop features will be implemented (if possible)
- No group grading support
To be discussed/decided
- MDL-17827 Workshop upgrade/conversion from 1.9 to 2.0 (META)
- Yet another attempt to rewrite Workshop for 2.0 forum thread
- New Workshop Module forum thread
- the new workshop module forum thread
- a lot of interesting feature requests
Many thanks to Stephan Rinke for his valuable comments and ideas.