Policy - Retroactive effects of completion settings
Summary
The question was:
Should past activity completions be retroactively affected by changes to settings? That is, after a student completes an activity and it's marked as "completed-passed", "completed-failed", etc., might it later become "incomplete"?
This question was decided by by Martin Dougiamas on Tue Mar 25 per tracker issue MDL-39624. Dougiamas decided:
Completion of an activity is final and not affected by future settings changes (unless you manually reset it and are warned loudly and students are informed).
Documentation of the reasoning, pro and con, is below as it was posted prior to the decision being made.
As one example, a student might complete a Quiz, with a passing grade. The quiz is marked as completed-passed, which opens up a Certificate. The student prints the certificate and files it with regulators. Six months later, the teacher decides that from now on the passing grade should be higher. The teacher changes the "grade to pass" setting. Should the student who completed and passed six months prior now be marked as "failed", and their certificate revoked?
Two major categories of actions were mentioned, and two types of "intentions", or use-case scenarios, with different policies appearing reasonable depending on the type of
action and use-case or intention of the user. Actions can be a) changes to what a student has done , such as submitting or deleting an assignment or b) settings/requirements adjustments by the teacher.
Settings changes may be the result of two types of intentions - a decision that the new requirements should be different from the old, or a correction to an incorrect setting.
Changes to what a student has done
Sam Marshall gave two examples suggesting that when there is a change to what a student has done, that could change the fact that something the student completed something.
Example 1: If you just use 'on grade' completion, if a student does something and gets a grade, it will be marked complete. If the teacher then deletes their grade from gradebook, it will be marked incomplete again.
Example 2: In forum, if you set it to be complete after making 3 posts, and the student makes 3 posts, it will be marked complete. But if the teacher then deletes one of the posts, it will be marked incomplete again.It could be argued that things should not be automatically 'un-completed' in this way (there are cases when both the above examples definitely seem appropriate, but it might be OK if the teacher had to manually un-complete them as well)
No reasoning was posted with an opposing view on what should happen when changes are made to what a student has done.
Changes to settings
There are different opinions in regards to changes to settings, such as the grade required to pass. If a student completes an activity, such as passing a test, and some time later the "grade to pass" is changed, should their "completed-passed" be changed to "completed-failed", or should the changes apply "from now on"?
Per Ekström wrote:
I would say, no.
As a student, you're given a deal; "Get a score of X or higher and you pass." If the bar is raised, and it's applied retroactively, then you are basicly breaking your end of the bargain. A pass is a pass no matter what, even if a teacher afterwards raise to 80%. ...
The most important thing with a grading system is that it's predictable and perceived fair. Retroactivity makes it everything but. Therefore I do not recommend it.
Where Ekström uses the word "deal", common law uses the word "contract", and "breaking your end of the bargain" could be called "fraud".
Another example supporting this view was posted:
Working as a private investigator without passing the required classes and staying up with continuing education is a crime. If I took the class and later the teacher changed the requirements and the system retroactively failed me, that would expose me, the student, to criminal charges.
A different view was expressed by Tim Hunt:
As a teacher, you are only human. Sometimes you make mistakes. You accidentally create a quiz with a passing grade of 90% (or forget to set the passing grade so it defaults ot 0%?). Then some students attempt the quiz and point out the mistake to you, so then you correct it to 80% and expect Moodle to automatically fix the completion state for students who already got between 80 and 90%.
It was suggested that if the teacher knows they made a mistake and are correcting it, they can also correct the results of that mistake - manually changing passes to fails or vice-versa.
That is the open question - if settings/requirements are changed, should Moodle automatically, retroactively change prior completions based on the new settings?
Can changes for these two reasons be separated? (Not really.)
(sam marshall)
It will be technically difficult to separate the 'changing based on what a student has done' from 'changing settings'. I think we will need to make the behaviour consistent so that either of two rules applies:
OPTION B: Once a 'completed' or 'completed-passed' completion status is obtained, it should not change unless a teacher manually resets it.
OR (as supposedly true at present)
OPTION A: All completion statuses should change if the conditions or data changes.
(Note that I left completed-failed undefined because we want to let people retry things so it can't be 100% consistent this way...)
In other words I don't think we should attempt to treat 'changes in data' and 'changes in settings' as distinct situations that can be handled differently, as the page above might imply.
Example
Here's why I think it is impossible to treat the two cases (changes to actual data/user input vs changes to settings) as separate in general. Imagine that we are trying to do so (update 'completed' to 'not completed' for data changes, but not for setting changes). Consider this sequence:
1. In a forum, 3 posts are required for completion.
2. Student A makes 4 posts and is marked complete. Student B makes 3 posts and is marked complete.
3. Somebody changes the settings so that 5 posts are now required. (This is a settings change, so completion states aren't updated.)
4. Somebody deletes one of Student A's posts so that they now only have 3 posts. (This is a data change, so completion state is updated.)
In this situation, there has been a change to the data, so if we are treating those as separate then it is correct to potentially mark the student as not complete. But we do not store the historical values of all the setting, so we have to consider this according to current data; i.e. every time somebody gets a post deleted the system needs to check if they still have more than 5, and student A does not. As a result, student A would be marked 'not complete'.
This would be unfair because both students A and B have made 3 valid posts but B is still marked complete and A is not. (Note: In the current system, both A and B would have been marked incomplete at stage 3. In Option B, both A and B would remain complete; a student C who only made their posts after step 3 would have a disadvantage, but at least that's more easily understood i.e. when teachers change the settings they know they are making it harder for people from now on.)
[In other words, the new criteria would apply only to currently active students, and removing a post indicates activity. This would not be ideal, but it separating them may be better than the alternatives. - Ray Morris]
