Lightwork

Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.

Lightwork
Project state	In Development
Tracker issue	http://tracker.moodle.org/browse/MDL-31409
Discussion	http://moodle.org/mod/forum/discuss.php?d=187818
Assignee	Lightwork

Moodle 2.4

Summary

Lightwork is a marking tool for managing and marking student assignments. See http://lightwork.massey.ac.nz/projects/show/fat and http://lightworkmarking.org/. It consists of a client application that was created using the Open source Eclipse Rich Client Platform see http://wiki.eclipse.org/index.php/Rich_Client_Platform and web services for communication with Moodle. Lightwork downloads and manages student submissions so that marking can be done on the client computer. It can be used offline with changes synchronized to Moodle the next time the teacher connects.

Currently, the Lightwork web services are installed in a Moodle installation by using the local customisations hook and the web services are implemented using the OKTech framework. See https://docs.moodle.org/20/en/Web_Services:OK_Tech_Web_Services. It is now proposed to add the Lightwork web services to the core Moodle 2.4 release as described in the rest of this document.

Using Lightwork with Moodle for offline marking

The following user stories are designed to help explain how Lightwork is used with Moodle for offline marking:

Creating a Marking Guide and setting it ready for marking

A teacher decides that they would like to use Lightwork's marking guide (introduced to Moodle in 2.3) for an assignment that they have given to their students. The teacher is a busy person who spends a lot of time out of the office without an internet connection and wants to be able to continue developing the Marking Guide in any spare moment that they have. They decide to use Lightwork to do this:

1. In their office on their work PC the teacher begins work on the Marking Guide
2. The teacher remembers that they will be out of the office for the next few days. There is still lots to do in order to finish the marking guide. They remember that Lightwork allows them to work offline.
3. The teacher installs Lightwork on their new laptop. They connect to the internet and synchronize Lightwork with their Moodle instance.
4. Within Lightwork they can now all their courses and assignments. They can see their partially completed Marking Guide.
5. The next day, the teacher is waiting in their car to meet with a colleague and remembers the unfinished Marking Guide. They open Lightwork in their laptop and continue developing the Marking Guide in Lightwork's interface. They add/modify criteria and frequently used comments to their marking guide. When their colleague arrives they still haven't finished. The nearly completed Marking Guide is stored in Lightwork but cannot be synchronized to Moodle since there is no internet connection.
6. After several more offline sessions, the teacher completes the marking guide
7. The teacher arrives home at the end of the week. They want the other teachers to review their Marking Guide.
8. The teacher connects their laptop to the internet, opens Lightwork and synchronizes it with their Moodle instance.
9. The teacher's completed Marking Guide is now synchronized up to Moodle. Since they've been away more students have enrolled on the Moodle course, this new information is synchronized down to Lightwork.
10. The teacher phones the other 2 teachers who help to teach the course. The first one only uses Moodle to mark, the second only uses Lightwork. The first teacher views the Marking Guide in Moodle. The second teacher synchronizes Lightwork and views the Marking Guide.
11. The next the teachers all get together to discuss the Marking Guide. They agree some changes which they make in Lightwork. They then set their Marking Guide ready for marking and synchronize their changes with Moodle.

Challenges and issues

Lightwork stores marker allocation, rubrics and rubric instance information in its own table structure. This structure must be modified to integrate with and enhance Moodle.

• Lightwork makes the use of a rubric compulsory. This is not the case in Moodle.
• Moodle already has the ability to define and use a rubric which was introduced in Moodle 2.2.
• Moodle defines its rubric in database table structure whereas Lightwork uses XML
• Moodle has no marker allocation capability. Lightwork's marker allocation must be introduced so that it can be used by both Moodle and Lightwork

Design goals

Rubric compatibility between the Moodle rubric and Lightwork

Lightwork supports 2 types of rubrics. I'll refer to these as rubric type 1 and rubric type 2. These rubrics are currently stored as XML and there is the potential to create other rubric types. Moodle has one type of rubric, the advanced grading rubric. This is equivalent to Lightwork's type 1 rubric. The type 2 rubric allows the teacher to choose a mark for a criterion as opposed to selecting from fixed defined levels. A possibility for a name for the type 2 rubric might be marking guide, see http://moodle.org/mod/forum/discuss.php?d=195738&parent=854867.

Since Moodle has already implemented rubrics by using database tables instead of XML, Lightwork on Moodle will have to change to follow this approach.

Lightwork will need to store its type 1 rubric and rubric instances using the rubric grading tables in Moodle. These tables are:

• mdl_grading_definitions
• mdl_grading_instances
• mdl_gradingform_rubric_criteria
• mdl_gradingform_rubric_fillings
• mdl_gradingform_rubric_levels

In order to implement the type 2 rubrics we could either:

• Create a new gradingform plugin for the type 2 rubric. This might be called gradingform_markingguide
• Modify the existing gradingform_rubric so that it supports the type 2 rubric. The table structure required for the type 2 rubric is very similar and differs only in that mdl_gradingform_rubric_fillings would require an additional field to store the mark and mdl_gradingform_rubric_levels is not required.

Issues:

• The Lightwork type 1 rubric allows weightings to be defined for criteria. The Moodle rubric does not support this.
• All Lightwork rubrics support the use of a bank of frequently used comments. The Moodle rubric does not support this but could be extended to use this feature.
• It would be useful to keep the ability to store rubrics in XML format since this would allow transformation into different document types.
• Lightwork would need to be modified to store rubrics and rubric instances in the mdl_grading* tables

Marker allocation and rubric definitions

Lightwork requires marker allocation and a rubric to be used. This was a deliberate design decision to enforce the use of rubrics. Marker allocation will be introduced as part of the new assignment module and will be compatible with Lightwork's current marking process. The following scenarios will be possible in Moodle 2.3:

Scenario in Moodle	Impact of choosing to use Lightwork
Teacher creates an assignment. They choose to use the new Moodle 2.3 marker allocation process and the rubric grading method	Lightwork marking as usual but using the advanced grading rubric already created in Moodle
Teacher creates an assignment. They choose to use the simple grading method and no marker allocation	Choosing to use Lightwork will create marker allocation. Teachers and markers can use this allocation for grading within Moodle if they choose. If Lightwork uses a type 1 rubric it will be saved using the current rubric grading definition. The teacher can now switch to the rubric grading method to take advantage of the rubric created in Lightwork. If Lightwork uses a type 2 rubric it will be saved in Moodle using the same advanced grading mechanism.

Marking and rubric grading instances

Moodle uses the mdl_grading_instances and the mdl_gradingform_rubric_fillings tables to store the results achieved by the students based on the rubric. When marking a Moodle compatible rubric (Lightwork type 1 rubric) in Lightwork, the results will be stored in the same way.

The new proposed Lightwork and marking allocation database structures

This new database structure needs to support the ability for markers (usually teachers and non-editing teachers) to allocate students to markers for the purpose of grading their work. It will be used by both Lightwork and Moodle. This will require new tables to be designed and implemented

TODO - design the marking management tables

The existing mdl_grading* tables will be used to store the Lightwork rubrics. Depending on the approach taken (see previous section) these tables may either need to be modified or a new plugin created to support the Lightwork type 2 rubric.

The new Moodle 2.3 assignment module

In addition to adding Lightwork's web services into the Moodle core, many of Lightwork's features are being included in the Moodle 2.3 assignment module. This work is being done in conjunction with https://docs.moodle.org/dev/Assignment. The new features are:

• An additional type of rubric that allows markers to specify the mark given for a criterion. This differs from the current rubric where the marker must select from one or more fixed marks. We are currently undecided on the best name for this type of rubric. We are currently calling it a Marking guide.
• Frequently used comments. A bank of Frequently used comments can be defined for a rubric. Markers can select and add these comments when providing feedback based on a rubric.
• Marking management. A marking manager can allocate students to markers. A marking process is used to ensure consistency of marking.
• Team submission. Students work as a team and submit a single team submission.

The new Lightwork web services

These web services use the Moodle web services with the REST protocol. The login and password mechanism in the old web services is being replaced by tokens. Token based authentication is replacing password based authentication in Moodle.

methodname	classname	classpath	parameters	type	description and comments
core_webservice_get_site_info	core_webservice_external	webservice/externallib.php	None	read	This is required to ensure that the Lightwork client is compatible with the Moodle version it is calling. This existing Moodle function returns a build number for each web service function. Currently, for core web services, the function just returns the build number in the main version.php file. This function needs to be modified to return the release number and the version number. The release number is a user friendly number that the client can display and it indicates the Moodle version number for which the current version of the client has been designed to function. When the client detects that the Moodle version has changed, the client will either automatically update itself or it will prompt the user to perform the update to match the Moodle version to which it is connecting.
mod_assign_get_assignments	mod_assign_external	mod/assign/externallib.php	courseids (optional), capabilities (optional)	read	http://tracker.moodle.org/browse/MDL-31683 Return list of courses and assignments that this user has a particular capability in.
core_users_get_users_with_capability	core_course_external	course/externallib.php	List of course ids, Flag to specify whether all students are returned or just the ones that have submitted a submissions. List of capabilities to filter the results.	read	http://tracker.moodle.org/browse/MDL-31859 Return list of participants to specified courses. This should include students and teachers. User details are returned together with capabilities.
mod_assign_get_submissions	mod_assign_external	mod/assign/externallib.php	List of assignment ids (required), status (optional), since (optional), before (optional)	read	http://tracker.moodle.org/browse/MDL-31682 Return assignment submissions for the specified assignments
mod_assign_get_grades	mod_assign_external	mod/assign/externallib.php	List of assignment ids (required), since (optional)	read	http://tracker.moodle.org/browse/MDL-31873 Return assignment grades (from the assign_grades table) for the specified assignments
core_grade_get_rubric_grading_definitions	core_grade_external	grade/externallib.php	List of grading area ids	read	http://tracker.moodle.org/browse/MDL-31681 This is equivalent to the old getMarkingRubrics method. It returns the details for the 'rubric' grading definitions from the mdl_grading_definitions table and the mdl_gradingform* tables

Migration of existing SOAP OKtech web services to Moodle core web services

These web services use the OKtech SOAP implementation. They are being replaced by the Moodle web services described in the previous section.

Operation name	Input message	Output message	Modifications for migration to Moodle core web services
login	username, password	the session key	No longer required. Moodle's token authentication will be used
logout	session key	true or false	No longer required. Moodle's token authentication will be used
getPublicKey	empty	the public key	No longer required. Moodle's token authentication will be used
getServiceVersion	empty	The Lightwork web services version number	We will use the Moodle version number instead. This is returned by calling service core_webservice_get_site_info
getCourses	the session key	An array of 0 or more course records. An array of 0 or more error messages.	We will no longer use capabilities. Instead we will use the teacher and non editing teacher roles which will be the marking manager and marker respectively
getCourseParticipants	the session key, an array of course ids, allStudents(boolean)	An array of 0 or more course participant records. An array of 0 or more error messages.	when allStudents is true, all course particpants students are returned, otherwise only students that have submitted work are included
getTeams	the session key, an array of assignment ids, a list of team ids, allStudents(boolean)	An array of 0 or more team records, an array of 0 or more error messages.	This web service is used when the team assignment type has been installed
getSubmissions	the session key, an array of activity ids, allStudents(boolean)	An array of 0 or more records from the assignment_submissions table, an array of 0 or more error messages.	When allStudents is set to true, draft submissions are not included
getRubrics	the session key, an array of activity ids, a timemodified value >=0	An array of 0 or more records from the lw_rubric table, an array of 0 or more error messages.	The timemodified parameter can be used to reduce the number of records returned when the client knows they have already received updates for all rubrics at a certain point in time.
saveRubrics	the session key, an array of rubric records	An array of saved rubric records, an array of 0 or more error messages.	Updates the lw_rubric table with new and updated rubrics
getFeedbackSubmissions	the session key, an activity id, a timemodified value >=0	An array of feedback submission records, an array of 0 or more error messages.	This web service is used when the feedback assignment type has been installed
getUserInfo	The session key, a multi-dimensional array with user ids associated with a timemodified value >=0, an assignment id	An array of user_info records, an array of 0 or more error messages.	Returns information from the user_info_data for a category in the user_info_category table
getModifiedMarkingCount	The session key, an array of activity ids, the marking type, a timemodified value >=0	The number of records in the lw_marking table modified after timemodified	Used to prevent making queries on the lw_marking table when there are no new records
getmarking	The session key, an activity id, the marking type, a timemodified value >=0, allstudents(boolean)	An array of records from the lw_marking table, an array of 0 or more error messages.	Returns records from lw_marking for the specified activity id that have been modified after the specified timemodified value
getMarkingHistory	The session key, the marking type, an array of unique marking identifiers	An array of records from the lw_marking_history table, an array of 0 or more error messages.
saveMarking	The session key, array of lw_marking records, the marking type, allstudents(boolean)	An array of unique marking identifiers, an array of 0 or more error messages.	Updates the lw_marking and lw_marking_history table with new and updated marking and marking history records
releaseMarking	The session key, array of lw_marking records, the marking type	An array of unique marking identifiers, an array of 0 or more error messages.	Releases the marking to the Moodle grade book. Also uploads marking feedback as attachments.
releaseTeamMarking	The session key, array of lw_marking records, the marking type	An array of unique marking identifiers, an array of 0 or more error messages.	This web service is used when the team assignment type has been installed
uploadAssignmentDocuments	The session key, array of assignment ids associated with a file reference	an array of 0 or more error messages	Uploads file attachments for the specified assignments. The file attachments can be any documents that have been annotated during marking and PDF rubrics for the students
downloadAssignmentDocumentsMetaData	The session key, course id, assignment id, includeannotatedfiles	An array of metadata about the files stored for the assignment, an array of 0 or more error messages.	This web service is used by the Lightwork client to decide whether to replace or upload its own local versions of the files
repairLightworkData	The session key, an array of assignment ids	An array of unique identifiers for records that were deleted, an array of 0 or more error messages.	Database records such as assignments and users may be deleted or updated in Moodle at any time. This web service can be called by the Lightwork client when it suspects that data has become invalid. The returned results are used to update the Lightwork client's data model. For example, if a teacher unenrolls from a course then they no longer have permissions to view this course and it must be removed from the Lightwork client's local database.
getAssignmentDocuments	The session key, course id, assignment id, an array of filenames to be downloaded	An array of file references, an array of 0 or more error messages	Downloads files as attachments which are accessed using the file references
getSubmissionReport	The session key, course id, start date, end date	An array of submission report records, an array of 0 or more error messages	Downloads information about all student submissions for a course between a specified time period. This information is used by the Lightwork client to produce a submission report.
getSubmissionFiles	The session key, the activity id, an array of submission ids	An array of assignment_submissions table records with file references, an array of 0 or more error messages	An array of records from the assignment_submissions table with references to a zip file attachment containing files submitted by the student

Capabilities and permissions

• moodle/lightwork:managelightworkmarkers - create and modify rubrics, allocate markers to students, release marks to the gradebook
• moodle/lightwork:marklightworksubmissions - Mark student submissions
• webservice:createtoken
• webservice/soap:use

Normally, capabilities would be set up as follows:

Teacher role

• moodle/lightwork:managelightworkmarkers
• moodle/lightwork:marklightworksubmissions
• webservice:createtoken
• webservice/soap:use

Non-editing teacher role

• moodle/lightwork:marklightworksubmissions
• webservice:createtoken
• webservice/soap:use

Administration and configuration

The change to use the core Lightwork web services will be transparent to Lightwork users. However, the Moodle administrator will need to configure the services as follows:

• Enable Web services in the Site Administration - Advanced features page
• Enable the SOAP protocol in the Site Administration - Plugins - Web services - Manage protocols page
• Add a new custom external service in the Site Administration - Plugins - Web services - External services page
  • Name - Lightwork
  • Enabled - Checked
  • Authorised users only - Unchecked
  • Required capability - No required capability
• Select Add Service to add the new Lightwork web service
• Select the Functions link of the new Lightwork web service
• Select Add Functions and add all the functions that are prefixed with lightwork_

Assuming that the correct capabilities have been setup for them as described in the capabilities section, Lightwork users will now be able to connect to Moodle.

Moodle APIs used by Lightwork in Moodle 2.1

• mod/assignment/lib.php (will be replaced with mod_assign in Moodle 2.3)
• mod/assignment/type/<assignment type>/assignment.class.php (will be replaced with mod_assign in Moodle 2.3)
• mod/resource/locallib.php
• lib/dml/moodle-database.php
• lib/accesslib.php
• lib/grade/
• lib/uploadlib.php
• lib/filebrowser/
• lib/filestorage/

User Interface - Create assignment, define Marking guide and mark student work according to Marking guide

Create and update assignment

Create and update assignment page

Advanced grading, Define Marking guide page

The marking manager selects Define Marking guide under Settings-Assignment administration-Advanced grading

Advanced grading, Define Marking guide page

Advanced grading page with active grading method Marking guide

The marking manager selects Advanced grading under the Settings-Assignment administration link

Advanced grading page with active grading method marking guide

Mark student using the Marking guide

The marker selects the student submission from the submissions page and chooses to start marking

Mark student using the Marking guide

Student views their feedback

The marking manager has reviewed and released the marks to the student. The student logs in and views their feedback.

Student views their feedback

User Interface - Marker allocation and management

Allocate markers to students

The teacher commences allocating markers to students

Allocate markers to students

Review the marking

This assumes that this assignment is being marked using the new "marking guide" advanced grading method

Review the marking

Release the marking

Release the marking

Marking guide - Screenshot and Testing

Call for testing

Please help us with testing the marking guide functionality.

Link to marking guide code: https://github.com/Lightwork-Marking/moodle-gradingform_guide

A test plan is available from: TestPlan_MarkingGuide

Testing results can be recorded in: Testing_Record_MarkingGuide