SCORM Test Harness
Project state	Planning
Tracker issue	MDL-26912
Discussion	Using Moodle: SCORM Test Harness- GSoC
Assignee	Mayank Gupta

Project specification to be updated soon...

Overview

Abstract

This document is the specification for SCORM Test Harness, a Google Summer Of Code (GSoC) 2011 project. The SCORM test harness will allow automatically running the SCORM ADL test on the latest Moodle Code.

The project mentor is Dan Marsden and the program administrator is Helen Foster.

Context

Currently running the SCORM ADL tests is a time consuming task and requires a specific environment to be set up. The tests can only run in Internet Explorer. This project aims at setting up an automated testing framework that will allow to automatically run the tests on a Microsoft Windows machine running Internet Explorer 8 or higher. The Moodle install need not be on the same Microsoft Windows machine and could be on a remote Linux machine too. The framework would also provide the ability to schedule or run the SCORM ADL tests automatically between the desired time intervals. On the completion of the tests an HTML page, containing the test's report will be displayed to the user.

The later part of the project itself involves identifying the issues with SCORM 2004 support, reporting them to the Tracker and writing patches for the same. This will help in the ongoing development of providing SCORM 2004 support in Moodle, without breaking compatibility with previous tests.

Project Deliverables

Script that downloads the latest Moodle Code from GitHub, ugrades the Moodle install (if necessary) and prepares it for executing the SCORM ADL tests.
Script that automates the ADL SCORM 1.2 tests and produces an HTML file with the results of the tests (pass/fail).
Script that automates some of the ADL SCORM 2004 tests on Moodle (it may not be possible to script all of the 2004 ADL tests due to existing failures).
Documentation for how to install the scripts on a Microsoft Windows machine and have them run on an automated cycle.
Identifying the issues with SCORM 2004 support in Moodle, reporting them to the Tracker and writing patches for the same.

Implementation Details

The following are the implementation details in breif:

Download and Update - The ADL SCORM tests should run on the latest Moodle code. The script using CLI will download the latest Moodle code from GitHub and upgrade the Moodle install (if necessary).
Add Test Course - Set up the users/course/activities required for the tests. This will be done by restoring the Moodle backup file into a new. This will again be implemented using CLI at the time of upgrading the install.
Execute Tests - Run the browser based test script. Selenium RC will be used to write the automated web tests.
Test Completion Report - At the end of the tests, the user will be displayed an HTML file with the results of the test (pass/fail).
Delete Test Course - After the tests have been executed, the new course that was added before the execution of test will be deleted.

Running ADL SCORM Test

The ADL SCORM tests can be executed in the following two ways -

Running from the Command Prompt or,
Running directly from Eclipse IDE (for Java Developers).

Environmental Setup

Install Java - If you do not have Java installed on your machine you need to download and install it. You can download it from here - Download Java.
Download Selenium - After ensuring that you have Java installed on your machine, you next need to download Selenium RC (or Remote WebDriver). To download Selenium RC (or Remote WebDriver) you can simply download the selenium-server-standalone jar file (selenium-server-standalone-x.x.x.jar) from here - Download Selenium. Save the jar file in the working directory.

In this documentation directory C:\ADLTests will be considered as the [working-directory] for the tests.

Install Junit - Finally, you need to download and install Junit on your machine. To install Junit download the zip file from here - Download Junit and extract the zip file in a directory say - C:\ADLTests\junit.

Eclipse IDE - To run the ADL SCORM tests using Eclipse IDE, you need to download Eclipse IDE for Java Developers from here - Download Eclipse.

Install ADL SCORM 1.2 Test Suite - Download and install the ADL SCORM 1.2 Conformance Test Suite. You can download the ADL Test suite from here - Download ADL 1.2 Test Suite.
Setup course - To test Moodle for ADL SCORM 1.2 Compliance you need to setup a course by following the instructions here - SCORM 1.2 compliance test instructions. The SCORM 1.2 LMSTestCourse packages that you need are available here -LMSTestCourse01.zip , LMSTestCourse02.zip.
ADL 1.2 Test Script - Download and the save the ADL 1.2 Test script as scormADL12.java in [working_directory]\ADL12\ . You can download the ADL 1.2 Test script from here - ADL 1.2 Test Script (Please note - this is a temporary URL for the test script.)

Executing the Tests

To execute the tests from the Command prompt follow the steps below -

Modify Test Script - The current version of the test script does not have support for command line arguments. Thus you need to specify the path to the ADL SCORM test suite and Moodle test site at line numbers - 26 and 27.
Compile Test Script - To compile the test script open the Command Prompt, change the directory to [working_directory]\ADL12 (you can also specify the full path to the scormADL12.java file from your current directory) and run the following command -

javac path\to\scormADL12.java

If you get the error - 'javac' is not recognized as an internal or external command, operable program or batch file. Then you need to specify the full path to the javac.exe executable inside the bin directory of the Java installation on your machine. Thus the above compilation command needs to be executed on the command prompt window by specifying the full path to javac like -

"C:\Program Files\Java\jdk1.6.0_24\bin\javac.exe" path\to\scormADL12.java

Set CLASSPATH - In order to execute the Selenium RC and Junit Tests you need to set the class path on the command prompt.

The elements that need to be added in classpath are - Junit class files, scormADL12 class files and Selenium jar class files. To set the CLASSPATH before executing the tests use the following command -

set CLASSPATH=[working_directory]\junit\junit.jar;[working_directory]\ADL12;[working-directory]\selenium-server-standalone.jar

For instance the actual (after replacing [working_directory])' command to set CLASSPATH will be like -

set CLASSPATH=C:\ADLTests\junit\junit4.9b2\junit-4.9b2.jar;C:\ADLTests\ADL12;C:\ADLTests\selenium-server-standalone-2.0.0.jar

Invoke the Runner - Finally, to run the tests you need to invoke the Junit runner with the command -

java org.junit.runner.JUnitCore scormADL12

Schedule

April 26th - May 29th: Get familiar with the Selenium RC, discuss functionality with mentor.
30th May - 5th June: Write script to Download lates code, Upgrade Moodle, Restore and Delete test course.
6th - 15th June: Write the automated web tests.
16th - 26th June: Reproduce ADL SCORM 1.2 Tests.
27th June - 6th July: Reproduce ADL SCORM 2004 Tests.
7th - 11 July: Check for regressions and fix issues related with it, if any.

This will be the midterm evaluation target - Script to Download, Upgrade, Restore / Delete Backup, code for Test harness, script for ADL SCORM 1.2 and 2004 tests and preliminary report on regressions issues.

12th July - 17th July: Check for regressions and fix issues related with it, if any.
18th July - 13th August: Test SCORM 2004 packages report bugs in the Tracker and write patches.
14th - 19th August: Documentation for installation and configuration of scripts on Windows machine.

would be a nice task: A developer documentation, to explain the the test harness.

Documentation

SCORM Test Harness

Contents