SCORM Test Harness

Revision as of 20:23, 10 March 2012 by Mayank Gupta (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
SCORM Test Harness
Project state Completed
Tracker issue MDL-26912
Discussion Using Moodle: SCORM Test Harness- GSoC
Assignee Mayank Gupta


Overview

Abstract

This document is the specification for SCORM Test Harness, a Google Summer Of Code (GSoC) 2011 project. The SCORM test harness will allow automatically running the SCORM ADL test on the latest Moodle Code.

The project mentor is Dan Marsden and the program administrator is Helen Foster.

Context

Currently running the SCORM ADL tests is a time consuming task and requires a specific environment to be set up. The tests can only run in Internet Explorer. This project aims at setting up an automated testing framework that will allow to automatically run the tests on a Microsoft Windows machine running Internet Explorer 8 or higher. The Moodle install need not be on the same Microsoft Windows machine and could be on a remote Linux machine too. The framework would also provide the ability to schedule or run the SCORM ADL tests automatically between the desired time intervals. On the completion of the tests an HTML page, containing the test's report will be displayed to the user.

The later part of the project itself involves identifying the issues with SCORM 2004 support, reporting them to the Tracker and writing patches for the same. This will help in the ongoing development of providing SCORM 2004 support in Moodle, without breaking compatibility with previous tests.

Project Deliverables

  • Script that downloads the latest Moodle Code from GitHub, ugrades the Moodle install (if necessary) and prepares it for executing the SCORM ADL tests.
  • Script that automates the ADL SCORM 1.2 tests and produces an HTML file with the results of the tests (pass/fail).
  • Script that automates some of the ADL SCORM 2004 tests on Moodle (it may not be possible to script all of the 2004 ADL tests due to existing failures).
  • Documentation for how to install the scripts on a Microsoft Windows machine and have them run on an automated cycle.
  • Identifying the issues with SCORM 2004 support in Moodle, reporting them to the Tracker and writing patches for the same.

Implementation Details

The following are the implementation details in breif:

  • Download and Update - The ADL SCORM tests should run on the latest Moodle code. The script using CLI will download the latest Moodle code from GitHub and upgrade the Moodle install (if necessary).
  • Add Test Course - Set up the users/course/activities required for the tests. This will be done by restoring the Moodle backup file into a new. This will again be implemented using CLI at the time of upgrading the install.
  • Execute Tests - Run the browser based test script. Selenium RC will be used to write the automated web tests.
  • Test Completion Report - At the end of the tests, the user will be displayed an HTML file with the results of the test (pass/fail).
  • Delete Test Course - After the tests have been executed, the new course that was added before the execution of test will be deleted.

Running ADL SCORM Test

The ADL SCORM tests can be executed in the following two ways -

  • Running from the Command Prompt or,
  • Running directly from Eclipse IDE (for Java Developers).

Environmental Setup

  • Install Java - If you do not have Java installed on your machine you need to download and install it. You can download it from here - Download Java.
  • Download Selenium - After ensuring that you have Java installed on your machine, you next need to download Selenium RC (or Remote WebDriver). To download Selenium RC (or Remote WebDriver) you can simply download the selenium-server-standalone jar file (selenium-server-standalone-x.x.x.jar) from here - Download Selenium. Save the jar file in the working directory.

In this documentation directory C:\ADLTests will be considered as the [working-directory] for the tests.

  • Install Junit - You also need to download and install Junit on your machine. To install Junit download the zip file from here - Download Junit and extract the zip file in a directory say - C:\ADLTests\junit.

Note - The SCORM Test Harness has been tested with Selenium 2.3.0 and Junit 4.9b3 (the latest versions available while writing this documentation).

  • SCORM Test Harness Repository' - Finally, you need to download the Moodle SCORM Test Harness Repository from GitHub and save it in [working_directory]. You can download the repository from here - Moodle SCORM Test Harness Repository.

Note - If you do not wish to download/clone the complete repository, you can download the individual files required and create necessary folders as indicated in the steps below.


  • Eclipse IDE - To run the ADL SCORM tests using Eclipse IDE, you need to download Eclipse IDE for Java Developers from here - Download Eclipse.

ADL 1.2 Tests

  • Install ADL SCORM 1.2 Test Suite - Download and install the ADL SCORM 1.2 Conformance Test Suite. You can download the ADL Test suite from here - Download ADL 1.2 Test Suite.
  • Setup course - To test Moodle for ADL SCORM 1.2 compliance you need to setup a course by following the instructions here - SCORM 1.2 compliance test instructions. The SCORM 1.2 LMSTestCourse packages that you need are available here -LMSTestCourse01.zip , LMSTestCourse02.zip.
  • ADL 1.2 Test Script - If you have not downloaded/cloned the full repository then download/copy the ADL 1.2 Test Script class files from ADL 1.2 class files and save them in the folder [working_directory]\ADL1.2\classFiles\ ADL 1.2 classFiles].

ADL 2004 Tests

  • Install ADL SCORM 2004 Test Suite - Download and install the ADL SCORM 2004 Conformance Test Suite. Download the ADL 2004 Test Suite from here - [
  • Setup course - To test Moodle for ADL SCORM 2004 compliance you need to setup a course in the following manner -
    • Course Name - Setup a course named as - ADL SCORM Test 2004. The student, whose login details you will provide later on should be able to enter this course and should be able to access all SCORM activity within this course.
    • SCORM Package - Turn editing on and add SCORM Test packages as activity inside the course. You may add all the SCORM Test packages available in the Moodle SCORM Test Harness repository here - ADL SCORM 2004 Test Packages. You must ensure that you have uploaded all the Test Packages for all the ADL 2004 tests that you want to execute. While Adding a new SCORM package the name field should contain the test package name at the end of the package name that you enter. For instance - LMSTestPackage_API.zip when uploaded should have API at the end of the name, like - ADL SCORM 2004 API will be a valid name for API Test Package.
  • ADL 1.2 Test Script - If you have not downloaded/cloned the full repository then, you need to download/copy the ADL 1.2 Test Script class files from ADL 1.2 class files and save them in the folder [working_directory]\ADL2004\classFiles\ ADL 2004 classFiles]

Executing the Tests

To execute the tests from the Command prompt follow the steps below -

Set CLASSPATH

  • Set CLASSPATH - In order to execute the Selenium RC and Junit Tests you need to set the class path on the command prompt.

The elements that need to be added in classpath are - Junit class files, scormADL12 class files and Selenium jar class files. To set the CLASSPATH before executing the tests use the following command -

set CLASSPATH=[working_directory]\junit\junit.jar;[working_directory]\ADL1.2;[working-directory]\selenium-server-standalone.jar

For instance the actual (after replacing [working_directory]) command to set CLASSPATH will be like -

ADL 1.2 Tests

set CLASSPATH=C:\ADLTests\junit\junit4.9b3\junit-4.9b3.jar;C:\ADLTests\ADL___x\classFiles;
C:\ADLTests\selenium-server-standalone-2.3.0.jar

ADL 2004 Tests

set CLASSPATH=C:\ADLTests\junit\junit4.9b3\junit-4.9b3.jar;C:\ADLTests\ADL2004\classFiles;
C:\ADLTests\selenium-server-standalone-2.3.0.jar

Invoke Runner

  • Invoke the Runner - Finally, to run the tests you need to invoke the Junit runner and specify the Moodle Test Site URL along with the full URL of the "Learning Management System (LMS) Run-Time Environment Conformance Test" of SCORM ADL 1.2 or SCORM 2004 Test Suite. For ADL 2004 Test you can also specify which tests to execute.

ADL 1.2 Tests

To execute the ADL 1.2 tests you need to specify the Moode Test Site and "Learning Management System (LMS) Run-Time Environment Conformance Test" of SCORM ADL 1.2 URLs as system property and invoke the Junit runner. Execute the following command -

java -DmoodleURL="http://moodletestsite.org" -DtestSuiteURL="http://fulltestsuiteurl" org.junit.runner.JUnitCore scormADL12 

The full URL to "Learning Management System (LMS) Run-Time Environment Conformance Test" of SCORM ADL 1.2 test suite should be of LMSRTEMain.htm page, eg - http://localhost/Web/ADL/TestSuite1_2_7/TestSuite/lmsrte/LMSRTEMain.htm

ADL 2004 Tests

To execute the ADL 2004 tests you need to specify the Moode Test Site URL, Moodle site student username and password, "Learning Management System (LMS) Run-Time Environment Conformance Test" of SCORM ADL 1.2 URL as system property and invoke the Junit runner. If you do not provide username, password combination the test script will use default username - joestudent and password - password

  • Execute all tests - To execute all the tests you simply need to execute the following command -
java -DmoodleURL="http://moodletestsite.org" -Dusername="studentUsername" -Dpassword="password" 
-DtestSuiteURL="http://fulltestsuiteurl" org.junit.runner.JUnitCore scormADL2004
  • Specify Tests - To execute specific ADL SCORM 2004 tests you can use the following two ways to specify which tests to execute -
    • test system property - You can specify the tests you want to execute by using -Dtest="testName1,testName2,.."
java -DmoodleURL="http://moodletestsite.org" -Dusername="studentUsername" -Dpassword="password" 
-DtestSuiteURL="http://fulltestsuiteurl" -Dtest="API, CM-01,CM-02a,CO-01" org.junit.runner.JUnitCore scormADL2004 

Note - you can specify as many tests as you wish to run with -Dtest, but it is recommended to enter the test names in a file and use -DfilePath instead.

    • Specify filePath - You can specify the tests you want to execute in a in a file and feed the test script with the file path with -Dfilepath="path\to\file" . This file should contain each test name in a new line.

To execute the test script with the filePath execute the following command -

java -DmoodleURL="http://moodletestsite.org" -Dusername="studentUsername" -Dpassword="password" 
-DtestSuiteURL="http://fulltestsuiteurl" -DfilePath="path\to\file" org.junit.runner.JUnitCore scormADL2004 

Compile from Source

Optionally, you can compile the test scripts to generate the class files (if you do not wish to use the supplied class files)

  • Compile Test Script - To compile the test script open the Command Prompt, change the directory to [working_directory]\ADL1.2 or [working_directory]\ADL2004 (you can also specify the full path to the scormADL12.java or scormADL2004.java file from your current directory) and execute the following command -

ADL 1.2 Tests

javac [working_directory]\ADL1.2\src\scormADL12.java

ADL 2004 Tests

javac [working_directory]\ADL1.2\src\scormADL12.java

If you get the error - 'javac' is not recognized as an internal or external command, operable program or batch file. Then you need to specify the full path to the javac.exe executable inside the bin directory of the Java installation on your machine. Thus the above compilation command needs to be executed on the command prompt window by specifying the full path to javac like -

"C:\Program Files\Java\jdk1.6.0_24\bin\javac.exe" [working_directory]\ADL1.2\src\scormADL12.java

Further Reading

For more information please visit the following links: