Latest revision as of 06:01, 12 November 2008

This article is a stub. Please help improve Moodle Docs by editing it and adding some content.

Preface

Introduction

Moodle team uses Atlassian Jira as a Bug Tracker system. With a bit of tweaking Jira can become a Test Case Management software. For Moodle 2.0, we improved our QA department. We've been through testing a range of test management software that we could integrate to Jira. We tried more deeply Test Link, an open source software. It gave a us a good understanding of the basic Test Case Management software functionalities. Finally we decide to setup Jira for QA testing purpose. The main reason is that people working on Moodle are well familiar with Jira interface, and it will be easy for them to learn how to create, run test case, or generate report.

Jira and Test Case Management

Jira as a Test Case Management is attractive since it:

Uses an existing and known tool
Provides an easy way to link bugs to test cases and results
Provides decent reporting of the state of testing
Supports multiple releases

Scope of this document

This document describes how to install and setup Jira as a Test Case Management software.

Installing Jira (Linux)

Download the enterprise edition zip file on the Jira website. Extract the the files. Run jira-enterprise/bin/startup.sh. You should be able to connect to http://localhost:8080

Setup the entire Jira QA environment

Create a project

Create the QA environment

Create new Issue Type

Test Plan

Test Case

New custom fields

...
...

QA screen

QA screen scheme

Issue type screen scheme

Field configuration

Associate new Field configuration scheme to project

Summary:
Description:
Build Run Against: Moodle Version List
Component:
Assignee:

Screen

Screen scheme

Issue Type Screen Scheme

Field Configuration Scheme (associate Test Case Execution to TC field conf)

View Field Configuration

We need to finalize which field are going to be displayed on the which screens.

Description: both
Summary: both
Affects Version: TCE
Date Last Run: TCE
Build Run Against: TCE
Component: both
Assignee: both

Status and Workflow

Status

Create three new status:

Not Run (transition to Passed/Failed)
Passed (transition to Not Run)
Failed (transition to Not Run)
Obsolete/Inactive (transition to Not Run)

Workflow

Workflow Scheme

Associate project to Workflow

Add Timecharts plugin

Download the Charting plugin http://confluence.atlassian.com/display/JIRAEXT/JIRA+Charting+Plugin
Download the Timechart plugin http://confluence.atlassian.com/display/JIRAEXT/Timecharts
Install the two file jar files into WEB-INF/lib
Run shutdown.sh and startup.sh

Running a QA test cycle

Write Test Case

Create a task: Test Case

This will be created probably only once. It has to been written when you decide that your project get a new functionality. The description contains

Create a subtask: Test Case Execution

This will be created everytime you need to run a new bunch of test case. There should be a unique subtask by Affected Version, so we can generate correct report. The description of the subtask describes Steps and Expected Results. Note that these steps and expected results can be different in each substaks of a test case, if the affected versions should behave differently.

Run Test Case

Run a test

You should follow the steps from the subtask which is assigned to you.

Report a bug

If you encounter a bug, create a new issue and link the bug to this test case execution. Mark this test case execution as failed.

Read Report

Generate a report

When a QA report need to be created for the current state of a release, we search for all TestCase issues with the Affects Version field set to the required release. The resulting set of issues can then be sorted and counted by number passed, failed, not run etc. Producing historical reports can be done with the Timecharts plugin for Jira, which shows a graph of how the results in a report change over time.

Interpret the report

We created a new link type to connect test cases to the bugs that they created or are verifying. This is the biggest advantage that I see of using an issue tracker to track test cases and bugs together. TBD: we could also add a box to all bugs to indicate whether a Test Case is expected by someone for a bug.

@@ Line 1: / Line 1: @@
+{{stub}}[[Category:Quality Assurance]]
 =Preface=
 ==Introduction ==
-Moodle team uses Atlassian Jira as a Bug Tracker system. With a bit of tweaking Jira can become a Test Case Management software. For Moodle 2.0, we improved our QA department. We've been through testing a range of test management software that we could integrate to Jira. We try more deeply Test Link, an open source software. Finally we decide to setup Jira for QA testing purpose. The main reason is that people working on Moodle are well familiar with Jira interface, and it will be easy for them to learn how to create, run and read test case, or generate report.
+Moodle team uses Atlassian Jira as a Bug Tracker system. With a bit of tweaking Jira can become a Test Case Management software. For Moodle 2.0, we improved our QA department. We've been through testing a range of test management software that we could integrate to Jira. We tried more deeply Test Link, an open source software. It gave a us a good understanding of the basic Test Case Management software functionalities. Finally we decide to setup Jira for QA testing purpose. The main reason is that people working on Moodle are well familiar with Jira interface, and it will be easy for them to learn how to create, run test case, or generate report.
-Jira as a Test Case Management is attractive, it:
+==Jira and Test Case Management==
+Jira as a Test Case Management is attractive since it:
 #Uses an existing and known tool
 #Provides an easy way to link bugs to test cases and results
@@ Line 13: / Line 15: @@
 =Installing Jira (Linux)=
-Download the zip file
+Download the enterprise edition zip file on the Jira website. Extract the the files. Run jira-enterprise/bin/startup.sh. You should be able to connect to http://localhost:8080
 =Setup the entire Jira QA environment=
 ==Create a project==
 ==Create the QA environment==
-===TestCase Issue Type===
+===Create new Issue Type===
+====Test Plan====
+====Test Case====
 ===New custom fields===
-===Test case screen===
+* ...
-===Test case screen scheme===
+* ...
+===QA screen===
+===QA screen scheme===
 ===Issue type screen scheme===
-===?? Field configuration ??===
+===Field configuration===
-===?? Field configuration sheme ??===
+===Associate new Field configuration scheme to project===
+* Summary:
+* Description:
+* Build Run Against: Moodle Version List
+* Component:
+* Assignee:
+====Screen====
+====Screen scheme====
+====Issue Type Screen Scheme====
+====Field Configuration Scheme (associate Test Case Execution to TC field conf)====
-===Create sub-task===
+===View Field Configuration===
-===??????===
+We need to finalize which field are going to be displayed on the which screens.
+* Description: both
+* Summary: both
+* Affects Version: TCE
+* Date Last Run: TCE
+* Build Run Against: TCE
+* Component: both
+* Assignee: both
-===?? Intall report tool as Timecharts plugin for Jira ??===
+===Status and Workflow===
+====Status====
+Create three new status:
+* Not Run (transition to Passed/Failed)
+* Passed (transition to Not Run)
+* Failed (transition to Not Run)
+* Obsolete/Inactive (transition to Not Run)
+====Workflow====
+====Workflow Scheme====
+====Associate project to Workflow====
+=== Add Timecharts plugin ===
+# Download the Charting plugin http://confluence.atlassian.com/display/JIRAEXT/JIRA+Charting+Plugin
+# Download the Timechart plugin http://confluence.atlassian.com/display/JIRAEXT/Timecharts
+# Install the two file jar files into WEB-INF/lib
+# Run shutdown.sh and startup.sh
 =Running a QA test cycle=
 ===Write Test Case===
+==== Create a task: Test Case====
+This will be created probably only once. It has to been written when you decide that your project get a new functionality.
+The description contains
+==== Create a subtask: Test Case Execution====
+This will be created everytime you need to run a new bunch of test case. There should be a unique subtask by Affected Version, so we can generate correct report. The description of the subtask describes Steps and Expected Results. Note that these steps and expected results can be different in each substaks of a test case, if the affected versions should behave differently.
 ===Run Test Case===
-====Run the test====
+====Run a test====
+You should follow the steps from the subtask which is assigned to you.
 ====Report a bug====
+If you encounter a bug, create a new issue and link the bug to this test case execution. Mark this test case execution as failed.
 ===Read Report===
 ====Generate a report====
+When a QA report need to be created for the current state of a release, we search for all TestCase issues with the Affects Version field set to the required release. The resulting set of issues can then be sorted and counted by number passed, failed, not run etc. Producing historical reports can be done with the Timecharts plugin for Jira, which shows a graph of how the results in a report change over time.
 ====Interpret the report====
+We created a new link type to connect test cases to the bugs that they created or are verifying. This is the biggest advantage that I see of using an issue tracker to track test cases and bugs together. TBD: we could also add a box to all bugs to indicate whether a Test Case is expected by someone for a bug.