Acceptance testing for the mobile app

Revision as of 15:33, 22 November 2018 by sam marshall (talk | contribs) (Work in progress)

Jump to: navigation, search

This is only a proposal, MDL-63977, and is not yet available in Moodle. Please contact sam marshall if you have any queries and do not use this information for anything yet!

From Moodle 3.7 if this change is accepted it will be possible to write Behat tests for mobile app features.


There are now Behat tests for the mobile app. These are used for automated functionality testing of the mobile app.

By default, these do not run as part of a normal Behat run. This page tells you how to run the tests, and how to write them.

A key point is that these tests for some parts of the mobile app (and for its plugins) are included within the Moodle codebase, not within the app codebase, because they are run using the Moodle Behat infrastructure..

Running Behat tests for the mobile app

Set up a mobile app development environment

First you will need to set up a mobile app development environment.

Follow the first part of the instructions on this page:

You need to get as far as the part in section 5 where you open the app in the browser; this is what Behat will do. You don't need to complete the later steps.

  • You will need to update this environment periodically, for example when a new version of the mobile app is released. Behat does not do this for you.
  • If you need to run tests against multiple versions of the mobile app, you can do this by updating the code in this mobile app workspace and then running the Behat tests again, or by having multiple copies of the mobile workspace and changing which one Behat points to.

Add the mobile app Behat configuration

You need to add a line to your config.php to enable app testing. There are two ways to do this. First, you need to be aware of a couple of facts about the Ionic server used for app testing:

  • Depending on your computer, it may take about 3 minutes to start up.
  • The server uses about 1GB RAM.

Let Behat launch the app environment

If you want Behat to launch the app environment for you, then use this line in config.php:

$CFG->behat_approot = '/path/to/app/workspace/moodlemobile2';

When you do this, the Ionic server will be started automatically when Behat runs a test that uses the 'I enter the mobile app' step. It will be automatically terminated when the Behat test run finishes.

This is simple and convenient, but it is probably not a good approach for developers who frequently re-run a short Behat run (as you have to wait for it to start Ionic every time) or for complex systems that run Behat in parallel (as you may end up with multiple copies of Ionic eating up your RAM).

Manually launch the app environment yourself

The other option is to launch the Ionic server yourself (using ionic serve -b) before running any Behat tests, and also kill it yourself once you are finished with it. In this case, after launching it you will see output like:

[OK] Development server running!
     Local: http://localhost:8100

To use it in Behat, add this line to the Moodle config.php:

$CFG->behat_ionicaddress = 'http://localhost:8100';

The advantage of this approach is that you are in charge of bringing up and taking down the Ionic server, so you can do this efficiently, share a copy between parallel runs, etc. The disadvantage is that you do have to remember to do it; if the server isn't running, tests which use the app will fail.

Browser profiles

Mobile tests only run in Chrome, so you need to make sure you have a Chrome profile set up in your config.php Behat settings.

Behat will automatically run app tests (those with @app tag) only in a Chrome browser profile. So, if you run multiple browser tests, it won't waste time trying to run the app tests in each one.

Behat init

After you have set up the config.php, you will need to re-run Behat init:

php admin/tool/behat/cli/init.php

This is necessary because by default, Behat won't run app tests (those with @app tag) at all, since you didn't have it configured.

Running Behat

To run mobile tests in Behat, simply launch Behat in the usual way, but make sure you are using a Chrome profile. (Depending on your setup, this might mean using --profile=chrome.)

You can specify the scenarios to run as normal. The app tests all have the @app tag, so if you want to run all the mobile tests you can specify --tags=app, but you can also run any other set of scenarios. It is OK to combine app and normal tests in the same run.

Writing tests

Work in progress

I haven't finished writing this page yet