Acceptance testing/Browsers

This page complements Acceptance testing providing info about how to run the acceptance tests suite in different browsers.

Drivers

There are Selenium drivers (http://docs.seleniumhq.org/projects/webdriver/) to run acceptance tests in different browsers:

Firefox - https://code.google.com/p/selenium/wiki/FirefoxDriver
Chrome - https://code.google.com/p/selenium/wiki/ChromeDriver
Safari - https://code.google.com/p/selenium/wiki/SafariDriver
Internet Explorer - https://code.google.com/p/selenium/wiki/InternetExplorerDriver
PhantomJS (Webkit) - http://phantomjs.org/
IPhone - https://code.google.com/p/selenium/wiki/IPhoneDriver

Each driver should be downloaded and Selenium .jar should be started specifying the path to the driver; depending on the driver there could be other requirements.

PhantomJS

PhantomJS is different as it is a headless browser as it is quite faster than other drivers, it doesn't need a GUI to run and can execute JS, it doesn't even need to be used through Selenium (you can do it though) and you can do it

Download PhantomJS: http://phantomjs.org/download.html
/path/to/your/phantomjs/bin/phantomjs --webdriver=4444

Note that 4444 is the default port used by Selenium so you must specify another one if you want to run them together and specify the port in $CFG->behat_config.

Examples

 # Selenium in Linux (firefox by default + chrome)
 java -jar /opt/selenium-server-standalone.jar -Dwebdriver.chrome.driver=/opt/chromedriver

 # Selenium in OSx (firefox & safari by default + chrome)
 java -jar /Users/moodle/Downloads/selenium-server-standalone.jar -Dwebdriver.chrome.driver=/Users/moodle/Downloads/chromedriver

 # Selenium in Windows (started using git bash) (firefox by default + chrome + internet explorer)
 java -jar /c/seleniumdrivers/selenium-server-standalone.jar -Dwebdriver.chrome.driver=/c/seleniumdrivers/chromedriver.exe -Dwebdriver.ie.driver=/c/seleniumdrivers/IEDriverServer.exe

 # PhantomJS
 /path/to/your/phantomjs/bin/phantomjs --webdriver=4444

Compatibility

Not all the drivers can execute all of Moodle's step definitions; we tagged the step definitions that are using feature not supported by all browsers or that should be considered:

	File uploads (@_file_upload)	Browser dialogs (@_alert)	Switch window (@_switch_window)	Switch frame (@_switch_iframe)
Firefox	Yes	Yes	Yes	Yes
Chrome	Yes	Yes	No	Yes
Internet Explorer	Yes	Yes	No	Yes
Safari	Yes	No	No	Yes
PhantomJS	No	Yes	No	Yes

Moodle configuration

You need to tell behat what browser do you want to run and where it is. The best option could be to use behat profiles (http://docs.behat.org/guides/7.config.html#profiles).

The profiles can be specified in config.php through $CFG->behat_config like all other behat.yml specific configurations.

This is an example of $CFG->behat_config to run behat in multiple environments (Windows, Linux and OSx) to cover all the supported browsers. Note that we are using multiple servers so we skip the tests that involves files uploads when using a remote server.

   // Note that in this case $linuxip is the local server.
   // Set servers IPs.
   $linuxip = '127.0.0.1';
   $windowsip = '192.168.xxx.xxx';
   $osxip = '192.168.xxx.xxx';
   $CFG->behat_config = array(
       'phantomjs-linux' => array(
           'filters' => array(
               'tags' => '~@_switch_window&&~@_only_local'
           ),
          'extensions' => array(
              'Behat\MinkExtension\Extension' => array(
                  'selenium2' => array(
                      'wd_host' => 'http://' . $linuxip . ':4444/wd/hub'
                  )
              )
          )
       ),
       'chrome-linux' => array(
           'filters' => array(
               'tags' => '~@_switch_window'
           ),
           'extensions' => array(
               'Behat\MinkExtension\Extension' => array(
                   'selenium2' => array(
                       'browser' => 'chrome',
                       'wd_host' => 'http://' . $linuxip . ':4444/wd/hub'
                   )
               )
           )
       ),
       'chrome-osx' => array(
           'filters' => array(
               'tags' => '~@_switch_window&&~@_only_local'
           ),
           'extensions' => array(
               'Behat\MinkExtension\Extension' => array(
                   'selenium2' => array(
                       'browser' => 'chrome',
                       'wd_host' => 'http://' . $osxip . ':4444/wd/hub'
                   )
               )
           )
       ),
       'chrome-win' => array(
           'filters' => array(
               'tags' => '~@_switch_window&&~@_only_local'
           ),
           'extensions' => array(
               'Behat\MinkExtension\Extension' => array(
                   'selenium2' => array(
                       'browser' => 'chrome',
                       'wd_host' => 'http://' . $windowsip . ':4444/wd/hub'
                   )
               )
           )
       ),
       'firefox-linux' => array(
           'extensions' => array(
               'Behat\MinkExtension\Extension' => array(
                   'selenium2' => array(
                       'browser' => 'firefox',
                       'wd_host' => 'http://' . $linuxip . ':4444/wd/hub'
                   )
               )
           )
       ),
       'firefox-osx' => array(
           'filters' => array(
               'tags' => '~@_only_local'
           ),
           'extensions' => array(
               'Behat\MinkExtension\Extension' => array(
                   'selenium2' => array(
                       'browser' => 'firefox',
                       'wd_host' => 'http://' . $osxip . ':4444/wd/hub'
                   )
               )
           )
       ),
       'firefox-win' => array(
           'filters' => array(
               'tags' => '~@_only_local'
           ),
           'extensions' => array(
               'Behat\MinkExtension\Extension' => array(
                   'selenium2' => array(
                       'browser' => 'firefox',
                       'wd_host' => 'http://' . $windowsip . ':4444/wd/hub'
                   )
               )
           )
       ),
       'safari-osx' => array(
           'filters' => array(
               'tags' => '~@_alerts&&~@_switch_window&&~@_only_local'
           ),
           'extensions' => array(
               'Behat\MinkExtension\Extension' => array(
                   'selenium2' => array(
                       'browser' => 'safari',
                       'wd_host' => 'http://' . $osxip . ':4444/wd/hub'
                   )
               )
           )
       ),
       'ie-win' => array(
           'filters' => array(
               'tags' => '~@_switch_window&&~@_only_local'
           ),
           'extensions' => array(
               'Behat\MinkExtension\Extension' => array(
                   'selenium2' => array(
                       'browser' => 'iexplore',
                       'wd_host' => 'http://' . $windowsip . ':4444/wd/hub'
                   )
               )
           )
       )
   );

Documentation