Load testing Moodle with JMeter: Difference between revisions

Latest revision as of 17:55, 7 January 2022

Being the original load testing tool for servers, introduced in 1998(!), JMeter [1] has always been the number one load testing resp. performance measuring tool for Moodle. Around its release 2.5, Moodle developers made the testing process vastly simpler by adding many scripts. The topic of this page is the usage of the two scripts Make test course [2] and Make JMeter test plan [3] and JMeter running on a client machine.

Warning: Do not run these scripts nor JMeter targeting a production system! They generate a large amounts of artificial data and load the server to its limit and beyond, making it to bloat and/or become non-responsive. Don't even run them on a separate Moodle instance in the production server: their effect on the DBMS is undocumented.

Prerequisite

JMeter installed on your client computer

JMeter is a Java application so it can run on Windows, Mac, or Linux. It can be run on the command line or on its graphical client on your desktop. First check if Java is installed in your client computer by opening a console and entering 'java -version'. If it doesn't return a version, first install Java on your client. The instructions vary with the operating system.

To install JMeter go to the JMeter page, download the zip and unzip it to the directory of your choice.

A test plan

You can make a test plan from the JMeter client in DIY style. We do not describe that path here, assuming you know JMeter. This page describes how to make a test plan with the scripts built in to Moodle and run it.

Make test course

See https://docs.moodle.org/en/Test_course_generator

Make JMeter test plan

See https://docs.moodle.org/en/JMeter_test_plan_generator

Running the test plan on JMeter

To run JMeter open the /bin subdirectory and use either jmeter.bat on Windows or ./jmeter on Linux. This will open the JMeter GUI.

First step is to go to File -> Open and select the testplan.jmx file you downloaded above. This will open a tree on the left hand side of JMeter. Open the Warm-up site -> Default site request and enter the protocol (http or https) and Server name or IP. Then go to CSV Users Data and locate your users.csv file under Filename. Repeat this for Moodle Test.

Right mouse click on the Test Plan, then Add -> Listener. Choose Aggregate Report so you can follow the run as it happens. Save the changes in your test plan and run it.

References

[1] Apache JMeter homepage
[2] Moodle Docs: Performance -> Obtain_a_baseline_benchmark
[3] Jmeter or loading script/app to simulate concurrent users taking quizzes (October 2018)
[4] Has anyone successfully run a jmeter test with 1000 users? (April 2018)

@@ Line 1: / Line 1: @@
-Being the original load testing tool for servers, introduced in 1998(!), JMeter [http://jmeter.apache.org/] has always been the number one load testing resp. performance measuring tool for Moodle. Around its release 2.5, Moodle developers made the testing process vastly simpler by adding many scripts. The topic of this page are the two scripts  ''Make test course''  and ''Make JMeter test plan'', both available under ''Site administration > Development''. Together with the set of add-ons [https://github.com/moodlehq/moodle-performance-comparison Moodle performance comparison] they comprise a full set of load testing tools.
+Being the original load testing tool for servers, introduced in 1998(!), JMeter [http://jmeter.apache.org/] has always been the number one load testing resp. performance measuring tool for Moodle. Around its release 2.5, Moodle developers made the testing process vastly simpler by adding many scripts. The topic of this page is the usage of the two scripts ''Make test course'' [https://docs.moodle.org/en/Test_course_generator] and ''Make JMeter test plan'' [https://docs.moodle.org/en/JMeter_test_plan_generator] and JMeter running on a client machine.
-Warning:  Do not run these scripts on a production system! These scripts generate large amounts of artificial data and load the server to its limit and beyond, making it to bloat and become non-responsive. Don't even run them on a separate Moodle instance in the production server: their effect on the DBMS is undocumented.
+Warning:  Do not run these scripts nor JMeter targeting a production system! They generate a large amounts of artificial data and load the server to its limit and beyond, making it to bloat and/or become non-responsive. Don't even run them on a separate Moodle instance in the production server: their effect on the DBMS is undocumented.
-<h2>Make test course</h2>
+<h2>Prerequisite</h2>
-(Moved to https://docs.moodle.org/39/en/Test_course_generator)
+<h3>JMeter installed on your client computer</h3>
+JMeter is a Java application so it can run on Windows, Mac, or Linux. It can be run on the command line or on its graphical client on your desktop. First check if Java is installed in your client computer by opening a console and entering 'java -version'. If it doesn't return a version, first install [https://www.java.com/ Java] on your client. The instructions vary with the operating system.
-<h2>Make JMeter test plan</h2>
+To install JMeter go to the [http://jmeter.apache.org/download_jmeter.cgi JMeter page], download the zip and unzip it to the directory of your choice.
-This script creates a JMeter test plan file along with the user credentials file. Here too it is necessary to set ''Site administration > Development > Debugging'' to DEVELOPER. Also note that, unlike making the test course, this is a single script. Therefore the PHP max_execution_time need to be at least as long as the time for the whole script, which could be anything from a minute to hours depending on the size of the course you select.
-This test plan is designed to work along with Moodle performance comparison [https://github.com/moodlehq/moodle-performance-comparison], which makes it easier to run the test plan in a specific Moodle environment, to gather information about the runs and to compare the results, so you will need to download it and use it's test_runner.sh script or follow the installation and usage instructions.
-You need to set the same password for all the test users by adding the line "$CFG->tool_generator_users_password = 'YOURSECRET';" to config.php of Moodle. You might want to set "$CFG->passwordpolicy = 0;" too, in case the proposed user password does not satisfy the password policy of the site. There is no default value for this password to prevent unintended usages of the tool. You need to use the update passwords option in case your course users have other passwords or they were generated by tool_generator but without setting a $CFG->tool_generator_users_password value.
-This script is a part of tool_generator so it works well with the courses generated by the courses and the site generators, it can also be used with any course that contains, at least:
-- Enough enrolled users (depends on the test plan size you select) with the $CFG->tool_generator_users_password password set
-- A page module instance
-- A forum module instance with at least one discussion and one reply
-You have to consider your server's capacity when running large test plans as the amount to load generated by JMeter can be specially big. The ramp-up period has been adjusted according to the number of threads (users) to reduce this kind of issues but the load is still huge.
-Visit  ''Site administration > Development > Make JMeter test plan''. Under Size of course your have six possibilities: XS (1 user, 5 loops and 1 ramp-up period), S (30 users, 5 loops and 6 ramp-up period), M (100 users, 5 loops and 40 ramp-up period), L (1000 users, 6 loops and 100 ramp-up period), XL (5000 users, 6 loops and 500 ramp-up period) and XXL (1000 users, 7 loops and 800 ramp-up period). As target course select one of the courses created above. Also ''Update course users password'' should be ticked.
-Consider that the server resources you will need to make bigger test plans will be accordingly higher.
+<h2>A test plan</h3>
+You can make a test plan from the JMeter client in DIY style. We do not describe that path here, assuming you know JMeter. This page describes how to make a test plan with the scripts built in to Moodle and run it.
-Once successful you can download a test plan [[File:testplan_timestamp.jmx|testplan_timestamp.jmx]] and a users file [[File:users_timestamp.csv|users_timestamp.csv]]
+<h3>Make test course</h3>
+See https://docs.moodle.org/en/Test_course_generator
-There is also a PHP CLI version of the script. Run it with --help to see its options:
+<h3>Make JMeter test plan</h3>
+See https://docs.moodle.org/en/JMeter_test_plan_generator
-  $ php admin/tool/generator/cli/maketestplan.php --help
-  Options:
-  -h, --help              Print out this help
-  --shortname            Short name of the test plan's target course (required)
-  --size                  Size of the test plan to create XS, S, M, L, XL, or XXL (required)
-  --bypasscheck          Bypasses the developer-mode check (be careful!)
-  --updateuserspassword  Updates the target course users password according to $CFG->tool_generator_users_password
-  * XS (1 Nutzter, 5 Schleifen und 1 Steigerungsrate)
-  * S (30 Nutzter, 5 Schleifen und 6 Steigerungsrate)
-  * M (100 Nutzter, 5 Schleifen und 40 Steigerungsrate)
-  * L (1000 Nutzter, 6 Schleifen und 100 Steigerungsrate)
-  * XL (5000 Nutzter, 6 Schleifen und 500 Steigerungsrate)
-  * XXL (10000 Nutzter, 7 Schleifen und 800 Steigerungsrate)
-Example:
-  $ sudo -u www-data /usr/bin/php admin/tool/generator/cli/maketestplan.php --shortname="testcourse_12" --size=S
 <h2>Running the test plan on JMeter</h2>
-JMeter is a Java application so it can run on Windows, Mac, or Linux. It can be run on the command line or on its graphical client on your desktop. First check if Java is installed in your client computer by opening a console and entering '
+To run JMeter open the /bin subdirectory and use either jmeter.bat on Windows or ./jmeter on Linux. This will open the JMeter GUI.
-java -version'. If it doesn't return a version, first install [https://www.java.com/ Java] on your client. The instructions vary with the operating system.
-To install JMeter go to the [http://jmeter.apache.org/download_jmeter.cgi JMeter page], download the zip and unzip it to the directory of your choice.
-To run JMeter. open the /bin subdirectory and use either jmeter.bat on Windows or ./jmeter on Linux. This will open the JMeter GUI.
-First step is to go to File -> Open and select the testplan.jmx file. This will open a tree on the left hand side of JMeter.
+First step is to go to File -> Open and select the testplan.jmx file you downloaded above. This will open a tree on the left hand side of JMeter. Open the Warm-up site -> Default site request and enter the protocol (http or https) and Server name or IP. Then go to CSV Users Data and locate your users.csv file under Filename. <b>Repeat </b> this for Moodle Test.
-Open the Warm-up site -> Default site request and enter the protocol (http or https) and Server name or IP. Then go to CSV Users Data and locate your users.csv file under Filename. <b>Repeat </b> this for Moodle Test.
-Right mouse click on the Test Plan, then Add -> Listener. I choose Aggregate Report so I can follow the run as it happens. Save the changes in your test plan and run it.
+Right mouse click on the Test Plan, then Add -> Listener. Choose Aggregate Report so you can follow the run as it happens. Save the changes in your test plan and run it.
 ==References==
 * [1] [http://jmeter.apache.org/ Apache JMeter homepage]
-* [2] [https://docs.moodle.org/en/Performance MoodleDocs Performance]
+* [2] [https://docs.moodle.org/en/Performance_recommendations#Obtain_a_baseline_benchmark Moodle Docs: Performance -> Obtain_a_baseline_benchmark]
-* [3] [https://moodle.org/mod/forum/discuss.php?d=377231 Jmeter or loading script/app to simulate concurrent users takng quizzes] (October 2018)
+* [3] [https://moodle.org/mod/forum/discuss.php?d=377231 Jmeter or loading script/app to simulate concurrent users taking quizzes] (October 2018)
 * [4]  [https://moodle.org/mod/forum/discuss.php?d=368620 Has anyone successfully run a jmeter test with 1000 users?] (April 2018)
 [[Category:Developer tools]]

Documentation