Behat – part one

Sunday, 16 May 2021

Running, let alone writing the ‘ Moodle acceptance tests ’ is a bit tricky and complicated, and they can take ages. But what if there was a way to speed things up? In this first of a two part post, we’ll look at getting up and running with running tests one at a time. In part two, we’ll look at running tests in parallel and cover any questions raised in the comments of this part.

Disclaimers

Firefox® is a registered trademark of the Mozilla Foundation - www.mozilla.org/en-US/foundation/trademarks/policy . Ubuntu® is a registered trademark of Canonical Ltd – ubuntu.com/legal/intellectual-property-policy . Moodle™ is a registered trademark of ‘Martin Dougiamas’ – moodle.com/trademarks . Other names / logos can be trademarks of their respective owners. Please review their website for details. I am independent from the organisations listed and am in no way writing for or endorsed by them. The information presented in this article is written according to my own understanding, there could be technical inaccuracies, so please do undertake your own research.

References

The thought

When I started to run the acceptance tests, I noticed that the hard disk was doing a lot of activity, the screen was flashing away as the ‘browser’ was being controlled automatically. So as the database does not need to be kept then why not optimise it to be faster by using a ‘ Ramdisk ’ instead? And why use a GUI installation? So run the browser ‘ headless ’ too.

Starting position

The technical detail in this post begins at the point where you have a working 64bit Linux (in my case Ubuntu) server running an installed LAMP and Moodle installation (Moodle 3.10). This is covered in places such as on the ‘ MoodleBites Server Administrator course ’ and docs.moodle.org/310/en/Step-by-step_Installation_Guide_for_Ubuntu .

My install is running in ‘/var/www/moodle’ and ‘/var/moodledata’ on Ubuntu 20.04 server with 4GB of ram, no GUI as a virtual machine, allocated two of the CPU’s cores with no execution cap. For the purpose of screenshots I’m running a second Ubuntu virtual machine with a GUI. Both of them on ‘ VirtualBox ’ with the network adapter set as ‘Bridged’ (to the WiFi adaptor) so that the IP addresses are allocated by my router, and thus it knows and can send data between the real and virtual machines.

The actual machine I’m using has an Intel core i5 9th gen, i5-9500 CPU @ 3.00GHz with 16GB of ram.

So from this point onwards ‘http://REALIPOFMACHINE/moodle’ works with having gone through the Moodle install process and created the ‘config.php’ file successfully. The database is called ‘moodle’, with an username of ‘moodle’ and a database password of ‘yourdbpass’.

Getting Behat up and initialised

This is my interpretation of the Moodle instructions ( docs.moodle.org/dev/Running_acceptance_test ):

In your home folder create a ‘behat’ folder and change directory into it.

Get and install Chrome:

1wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
2sudo apt install ./google-chrome-stable_current_amd64.deb

Get the Linux Chrome driver from sites.google.com/a/chromium.org/chromedriver (Now sites.google.com/chromium.org/driver ) then download, extract the zip and copy ‘chromedriver’ via SSH to server in the ‘behat’ folder, then put chromedriver (with sudo) into /usr/local/bin:

1sudo cp chromedriver /usr/local/bin/

Then make sure go has +rx and u has +rwx - i.e. owned by root with rwxr-xr-x permissions:

1sudo chmod 0755 /usr/local/bin/chromedriver

Get Selenium server with Java:

1sudo apt install default-jre

Go to ‘ www.seleniumhq.org/download ’ and get the stand-alone server, i.e. find the link and in the ‘behat’ folder:

1wget https://selenium-release.storage.googleapis.com/3.141/selenium-server-standalone-3.141.59.jar

As we are running headless browser, we need ‘ xvfb ’:

1sudo apt install xvfb

Now we are ready to configure Moodle to use what we’ve installed, so:

1sudo mkdir /var/behatdata
2sudo chown -R www-data:www-data /var/behatdata

Add in Moodle’s config.php file, just before $CFG->directorypermissions:

1sudo -u www-data nano /var/www/moodle/config.php

 1$CFG->behat_wwwroot = 'http://localhost/moodle';
 2$CFG->behat_prefix = 'bht_';
 3$CFG->behat_dataroot = '/var/behatdata';
 4
 5$CFG->behat_profiles = [
 6    'default' => [
 7        'browser' => 'chrome',
 8        'extensions' => [
 9            'Behat\MinkExtension' => [
10                'selenium2' => [
11                    'browser' => 'chrome',
12                    'wd_host' => 'http://localhost:4444/wd/hub'
13                ]
14            ]
15        ]
16    ]
17];

This tells Moodle about the Behat data folder, the ‘different’ localhost wwwroot and what url Selenium will be running under (it drives the Chrome browser).

Now we can initialise the tests, so in ‘/var/www/moodle’:

1sudo -u www-data php admin/tool/behat/cli/init.php

Once completed then we can login to Moodle and navigate to ‘Site Administration > Development > Acceptance Testing’, where we should see the step definitions (this helps with writing tests):

If they are not listed then you’ll get an informational error message instead and it means that the initialisation has failed.

Running tests

Now that everything is in place, we can run some tests, however the whole lot takes ages, so we’ll only run the topics course format tests ‘/course/format/topics/tests/behat’. For this we need two terminals, one for Selenium and the other for running the tests themselves:

In the ‘behat’ folder:

1xvfb-run java -jar selenium-server-standalone-3.141.59.jar -port 4444

In /var/www/moodle:

1sudo -u www-data vendor/bin/behat --config /var/behatdata/behatrun/behat/behat.yml --format=pretty --out=report_behat.txt --tags format_topics

When when ‘behat’ finishes you can see the time it took with:

1tail report_behat.txt

Ramdisk configuration

I found it a bit tricky to change where the installed MySQL server was looking for its database, so instead after reading some articles, decided to use ‘ Docker ’ to run a MySQL instance as that allowed me to easily specify it on the command line. So first we need to install and configure Docker:

1sudo apt install apt-transport-https ca-certificates curl software-properties-common
2curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Find release of Ubuntu name from ‘/etc/apt/sources.list’ and use that with:

1sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
2sudo apt update
3apt-cache policy docker-ce
4sudo apt install docker-ce
5sudo systemctl status docker

To save typing ‘sudo’ every time we type ‘docker’:

1sudo usermod -aG docker ${USER}

Then we need to relogin to take effect, then we can get MySQL:

1docker pull mysql/mysql-server

and see it in the list of images:

1docker images

Now create ramdisk:

1sudo mkdir /mnt/ramdisk
2sudo mount -t tmpfs -o rw,size=2G tmpfs /mnt/ramdisk

and see it in the list of devices:

1df -h

listed at the bottom.

In a new terminal, run Docker MySQL with it:

1docker run --name=test-mysql -p 6603:3306 --env="MYSQL_ROOT_PASSWORD=yourdbpass" -v /mnt/ramdisk:/var/lib/mysql mysql/mysql-server

and once up and running we will need to use another terminal and run:

1docker inspect test-mysql

to get the ‘Gateway IP’ (in this case ‘172.17.0.1’) for ‘moodle’ user, then at each startup of the machine we will (after mounting the ramdisk), create the database for Moodle to use:

1docker exec -it test-mysql mysql -uroot -pyourdbpass

1CREATE DATABASE moodle DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
2CREATE USER 'moodle'@'172.17.0.1' IDENTIFIED BY 'yourdbpass';
3GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO 'moodle'@'172.17.0.1' WITH GRANT OPTION;
4exit

We now need to tell Moodle to use that database instead:

1sudo -u www-data nano /var/www/moodle/config.php

and comment out / add:

1//$CFG->dbhost    = 'localhost';
2$CFG->dbhost    = '172.17.0.1'; // Docker Gateway IP from 'docker inspect test-mysql'.
3
4$CFG->dboptions = array (
5  //'dbport' => '',
6  'dbport' => '6603', // Docker MySQL

Then check Moodle would install and connect to the database:

http://REALIPOFMACHINE/moodle

Running with Ramdisk

As before, we need to initialise the tests, so in ‘/var/www/moodle’:

1sudo -u www-data php admin/tool/behat/cli/init.php

then run them:

If Selenium is not already running, in the ‘behat’ folder:

1xvfb-run java -jar selenium-server-standalone-3.141.59.jar -port 4444

In /var/www/moodle:

1sudo -u www-data vendor/bin/behat --config /var/behatdata/behatrun/behat/behat.yml --format=pretty --out=report_behat.txt --tags format_topics

Again when ‘behat’ finishes you can see the run with:

1tail report_behat.txt

And hopefully you should be able to see a speed increase:

With before on the left and after on the right.

When you’ve finished, you can shutdown the machine, optionally stop and remove the Docker MySQL image beforehand:

1docker stop test-mysql
2docker rm test-mysql

Running again

After starting up the server, you’ll need to recreate the Ramdisk (the mount point already exists):

1sudo mount -t tmpfs -o rw,size=2G tmpfs /mnt/ramdisk
2
3docker start test-mysql

or if removed (with ‘docker rm test-mysql’):

1docker run --name=test-mysql -p 6603:3306 --env="MYSQL_ROOT_PASSWORD=yourdbpass" -v /mnt/ramdisk:/var/lib/mysql mysql/mysql-server

Then create the moodle user and DB:

1docker exec -it test-mysql mysql -uroot -pyourdbpass

1CREATE DATABASE moodle DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
2CREATE USER 'moodle'@'172.17.0.1' IDENTIFIED BY 'yourdbpass';
3GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO 'moodle'@'172.17.0.1' WITH GRANT OPTION;
4exit;

Then you’ll be able to initialise and run tests as before. Please refer to the Moodle documentation for this.

Troubleshooting

When switching between the local and Docker sites, then you might get in a situation where one of the sites is not accessible, in that case, just purge the caches on the command line:

1sudo php /var/www/moodle/admin/cli/purge_caches.php

Conclusion

My configuration is not the only one you can use, its just the one I’ve worked out for me. It might be possible to get this to work without Selenium ( docs.moodle.org/dev/Running_acceptance_test#Run_tests_directly_in_Chrome.2C_with_no_Selenium ), but I’ve not tried that yet.

I’m certainly not a Linux / Docker / Behat expert, but this seems to work for me!

In part ‘two’, we’ll look at running tests in parallel and cover any questions raised in this part.

What do you think please?

Theme selector block

What is a Byte anyway?