Skip to content

aboutRobotTests.md

Latest commit

 

History

History
169 lines (102 loc) · 8.61 KB

aboutRobotTests.md

File metadata and controls

169 lines (102 loc) · 8.61 KB

⬅️ Back to Main ReadMe File

Testing with Robot Framework

You can choose your preferred environment for robot testing. We recommend using Docker to avoid heavy multi-step installations on your local computer.

Reminder: You need to have the backend running for the robot tests to pass. Same with the github actions, the backend url should lead to a working backend.

🐳 Testing with Docker

✔️ Prequisites

  • Docker installed
  • Docker engine running on your computer before testing (e.g. Docker desktop opened)

📄 Create your local secret file for Docker

Create a file named docker.env to the root of the project folder. It should be already on .dockerignore and .gitignore, but please double-check that so that you don't accidentally commit it to GitHub!

The content of this secret file is described in the project's Teams, on #resources channel (fall 23).

🏃 Run the Ultimate™️ start-server-and-test-workflow with docker-compose

  1. Open project directory
  2. Type in terminal docker-compose -f docker-compose-test.yml up -d (this builds the image and starts the container in a configuration that is described in docker-compose.yml file)

That's it! You should see the npm start and test logs running in Docker images terminal. The first time can take a few minutes, but after that the process gets quicker when you have a cached version of the image.

After running the tests, the container exits automatically. Status code tells you immediately how many tests were failing. e.g. docker-robot exited with code 0 means that all tests have passed. You can also open the reports / logs created in robot/results folder for more details.

💡 Useful commands for Docker

Remove all images that are not currently used by a container: (saves space)

docker image prune -a

Start test container without building the image again:

docker compose up

TIP: Remember to clean up unnecessary containers or images regularly in order to save space. Too many dangling images and/or orphan containers take up VM memory and will slow down your computer when using Docker.

〰️ 🔸 🔸 🔸 🔶 ✴️ 🔶 🔸 🔸 🔸 〰️

💻 Testing on your local computer

_Please note that these instructions are only applicable to Windows 10. For other OS steps are same but might vary a bit (for example configuring PATH)

Table of Contents

  1. Prequisites
  2. Configure PATH
  3. Install Robot Framework
  4. Install Chrome Web Driver for Selenium
  5. Create local secret file
  6. Run tests locally
  7. (Nice to know:) About testing in GitHub Actions workflow

✔️ Prequisites

If you are not sure if you have any of these installed, you can check that by running the version checking commands with command line:

robot --version
python --version
pip --version

To install Robot Framework, you need to have:

  • Python installed
  • PIP package management tool installed

Install dotenv Python library

Library is needed to read .env file

pip install python-dotenv

🔀 Configure PATH

It's important to have all the needed folders in "PATH", so they can be accessed via command line from any folder or program. Please check first if you already have them there, no need to repeat these steps if they have appeared there automatically! The folder structure might differ depending on your Python version and/or where you have installed Python, so check the applicable folder structure from your Python folder. Remember also to give the full path of your folders, for example C:\Python39\Scripts\.

The folders that must be in 'Path' are:

  • Python, eg. Python39 for Python v3.9
  • Python Scripts, eg. Python39\Scripts for Python v3.9
  • Python site-packages, eg. Python39\Lib\site-packages for Python v3.9 (note: the Lib folder might not exist with newer Python versions)

➡️ Go to environment variable settings

Windows tip: Write "environment variables" (or if OS is in Finnish: muokkaa järjestelmän ympäristömuuttujia) to the seach box in your task bar; the right setup window opens immediately.

Windows 10:
Control panel > Settings > Related settings > Advanced system settings > Environment variables ;

➡️ Find 'Path' and check if you need to add folders

Windows 10:
Find 'Path' under System variables > 'Edit' > (Add folders if necessary) > 'OK' ;

Comprehensive guide (For other OS than windows)

https://gist.github.com/nex3/c395b2f8fd4b02068be37c961301caa7

🤖 Install Robot Framework

  1. Run command line tool (eg. PowerShell) as an administrator by right-clicking the icon and choosing "Run as Administrator"
  2. If you have Python version 3+, type pip3 install robotframework, if older version, type pip install robotframework
  3. Verify successful installation by running robot --version and you should see your Robot Framework version

💿 Install Chrome WebDriver for Selenium

The tests are using headless Chrome by default, so you have to install a driver for the tests to access your Chrome browser. You also need to have Chrome in your local machine beforehand.

📄 Update .env with correct variables

The content of this secret file is described in the project's Teams, on #resources channel (fall 23).

TODO: - Update tests to use GitHub secrets if possible for better continuous integration.

🎉 You should now be able to run the robot tests locally!

You can run robot tests with npm run startserverandtest command in your terminal (backend must be running).

NOTE: Please do not use custom robot ... commands (if you are not 100% sure of what flags to include in them) so that no log or report files will be accidentally committed & no secrets will be leaked. We have altered the npm scripts in package.json to have a custom command to run robot tests, include secret credentials, hide secrets from log files and output logs to robot/results folder.

❗ Remember these steps before pushing your code:

1️⃣ Run all the unit tests:

Terminal command: npm test ➡️ if no tests start running automatically, press a to run all the unit tests. Fix failing tests by either altering the test or altering your code.

2️⃣ Run all the robot tests:

Terminal command: npm run robottests ➡️ tests should start running ➡️ you can open report.html file that gets created in /results folder in your browser to view a more detailed report about the tests. Fix failing tests by either altering the test or altering your code.

〰️ 🔸 🔸 🔸 🔶 ✴️ 🔶 🔸 🔸 🔸 〰️

:octocat: GitHub Actions testing workflow

[Updated 7.12.2023]: Test pipeline is working as intended (on main and development branches) :)

GitHub actions currently runs all robot tests made in the project, however pipeline does not restrict pushing of the new image into GHCR but will warn if tests fail.

State of testing (fall 2023)

Tests have been mostly updated to the new UI and they run locally and in Docker. Here are some of the findings that migth help the next implementation.

  • A recommedation from the fall 2023 implementation. Consider developing the app so that whenever a developer modifies the code or adds new features, the same developer would also update the tests so that they're always up to date.