Robot & Integration Testing

Will Moore & Colin Blackburn

17th February 2015

Overview

Python integration tests: some updates

Robot framework: User's Guide

Python integration tests:
some updates

Python integration tests

  • build.py
  • setup.py
  • library.py
  • weblibrary.py
  • setup & teardown

build.py

  • Long running and broken tests excluded by default:
    • ./build.py -f components/tools/OmeroPy/build.xml integration
./build.py -f components/tools/OmeroPy/build.xml integration -DMARK="not (long_running or broken)"
  • To include all tests use an empty marker:
    • ./build.py -f components/tools/OmeroPy/build.xml integration -DMARK=

setup.py

  • Test target now specified with -t (was -s)
  • Option -s now available to show any output to console
    • cd components/tools/OmeroPy
./setup.py test -t test/integration/test_admin.py -s
  • Sets PYTHONPATH for all components...

library.py

  • Now a package under components/tests/python
  • setup.py sets PYTHONPATH for all components
  • Now simply import library as lib
  • Set-up and tear-down refactored as class methods...

setup & teardown

  • Class methods setup_class(cls) ...
  • ... and teardown_class(cls)
  • Create and destroy root and user clients in ITest
  • Overridden by some classes but still at class-level
  • More efficient, cleaner, simpler

weblibrary.py

  • Defines IWebTest class derived from ITest
  • Initializes Django test clients
  • Helpers refactored to this class
  • Redundant fixtures removed

More information

Robot framework:
User's Guide

Robot framework: User's Guide

  • Getting started (finding & running tests)
  • Current state of tests
  • Current state of CI robot jobs
  • How to write tests

Getting Started with Robot

  • Robot tests are under components/tests/ui/
  • See README.md for info on how to set-up and run tests.
  • Web tests are in testcases/web/.
  • Web keywords are in resources/web/.
  • Web custom Python keywords are in library/python.

Running tests

  • Set up users, groups and imports 2 images in P/D. 
    $ cd dist/
$ bash ../components/tests/ui/robot_setup.sh
  • Run tests. E.g. to run a single file of tests: 
    $ ./build.py -f components/tests/ui/build.xml web-firefox -DTEST=tree_test.txt
  • HTML report generated. Useful for tracking single failures locally.
  • No way to run only a single test within a file, except by commenting out all others.
  • See README.md for full details

Robot Tests: History...

  • Original Selenium tests now removed
  • Robot Framework, Summer 2013
    • UI tests for Web & Insight
    • Small number of example webclient & webadmin tests
    • No job to set-up DB, users etc
    • Tests not run by any CI job

CSRF Testing, Autumn 2014

  • Coverage of all POST urls (forms submits and ajax calls)
  • Only a single test for each url
  • Each test checks submit for 'success' (not 403)
  • CI job to set-up users and run tests on Firefox & Chrome
  • Usually 1 or 2 failing tests

Now, adding / updating tests for various PRs:

Writing tests

  • Use existing tests as examples
  • Robot Docs: Built-in | Selenium 2
  • Use Browser Dev tools to identify elements
  • xpath or css selectors: Use Google / Stack Overflow etc.

DB setup

  • Robot User and Root. 
    # ui/resources/config.txt
...                                
${ROOT USERNAME}        root
${ROOT PASSWORD}        omero

${USERNAME}             robot_user
${PASSWORD}             ome
...
  • Robot user has: Project/Dataset/2 Images
  • Most tests create their own data. E.g. Test Annotate annotates new Project