Introduction

This week I will be showing how to configure selenium to run on file change, in docker. This will essentially be an extension of the previous setup in that I will show my personal preferences when developing in Docker.

Docker really feels like this sometimes...

Motivation

I’ve been messing around with selenium and docker since around 2013. I iterated over several homebrew images to varied success, which would invariably break or I’d forget how to use because… ya know… no documentation… or it was just fragile to begin with. Then I found several public images… that would eventually do the same.

Eventually Selenium would release it’s own public docker image… or maybe i just finally found it… with very little guidance on how to leverage it… or maybe I’m just not good. Which is probably true. Anyways I thought i’d share my docker-compose setup that leverages selenium-docker. All the docs still dont really give you an example on how to link a container to it.

Requirements

You should check out how I auto run tests if you have not yet. This tutorial will be using;

  • Docker
    • SeleniumHQ’s image
    • My Personal Development Image (or any image with entr/ag installed)
    • Postgres
  • Django

Boot containers / bootstrapping a django app

Pretty straightforward goal here of having some simple running django app and a selenium test that probes that its running. Lets dive into the initial project setup.

docker-compose.yml

version: "3.9"
services:
  db:
    image: postgres:12
    volumes:
      - ./data/db:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
  web:
    build: .
    command: python ./manage.py runserver 0.0.0.0:8000
    volumes:
      - ./web:/code
    ports:
      - "8000:8000"
      - "8888:8888"
    depends_on:
      - db
      - selenium-hub
  selenium-hub:
    image: selenium/hub:3.141.59-20210607
    container_name: selenium-hub
    ports:
      - "4444:4444"
  firefox:
    image: selenium/node-firefox:3.141.59-20210607
    volumes:
      - /dev/shm:/dev/shm
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444

Dockerfile

FROM derekadair/python-workflow:dev
ENV PYTHONUNBUFFERED=1
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt

requirements.txt

Django>=3.0,<4.0
selenium>=3.141
pytest-django
pyinotify
psycopg2

Bootstrap A Django App

Now we’re ready to spin up a django project. The following command will;

  1. Build your docker containers
  2. Create a django project
  3. Give your current user permissions. NOTE: You may more may not need sudo here
docker-compose run --rm web django-admin startproject seleniumdemo . && sudo chown -R $(whoami):$(whoami) ./web
Connect the database

Now you will see the django code in the web directory. Open that up and you will fiiiind! ANOTHER WEB DIRECTORY. Personally this drives me crazy so I rename this directory settings, typically. You do you.

In settings.py

# Allow all hosts for development.  This can be 'web' but you'd need to add any other development hosts
ALLOWED_HOSTS = ['*']
# Change DB settings to connect to postgres container
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'postgres', #postgres docker image default
        'USER': 'postgres', #postgres docker image default
        'PASSWORD': 'postgres', #postgres docker image default
        'HOST': 'db', #set by docker-compose's container name
        'PORT': 5432,
    }
}

You can run the migrations if you want, but that wont affect the rest of the tutorial.

Add ./web/pytest.ini

We need to point pytest to our settings module

[pytest]
DJANGO_SETTINGS_MODULE = seleniumdemo.settings

Next Step: Running Some Tests

There are essentially two major pieces here that we need to connect.

  1. Code that is running in a container (my personal development image).
  2. The running selenium hub

This is highly dependent on the version of selenium you are running. Furthermore its vital that you match your python version of selenium to the server.

Stub out integration test

You can organize your tests however you’d like. I like to run my integration tests in a distinct loop from my unittests. With django-pytest it will automatically pick up all tests that start with test_. Create /web/tests/functional_test.py so pytest doesn’t pick this up by default.

Docker Live Server Test Case

Django comes with a really nice live server test environment… which doesn’t work out of the box with our docker setup.

functional_tests.py

from django.test import LiveServerTestCase
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

class DockerLiveServerTestCase(LiveServerTestCase):
    @classmethod
    def setUpClass(cls):
        cls.host = "0.0.0.0" # django runs on this ip
        cls.port = 8888 # we want the test server running on 8888
        cls.live_server_uri = "http://web:{}".format(cls.port) #format our own live_server_url
        super(DockerLiveServerTestCase, cls).setUpClass()
VERY simple test to see it all working

Note: I set self.live_server_uri because self.live_server_url breaks in this setup. cls.host needs to be 0.0.0.0 for docker and the hostname on selenium-hub is web. If you have any ideas on how to fix this contradiction please email me.

class DjangoSeleniumTestCase(DockerLiveServerTestCase):
    def setUp(self):
        self.browser = webdriver.Remote(command_executor='http://selenium-hub:4444/wd/hub', desired_capabilities=DesiredCapabilities.FIREFOX)
        self.browser.get("{}/admin".format(self.live_server_uri))

    def tearDown(self):
        self.browser.quit()

    def test_django_selenium(self):
        self.assertIn('Django', self.browser.title)

Invoke the tests

Now we can auto run selenium tests on file change! (Peep the previous article if ag/entr confuse you here)

docker-compose up && docker exec -it {YOUR_CONTAINER_NAME} bash -c 'ag -l | entr -dc pytest functional_tests.py'

Conclusion

All in all its pretty satisfying to be running my integration tests in docker, on file change. There is a lot of power behind this setup; anyone I collaborate can immediately be onboard with my exact setup. In all previous organizations Selenium tests were only supported by the organization in CI cycles, which can have quite a bit of latency. With this setup you and your team can now leverage integration tests effortlessly on your own machine.

Using Other Frameworks

This proof of concept should translate very well to other test frameworks than django. While there is a lot of django specific code and setup. There are several key takeaways here;

  1. Run your tests on 0.0.0.0
  2. Point your selenium code to web or whatever hostname you choose
  3. Check your selenium versions match (docker image/selenium library)

Next Week

I will be building a photo sharing app with Django Rest Framework and Vue.js. This will include multiple file uploads as well as a gallery to view the photos.