Getting to grips with Selenium

At Lyst we’ve been improving our testing environments over the last year or so, and one of the main elements we wanted to improve was our testing stack with Selenium. We’ve used Selenium in the past, but the tests grew old, were poorly maintained, and few people could work out how they worked after our shift to Docker (read more about that in a previous post.)

Because of this, we decided to start right back to the beginning again. Armed with the lessons we learned from our previous attempt, we aimed to create a new test framework. It was important that they were easy to update when we did breaking site changes, and that they ran quickly. Most importantly, they had to run consistently across multiple test runs.

We’ve had a few test cases run over the last month, and they have caught bugs before we merged.

To give a little bit of context, our website is Django based, so naturally our tests are built with Python. Our unit tests use Py.test, so we saw little reason to not use that as well along with the Py.test Django plugin. And while we investigated quite heavily into other libraries and frameworks, we just stuck with the standard Selenium Python API.

Decoupling page structure from tests

You look up a guide on how to write Selenium tests, they always point out that you can just use one of the many find functions or selector strategies along with a string to get the thing you want. These function much like jQuery or querySelectorAll.

Problems arise when you have a large test suite, and a continuously evolving product and code base. A developer who changes the CSS selector for a certain div element is going to have a hard time updating all occurrences of it in the tests. This can lead to fragile tests that can break on a whim.

The Selenium documentation provides a bit of guidance about Page objects, and we started building some of our own. Since we’re building in Python, we first started with having function attributes that simply do a find_element_by_css_selector (or other appropriate function) and returned the resulting WebElement reference.

That was fine, but we will probably be writing a lot of these which leads to a lot of properties/functions that simply just return a element.

It occurred to me that Python has a magic method that might help here - __getattr__ (read about it on the Python docs). The __getattr__ method is called whenever a property or function is called that doesn’t exist on an object. The object takes in the name and you can return whatever you want with that function. With a little bit of Python magic, I created a simple test function to see how viable it was,

def __getattr__(self, item):
    # put the item into the name format you are expecting
    prop_name = '{}_SELECTOR'.format(item.upper())
    # getattr here returns the value of the attribute on the object
    selector = getattr(self, prop_name)
    # run the standard selenium function for finding elements with
    # CSS selectors - selector should have the value of
    # the prop_name attribute
    return self.driver.find_element_by_css_selector(selector)

And now if I specified a constant string with a name ending in SELECTOR I could simply reference it like any other property - say BUTTON_SELECTOR, I could just reference it with page.button and get the Selenium WebElement object.

Now, sometimes just going by CSS selector isn’t always the best way, and sometimes you want more than one element returned. Our real getattr function is much powerful in these regards, you can specify tuples of (strategy, string) to get an element, and if you specify page.all_buttons, it will return a list of all elements matching that selector.

So referencing any part of the page is as easy as defining a few constants in a class. If a CSS class or path to an element changed, then all that would be needed was changing this constant in the page object. This makes them very powerful, while at the same time being just as flexible as defining the selectors as a string directly in the test.

Because __getattr__ is only called if the object doesn’t have a property, which gives some flexibility if you need something a bit more powerful to select an element.

Common components

Our pages can have the same thing on a page multiple times. That thing can even appear on different pages. For example, a product card not only appears while browsing, but also on search and product pages. How do we share this common pattern without duplicating the code across many page objects?

We have another type of object we call a Component object. It’s almost exactly like a Page object, the only real difference is that it’s based on the context of some WebElement (a Page object is based on a WebDriver).

A Page object can reference a Component object using the same SELECTOR magic described above, but instead of returning a Selenium WebElement, it will return an instance of that object wrapping the WebElement. You can even return lists of Component object instances. For example, calling shop_page.all_product_cards with a PRODUCT_CARDS_SELECTOR using a Product card component, will return a list of Product card instances.

This is very much akin to our component-based frontend architecture, so isn’t too big a paradigm shift.

Creating data

Since our website is data heavy, it’s hard to assume that we get the right scenarios every time we run our tests. We decided early on that Selenium tests should be able to run in their own sandboxed environment. This means they can do whatever they want - such as create data - to test what they need. This is exactly how our unit tests run to ensure they always get the same result time and time again.

Every test run starts with a clean database, and they fire up object factories to create the data they need to perform a test. These are the same factories we use in our unit tests (built using factory_boy and Faker).

But how do you get to a test web server that utilises the test database that it makes?

Fortunately, Django does allow you to create a live test web server, and with Pytest Django, you add the --live-server switch and bind to an IP address and port. Using Docker, you can bind to 0.0.0.0, and any port you chose. The test web server is then accessible via Dockers external IP address with the port you specified.

To test across many browser environments, we use BrowserStack. BrowserStack lets you create a tunnel so that you can access internal addresses in their virtual machines, and this works just as well with a server running inside a docker container.

An example

Here’s a simple example of how this all comes together. Let’s say I go to a product page, get the list of buy options and click on each button that’s relevant to it.

First, our Component. You’ll note we have a ROOT_NODE defined, this references the base DOM node that the component lies in. We use this so the component can find itself (there’s a component.locate() static method), but it can be overridden on page objects by specifying a different selector.

# components/buy_area.py
class BuyArea(BaseComponent):  # all our components are based on this
    ROOT_NODE = '.product-buy-area'
    BUY_BUTTON_SELECTOR = '.button'

Then a page. We can reference the components directly, or a CSS selector string.

# pages/product-page.py
class ProductPage(BasePage):
    # because we specify a root node, we can just include it
    BUY_AREA_SELECTOR = BuyArea

    PRODUCT_NAME_SELECTOR = '.product-name'

Finally, our test. We have to pass in the driver instance when creating a page, and the driver is a Py.test fixture that’s initiated for the session.

# products/test_product_page.py
def test_buy_option_on_product_page(driver):

    # we create a product using one of our factories
    product = ProductFactory()

    driver.get(product.get_absolute_url())

    # pass in the driver instance to the page constructor
    page = ProductPage(driver)

    # we can just simply refer to the lower cased version of the name
    # to get the web element reference directly
    assert page.product_name.text == product.name

    # and, we can prepend the name with all_ and pluralise to get a
    # list of elements matching the selector
    assert len(page.buy_area.all_buy_buttons) == 1

    # there are some context helpers that let us wait for certain
    # actions to happen after a code block has finished
    with driver.page_change_action():
        page.buy_area.buy_button.click()

    # we actually have a more sophisticated URL check, but for
    # this post...
    assert '/cart/' in driver.current_url

Using a context manager to wait for the page to change is an idea we borrowed from Obey the Testing Goat. This has worked quite well for us, so we expanded it to include a few more cases where we have to wait for stuff.

Looking ahead

We’re just at the start of running and building our Selenium tests with every frontend related change, and will eventually automatically test master builds and the website post-deployment as well.

There have been some moments of surprise to us about how easy it to construct a test in our little framework. It’s a project that has had many parts considered and reconsidered to make it as easy as possible for both our software and QA engineers to get a handle on without having to first read everything there is to know about Selenium.

I’ve only talked about what I thought were the more interesting implementation details. There is much more going on behind the scenes, feel free to ask any burning questions you may have below.