Showing post with tag python. Show all

Behaviour Driven Development (BDD) and testing in Python

by Gil BRECHBUEHLER, EBU on 07 Jul 2016

Behaviour Driven Development (BDD) and testing in Python


First we need to introduce the concepts of unit testing and functional testing, along with their differences:

  • Unit testing means testing each component of a feature in isolation.
  • Functional testing means testing a full feature, with all its components and their interactions.

The tests we present here are functional tests.

Behaviour driven development is the process of defining scenarios for your features and test that the features you are testing are behaving correctly under each scenario.
Actually this is only one part of behaviour driven development, it is also a methodology. Normally for each feature you have to:

  • write a test scenario for the feature
  • implement the feature
  • run the tests and update the code until the tests on the feature pass

As you can see the methodology of behaviour driven development is close to the methodology of test driven development.

The goal of this blog however is not to explain behaviour driven development, but to show how it can be implemented in Python. If you want to learn more about behaviour driven development you can read here for example.

Before starting: Python itself does not give us any BDD tools, so to be able to use BDD in Python we use the following packages:

Finally, here is the example Python function we will test with BDD:

def foo(a, b):    
    if a > 10 or b > 10:    
        raise ValueError    
    if (a * b) % 2 == 0:    
        return "foo"    
        return "bar"    

It is a simple example, but I think it is enough to explain how to do behaviour driven testing in Python. If you want to follow BDD methodology strictly, you have to write your functional tests before implementing the feature, however for an example it is easier to first introduce the functionality we want to test.

Note : the fact that the function "foo" does not accept numbers strictly bigger than ten is just for example purposes.

Gherkin language (link)

BDD has one great feature : it allows to define tests by writing scenarios using the Gherkin language.

Here are the scenarios we will use for our example :

Feature: Foo function    

    A function for foo-bar    

    Scenario: Should work    
        Given <a> and <b>    
        Then foo answers <c>    

        | a | b | c   |    
        | 2 | 3 | foo |    
        | 5 | 3 | bar |    

    Scenario: Should raise an error    
        Given <a> and <b>    
        Then foo raises an error    

        | a   | b  |    
        | 2   | 15 |    
        | 21  |  2 |    
        | 45  | 11 |    

A feature in Gherkin represents a feature of our project. Each feature has a set of scenarios we want to test. In scenarios we can define variables, such as <a> and <b> in this case, and examples that define values for these variables. Each scenario will be run once for each line of its Examples table.
Given lines allow us to define context for our scenarios. Then lines allow us to define the behaviour that our function should have in the defined context.

Features and scenarios are defined in .feature files.

Tests definition

Scenarios are great to describe functionalities in a largely understandable way, but scenarios themselves are not enough to have working tests. Along with our feature file we need a test file in which we define functions that correspond to each line of the scenarios. We will first show the full Python file and then explain it in details :

from moduleA import foo
from pytest_bdd import scenarios, given, then
import pytest

scenarios('foo.feature', example_converters=dict(a=int, b=int, c=str))    

@given('<a> and <b>')    
def numbers(a, b):    
    return [a, b]    

@then('foo answers <c>')    
def compare_answers(numbers, c):    
    assert foo(numbers[0], numbers[1]) == c    

@then('foo raises an error')    
def raise_error(numbers):    
    with pytest.raises(ValueError):    
        foo(numbers[0], numbers[1])    

In our case this file is named Note that for pytest to be able to automatically find your test files, they have to be named with the pattern test_*.py.

scenarios('foo.feature', example_converters=dict(a=int, b=int, c=str))    

This line tells pytest that the functions defined in this file have to be mapped with the scenarios in foo.feature file. The example_converters parameter indicates pytest to which type each variables from the Examples should be converted. This argument is optional; if omitted pytest will give us each variable as a string of characters (str).

Then :

@given('<a> and <b>')    
def numbers(a, b):    
    return [a, b]    

@then('foo answers <c>')    
def normal_behaviour(numbers, c):    
    assert foo(numbers[0], numbers[1]) == c    

@then('foo raises an error')    
def should_raise_error(numbers):    
    with pytest.raises(ValueError):    
        foo(numbers[0], numbers[1])    

In these three functions we define what has to be done for each line of the scenarios, the mapping is done with the tags used before each function. We get the values of the a, b and c variables by giving arguments with the same name to the functions.

Pytest-bdd also makes use of fixtures, a feature of pytest: giving the numbers function as an argument to the compare_answers and raise_error functions allows us to directly access anything the numbers function returned. Here it is an array containing the two integers to pass to the foo function. For more details on how fixtures work in pytest see pytest documentation.

Running the tests

To run the tests we simply call the py.test command :

$ py.test -v    
============================== test session starts ==============================    
platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 -- /home/gil/.pyenv/versions/2.7.11/envs/evaluate-tests/bin/python2.7    
cachedir: .cache    
rootdir: /home/gil/work/blog, inifile:    
plugins: cov-2.2.1, bdd-2.16.1    
collected 5 items[2-3-foo] PASSED[5-3-bar] PASSED[2-15] PASSED[21-2] PASSED[45-11] PASSED    

============================== 5 passed in 0.02 seconds ==============================    

If we launch pytest without giving any file to it, it searches for files names with the pattern test_*.py in the current folder and recursively in any subfolder.
We see that five tests have actually run, one for each line of the Examples section of the scenarios.


Behaviour Driven Development is a great tool especially because it allows us to define functionalities and their behaviour in a really easy and largely understandable way. Moreover writing BDD tests in Python is easy with pytest-bdd.

Note that pytest-bdd is not the only Python package that brings BDD to Python, there is also planterbox for Nose2, another testing framework for Python. Behave is another framework for behaviour driven development in Python.

BDD python Test Testing

vagrant 2

by Malik BOUGACHA, EBU on 03 Oct 2013



Last time, we discovered a problem inherant to the software world. Software change an d api to the software changes too. We also briefly talked about virtualenv in python. How does it work ?

Simple usage

You can specify a python virtualenv which is a set containing a python interpreter and a set of libraries. You can specify a version of each libary you want either by a specific version number or by a minimum version.
We define this set of libraries at the configuration time. Let's take a simple example :

mkdir mynewproject 
cd mynewproject 
touch pip_requirements.txt 

So now we will switch to the virtualenv

virtualenv . 
source bin/activate 

we can see that we have only a couple dependencies

pip freeze #this will list installed module that pip installed 

so let's add a couple of dependencies. To make it shorter and more easily manageable, we will put everything in the pip_requirements.txt. #we can also specify a http or ftp tar containing a 
requests=0.10.0 #yes I love old version 

then we install them in the virtualenvironnement :

pip install -r pip_requirements.txt 

As you can see, this installed the module in the new virtualenv.

pip freeze 
Now that we installed new dependencies, we will leave the virtualenv. 


we can check that we are outside of the virtualenv by doing  

pip freeze ```

python version

Now, imagine that you would like to test

System wide dependencies

Great now we can easily manage

  • python dependencies
  • python interepreter

What if we want to add system dependencies ? Say a mysql database and a memcache cache ? We could also want to test our system on multiple host configuration, some with mysql other with posgre, or even test it under different system, for example a BSD based system and a linux based one. both having their own library version or even software. We have another solution. Instead of a simple virtualenv, we will see how we can simply put all our environnement into a separated environnement.

python vagrant

vagrant 1

by Malik BOUGACHA, EBU on 14 Sep 2013

Developpement sandboxing

Vagrant is a way to sandbox the devloppement environement. I will take python as an example of sandbox and explain how we come from a simple os based devloppement environement and test to a vm env.


Let's start with a simple machine environement


The being a very simple flask server:

from Flask import Flask 

app = Flask() 

if __name__ == "__main__": 

Great right ?
But what happens if we add more dependencies, for example requests, a python module for doing http request.

from Flask import Flask 
import requests 

app = Flask() 

def main(): 
    return request.get("").json() 

if __name__ == "__main__": 

The example is pretty simple right ? What does happen when we want to deploy it ? It will crash depending on the version of the requests module we have. As we have a stable version of debian, we don't have the right version of the library and of course we don't want to install it at a system wide level But there is a solution for this in python: virtualenv.

dependencies python vagrant virtualenv