segunda-feira, 16 de janeiro de 2017

Contract tests

Today's post is to talk about contract tests, the value of having them and strategies to have trustable test suites.

Everyone know the value of tests, they will ensure that the system behaves as expected, be using BDD or TDD or whatever technique you prefer. Of course any kind of system/problem may have a more appropriated kind of test.

In the last few years, I've been working with APIs. Each day is a new challenge, sometimes we have the need to version it, change contracts, make new integrations, include new rules, etc.

I've been developing an API with NodeJS, and it was chosen CucumberJS to write tests, so then the team was be able to do BDD. Of course there are other libraries that would allow us to do the same, anyway...

When writing APIs, one important thing to think about is to have contract tests, so then I believe BDD was a good take.

Testing a contract is not something straight forward, you need to think about many things and create strategies.

Treating external dependencies with contract tests

When you write an API, there is a big chance that you have external dependencies, be it a database, other APIs, messaging mechanisms, cache, etc. You don't want to trust real dependencies when running contract tests, tests must be 100% independent and should not trust a real external dependency.

What's the chance of an external dependency be unavailable at the time you want to test your contract? Do you want your contract tests to fail if a external dependency is not available? No, right?!

There are some good tips to avoid this kind of issue to happen, depending on the kind of dependency we can have different strategies.

External APIs

Having an external API as dependency may be a problem as mentioned before, the API may be down, or with some delay, it can make your tests to fail or take a long time. A solution for this problem is to create stubs (Martin Fowler post about mocks and stubs).

Stub is a simple interface which will give you standard responses of a giving API. Let's say you have a dependency on a Products's API and your API needs to retrieve a list of products. This stub would always return a list of products, this stub could be started up with your suite of contract tests, and be stopped at the end of its execution.


Many times we are dependent on data, so a database dependency may be a big problem.

You cannot just trust your QA or DEV environment database, not just because of the fact that the database may be down, under stress, a dataload is happening, it can even be empty, or it can have any kind of unavailability. But also because you may need some data on it, and you also want to make sure it will always, no matter when, return the data you expect.

You can easily start up a database with docker and include some fake data on it before your tests execution. Remember that your tests must be 100% independent, which means that a database change that is done on the test 1 will not impact the test 2, and the test 2 is not dependent on a state change made on test 1.

These are just 2 good tips to make your contract test faster and trustable.

Containerize your tests

As mentioned before, containers are a good solution for some external dependencies, they're immutable and can be executed on any environment. Why not run your tests with docker-compose? You can start containers from the scratch, seed some data, run the tests, and kill it when the execution is finished.

If you need to run the tests again, you will make sure the initial state is always the same, this way you have a good and trustable test suite. It even facilitates your life when you want to include the tests on a CI server, you just need docker installed and that's it.

Contract assertions

One very common mistake I see on some contract tests suites is that people only check if the request worked properly.

The contract is composed by many other things, here are some things you can check when doing contract tests:

HTTP Status code

The HTTP status code must be checked, different requests should return different status codes. If you do a GET on a specific endpoint, it should return 200 (ok). Or 404 (not found) if the given resource was not found. if you are creating a record (doing a POST), the status code should be 201 (created). These are just some examples.

Response body

The response body is also part of the contract, when you send a request, you expect a response body always with the same shape. Let's say you have a products's endpoint which returns the fields: id, description, unitPrice, available. Everytime you GET a product, you may expect to receive back these fields.

It just remembered me of Consumer Driven Contracts, you should definitely consider it when you are consuming APIs. (here is a great post from Martin Fowler about it)

I think that's all, I hope I helped you understanding how a contract work and how it can be tested, please feel free to tweet me and discuss about this subject. here is a great post about contract breaks, enjoy it.