GraphQL integration testing

Originally posted on Medium.

At DueDil we’ve been working with GraphQL for the better part of a year now. We’ve learned a lot on this journey and would love to share some of our lessons learned with the community.

One of the first things we wanted to know was which server library to use. An important decision metric is the level of GraphQL spec compliance. In order to quantify this, we created a small suite of integration tests that was constructed to test all aspects of the spec. We then implemented the same sample schema in multiple libraries and were able to discover incompatibilities early on. This really helped us out in weeding out immature implementations that were claiming compatibility with the spec.

However, the integration testing tool we created was generic enough that we could late use it for integration testing of our business logic in our final GraphQL server. It’s a tiny Python script, but it’s rather useful so we thought we would share it.

Test format

file: suite_one/example.test
query Example {
  a {
    b
  }
}
<===>
{
  "data": {
    "a": {
      "b": "Expected test value"
    }
  }
}

Tests are written simply as a query and an expected response wrapped up in one file (separated by <===>). You can optionally use variables as well, I would suggest looking at the README for the full list of options. It runs all your test files in parallel by default. But you can choose to narrow it down to a single suite of tests, a single test or tests matching a pattern:

gqltester 'http://localhost:8000/' suite-name
gqltester 'http://localhost:8000/' suite-name/example\*

Again this is an extremely small python script so don’t expect anything too fancy. But it does have two neat features that we’ve found very useful.

Replace test expectations on data changes

Integration tests relying on dynamic data have a tendency to break as data is modified. These false negatives are frustrating to deal with. You could try to avoid this by having static test data, but it’s not always desirable. The -r option will replace all failing tests’ expectations with whatever is currently output by the GraphQL server under test. So you can run the tests against a known green commit with the option and get rid of these false negatives. A typical workflow for tests with dynamic data becomes running the tests with -r prior to making modifications.

Regression testing against production

It’s not uncommon to have different data in your testing environment and production. You will probably want to write your test expectations using testing data to prevent disruptions to production service because you’re running tests. But what happen when you’ve staged your new code and are ready to deploy? You can’t run your test suite against the staged instance as it’s using the production data.

gqltester --regression-server='https://production.acme.com' 'https://staging.acme.com'

Allows you to run all integration tests against the staging server but using the responses from production as the expectations. You can now verify that no regressions were introduced in the code you are about to deploy.

We hope others will find this useful. You can find the project on github.