The 1 Priority Change You Need to Make Your Tests More Effective
The 1 Priority Change You Need to Make Your Tests More Effective
Testing serverless apps requires you to think differently
This is the first post of a brand new series, “5 Strategies for Serverless Testing”. If you like this content, feel free to share and subscribe! Also, I would love to hear your feedback: let me know if this type of content is useful and what difference is it making in your day-to-day software decisions.
Most software engineers have been trained over the years to unit test everything, and to leave integration and end-to-end tests as an afterthought. However, this is not particularly effective with distributed systems such as most serverless applications. This is why I suggest that we change our priorities and focus on the types of tests that can really drive our confidence up.
Traditionally, we are taught that our testing strategy should look like a pyramid. You may have come across pictures like this one:
The vision presented by the “Testing Pyramid” is one where most of your business logic is tested using unit tests, followed by a decent amount of service (or integration) tests, all topped up by a relatively small number of UI (or end-to-end) tests.
This totally works within the context of a monolith. In a monolith, you often have a ton of intertwined code all over the place. Anything you can do to test out the individual bits (i.e. units) of it is helpful, and it’s what keeps the system (and the developers!) sane.
Then you’ll probably have a a gigantic database powering everything, and maybe some queues or a caching mechanism in place. Testing that these components talk to each other correctly is done via the integration tests, and it makes sense that you would want quite a few of those, but not nearly as many as the unit tests.
Finally, you’ll want to keep an eye on the most important flows that the end user has to go through when they use your app. Think things like signup or checkout. Just a few strategic end-to-end tests are usually sufficient.
With microservices (and doubly so if using serverless), there is a drastic change in your testing priorities. The testing pyramid becomes the “Testing Honeycomb”:
In this new paradigm, integration tests are the most important part of the equation. A lot of your Lambda code will be working as glue between the APIs and, say, a Dynamo table. Often enough, you’ll find that after you’ve mocked all the external services, a unit test adds very little value.
Integration tests, on the other hand, will give you the confidence levels necessary to know that the whole flow is working as expected. That you are writing the right kind of data into Dynamo and that your GraphQL queries are returning exactly what you expected.
Sometimes, you’ll be doing some more math-like operations with your Lambda code, in which case a bunch of unit tests are exactly what you need. But, realistically, you can expect that not to be a very common occurrence; nor should you allow yourself to fall into a false sense of confidence just because your unit test coverage is over 90%.
Lastly, we have end-to-end tests. Here, again, you want to identify your primary and most crucial flows and find ways of automating them so as to reproduce the end user experience as closely as possible.
The takeaway
Integration tests should be your most common type of test as they add the most value in a world of multiple (distributed) services talking to each other
Unit tests are useful when testing complex code logic, but they don’t tell you much about the overall health of your microservices
End-to-end tests can add value once you can identify your most valuable user journeys and want to ensure they are reliably working as expected