Don't Test the Cloud Service, Test Your Business Logic Instead
Assume the cloud provider knows what it's doing
This is the third post in the series, “5 Strategies for Serverless Testing”. Check out the first post and second post. If you like this content, feel free to share and subscribe!
I am a big believer in looking for ways to maximise my effort when testing a serverless application.
90%+ test coverage may be useful as a feel-good metric but sometimes it’s just an indicator that I spent far too much time writing tests. This is time that I could have spent building features and delivering value to customers.
The main question with testing is, I believe, not so much how much testing do I have in place (i.e. test coverage), but whether or not all my “critical paths” are covered. Am I testing the right things? Should something go wrong, would this test pick it up?
One area that does not bring any major gain is testing cloud services, particularly managed ones.
AWS is incredibly reliable. Dynamo, Lambda, API gateway, they just work. As a general rule of thumb, we should steer away from tests that just tell us whether these cloud services are responding correctly or not. It is far more likely that something will be broken due to a misconfiguration or bad business logic on our side rather than an AWS service being unavailable.
We have talked previously about the higher value of testing your function handler (inputs and outputs) as opposed to unit testing every internal detail of your application.
As you do so and think about the various testing scenarios (including what to mock and what not to), ask yourself whether the test you’re about to write is pointing to something unique about your microservice rather than simply proving that, say, given a certain query Dynamo responds with a certain result. There is nothing wrong with a test like that, of course, as it helps us validate the query in the first place. But one such test should be enough. Subsequent tests should instead focus on whatever you’re doing before and after that query.
An example
Here is an example, using the given-when-then formula:
Given a new user signs up
When signup is complete
Then a new User record exists in Dynamo
This test adds very little value. Once the signup is complete, we invoke the Dynamo APIs and we get a success status as a response. All that tells us is that Dynamo is alive and well.
A better way to test this would be to test for the integrity of the data we just saved in Dynamo, as well as any additional metadata. For example:
Given a new user signs up
with data {"firstName": "Marco", "lastName": "Troisi", "email": "m@troi.si"}
When signup is complete
Then a new Dynamo record should exist
with data {"fullName": "Marco Troisi", "firstName": "Marco", "lastName": "Troisi", "email": "m@troi.si", "createdAt": "date-here", , "updatedAt": "date-here"}
The example above is still relatively simple. In real world applications, you want to go even deeper than that. But the key point here is that I am not just testing for Dynamo to insert the new record correctly. That’s a given. I am not even that interested in whether I am calling the Dynamo APIs properly.
What I really want to know is (given that Dynamo works and I am using the APIs correctly): is any extra business logic on top of that actually working? In this case, there is probably code in my Lambda that aggregates the first name and last name into a fullName property. I am also attaching a createdAt and updatedAt timestamp.
In the real world, I might be generating some unique id, storing the geolocation of the user, or any number of other operations that are specific and valuable to my use case. Those are the things I ought to have good tests for.
The takeaway
When writing a test, avoid spending too much time testing integrations with cloud services
It’s almost always a good idea to assume that cloud services are going to work as expected
Focus most of your testing efforts in double-checking and securing your own business logic around those integrations