How To Stop Worrying and Love the Mock
# Who am I
My name is Roman, and I am the Tech Lead of the QA Automation team at Tabby. My team is responsible for developing automation tools and helping others to love automation as much as we do. In our work, we engage with a diverse array of tools including Buf, Pytest, and Playwright, while exclusively employing programming languages like Go, Python, and TypeScript to meet all our development needs. Our architecture is based on microservices, SAAS and cloud stuff.
# What is This Article About, and Who is it For?
This article is mainly about taking steps towards shifting left in automation testing. It’s all about explaining parts of the system that can help QAs set clear boundaries for automation and discover useful patterns.
The explanation was built around some proof-of-concept modeling with real Go tools like mockgen and protoc. So, it’s also handy for showing how to use these tools for testing purposes.
# What is the Problem and What Do We Want to Achieve?
I do believe it’s quite common, but let’s quickly set the scene. Our product teams are cross-functional, and it’s a standard practice for our QAs to focus on End-to-End testing, which significantly enhances their confidence in the scenarios they develop. However, they are often challenged by the fragility, slowness and tight coupling of this automation.
We’ve spotted a gap in this test strategy. It really stands out when you look at CI/CD pipelines, where at most stages of the pipeline we are only dealing with the service code or the service itself in isolation. That’s why we have to start to shift our automation to the left.
In this context, my task is to engage QAs, and the best way to encourage someone is through explanation. I’ve found that it’s more effective to talk not about the test pyramid or any specific forms or shapes, but about the clear boundaries of responsibility for each service (or team).
Talking about microservices and all the tools we use — it is pretty tricky. They’re packed with techy details and everything’s connected. Sometimes, you just need to show how it all works instead of just talking about it. It makes things way clearer.
In the next part I am going to put together a simple model that fits right in with what we’re mostly into: backend stuff, Golang, gRPC, and proto files. The aim is to build it with simple components but keep all the essential technical details. It’s designed to highlight how mocking integration contracts can address common issues.
# Building a Model for Better Understanding
Let’s dive into modeling.
As has been said, in most stages of CI/CD we typically have a single service working independently, so our model should represent that scenario.
First, for our service. It is meant to be tested by QA. Just put user interaction here like our service should let the cats 🐈 out into console. But the number of cats should depend on the external call.
Second, how can I accurately represent the dependencies of a service? A single RPC call might involve dozens of calls between services. Each call depends on data and business logic. Furthermore it is also exposed to network errors and service availability. A quick example here. Let’s say I’m testing an order object. One time, when I checked for its item availability, I might run into a 500 error. The next time I try, the item becomes available. And the next time all the items are gone because it is a Black Friday on my test contour.
Anyway it is still an RPC -Remote Procedure Call, but the function it calls is inherently complex. I know the term for a function that always returns the same results with the same parameters: it’s called a pure function. However, the function I’ve described is definitely not pure! So it might be considered random in a convenient range of values, as each call to this function could yield a different output. Let it be random then!
Lastly, I’ll narrow down the interaction between tested service and dependents to a single RPC call without losing any meaningful details. I’m not concerned with the composition of the RPC call message, nor do I care about how many of these calls are taken.
Ok, less talking more doing. Now I can try to put all things about cats vs. random together in a piece of Go code:
And now run it:
The main() function becomes our service, random() represents the external world. The signature of random() acts as a contract with our main(). If we want to avoid testing random() (of course we are) we can write some other function with the same signature and use it instead.
However, this is too simple, there is nothing about CI/CD here. Let’s not get too caught up in analogies when we can take a more realistic approach. The next thing I’m going to do is place random() into a Go package named unknown and use it as an import. Here, I can integrate a real CI process into our model.
Now, our model becomes this:
Everyone can point to the external repository and note that this library follows its own CI rules.
It is time to add the part that should represent a single call.There should always be some kind of well-defined contract between services like protobuf or Openapi. By moving from a random() function signature to a contract in .proto, our model will really start to mirror the real thing.
The contract is derived from the signature of the random() function.
I just found our signature, made some go code from it by protoc tool:
The last thing to do is to implement a service and move the random implementation to its handlers.
The playable version of the service can be found in its repository. This single service provides a gRPC Unary call named ‘Random’. Also, a Makefile is used as glue between tools and Go code for better portability. Now, I am building and starting it, and then setting it aside running.
A console tool that we want to test is placed at another repository. It also contains a client for the mentioned service, calls the function, and uses the result of the call to multiply by a cat 🐈.
In this model, the RPC call and the CI boundaries (repositories) are very realistic, they are made by real tools and no analogies here. And both the tested console tool and the external service are simplified without losing their meaningful properties.
# Is This Model Good Enough?
After some intensive work, let’s examine the potential downsides of automation that our model might reveal. Are you ready to delve into this?
I’m not particularly focused on testing whether the ‘Random’ side calculates real random. The function there could be replaced by another; my primary responsibility is to ensure the correct number of cats is displayed.
In situations where I can’t set up the remote side from running autotests, my only test option is to check if my result includes at least one cat.
And I’m unable to test negative scenarios, such as whether I’ll end up with -1 cat displayed in the console.
There’s also a problem with unmanaged dependency: I need the service to be running, and I must also handle various errors in the test code.
This model clearly demonstrates the real-world challenges QA faces, which is great because it leads us to the next part: finding solutions.
# Using Mocks
In the final part, I am going to set up a legitimate mock for the external call. This will allow us to fine-tune our automation just as we need.
Here’s the deal: For the remote service, I’ve only developed a ‘Random’ function and created a .proto file with the contract. The parts handling the gRPC interaction and the client were completely autogenerated. This client part is key — it handles talking to the external world and knows all about when and how to make the gRPC call.
Now, if I swap out this part with a mock, I don’t actually have to mess with the cat tool code at all. The logic stays untouched, separate from these external interactions. Remember, the main job of my cat tool is just to show the right number of cats.
If we examine the auto-generated client code more closely, we’ll find that it includes an interface.
Anything that behaves like a ComplicatedClient can seamlessly substitute the actual client. To qualify as a ComplicatedClient, an entity only needs to have the same Random method. So, should I write this method now?
There’s a straightforward method to create it using the mockgen tool:
I can even offer this mock as part of the service, ensuring it’s always up-to-date.
Here is how I can rewrite my test
Now, I can verify that the exact number of cats is produced, eliminating any flakiness in the design. Additionally, I can input any response I can imagine and check the outcome.
It seems like all the problems were fixed by using mocks.
# Recap
A quick recap is in order. It will help us gather all the crucial parts we’ve covered along this journey.
A simple lab was built that’s really up to the task. It’s good at tackling the problems we’ve got and uses real-world tools to solve them.
The challenges highlighted by our model were:
- Unclear test logic that can only be resolved within the system being tested.
- Inability to test negative scenarios.
- Unmanageable dependencies leading to more complex code.
The concise solution for this problem is a mock that can be easily autogenerated and provided as part of the dependency.
# Conclusion
Wrapping things up, our model really helped us get on the same page and talk things through. It’s great for showing anyone who’s skeptical about mocks how useful they can be.
We’ve only scratched the surface here, focusing on the simplest part where services interact through contracts. But remember, mocking is much more than that — for example, you can mock any interface inside a service too. That part gets more complex and is very specific to each service, so we didn’t dive into it here, but it’s definitely something worth exploring on your own.
Written by Roman Gordeev,
Tech Lead of the QA Automation team
