Unit Tests Aren’t Tests, They’re Sensors
A good unit test doesn’t need to check whether the code works, just whether it’s changed
The first thing I say when I’m explaining unit testing to a new grad or junior engineer?
“Forget about the word ‘test’.”
For something we do every day as a vital part of every code change, unit testing is surprisingly hard to explain convincingly. And the name isn’t helping. The word “test” has all kinds of meanings outside of software, and comes with a lot of baggage that just adds confusion.
I think it’s a whole lot easier to grasp what unit tests are for if you don’t think of them as tests at all.
Here’s a dictionary definition of the word “test”.
Noun. A procedure intended to establish the quality, performance, or reliability of something, especially before it is taken into widespread use.
Source: Oxford Languages
If that’s your definition of tests, it’s easy to dismiss unit testing as a strange idea and a waste of time. Machines are predictable, after all, and the computer is going to do exactly what the code tells it to do. The notion of “testing” code makes software engineers sound strangely incompetent, as if we’re just trying out different things until we stumble on something good enough.
If that were true, you’d need to write fewer tests as you got better at writing code, right? For a confident programmer, unit testing would soon start to feel like an unwelcome obligation imposed as part of a box-checking process.
Unit tests are not about checking that the code works.
In reality, the best engineers — the ones whose code is most likely to work first time anyway — write high quality unit tests for every single line of code they produce. Because here’s the thing: unit tests are not about checking that the code works.
Sure, a unit test can help give you some confidence that your function really does run the way you intended. But if you’re not sure the code you just wrote works correctly, your time would be better spent simplifying that code and working on your own skills in the process. Writing yet more code in the form of unit tests is unlikely to help you understand or fix a mess.
So why do the best engineers not only write exhaustive unit tests, but also commit them alongside the code and run them on every build? A line of code doesn’t degrade or wear out when you run it. If you’ve done it right, running the same tests against the same code is never going to produce a different result. Why not just throw your tests out once you’re satisfied the code passes them?
The answer is pretty simple. We don’t just write code once and then forget about it. Requirements shift constantly, and the code you wrote yesterday will probably need altering tomorrow.
The tests live alongside the code so that we can run them again when we change the code. In other words, unit tests aren’t about checking that new code works; they’re about checking that old code still works.
Armed with this newfound enlightenment, the new grad goes to work on their ticket, determined to diligently unit test the heck out of it. But the next day they’re back with a puzzled look and another question.
“I changed some existing code, but now the tests are failing so I have to change those too. What’s the point of unit tests if we just have to change them every time we change the code?”
It’s a good question, and a seeming paradox at the heart of unit testing. But once more it’s a simple problem of terminology. “Failure” sounds like a bad thing, but a well-designed failing unit test is anything but.
A unit test failure is a code change success.
After you make a code change, you should want to see a unit test failure. You should have a sinking feeling if the tests all pass. A unit test failure is a code change success.
Why do we want tests to fail? Because it lets us separate intentional changes from unintentional ones. Software systems are complex and interconnected, and a change in one part of the code can very easily cause something unexpected to happen in another part of the application.
These kinds of knock-on effects are all but inevitable when we have software components that share code or communicate with one another. To have any chance of working on software safely, we need tools to detect the accidental and dangerous side effects of our changes. Unit tests are one of those tools.
If you make a change to the code and it doesn’t cause any unit tests to start failing, one of two things has happened.
It’s possible you didn’t want to change the behaviour at all. Maybe you’ve altered the performance of the code, or the readability, without changing its functionality. Making non-functional changes in the presence of strong unit test coverage creates confidence and feels great.
The second possibility is that your unit test coverage wasn’t sufficient. A change to the behaviour of the system should always result in at least one unit test failure. If the tests don’t detect the change you were trying to make, what other accidental changes might have gone unnoticed? Without unit tests that fail reliably, you’re going to have a lot of work ahead of you to build up any kind of confidence that your change is safe.
If you’re lucky enough — or diligent enough — to have good unit test coverage, you’re probably used to seeing more than one failure for each code change you make.
What happens next is what separates good unit tests from bad unit tests. A failing test doesn’t just need to tell you that you changed something. It also needs to tell you exactly what you changed. If it’s something you intended to change, you can change the test to match. If it’s an accident, you leave the test as it is and go back to fix the production code.
Good unit tests are ones which separate the code cleanly into small pieces that can change independently. When a good unit test fails, it points directly at a specific piece of code or behaviour and says, “this changed: was that intentional?”
Bad unit tests are ones which cover large areas of behaviour and don’t help narrow down the reason for a failure. When a bad unit test fails, it just tells you, “something changed.” You’ve probably experienced this, and found it unhelpful and frustrating. If you didn’t want to change something, you wouldn’t have been touching the code in the first place!
When we write unit tests, what we’re really writing are change sensors. Each one detects movement in a specific area of the code and alerts us so we can decide how to react. They’re like the automated deformation monitoring systems that you see near subway tunnel construction sites to detect unintended movement of the surrounding buildings.
We’re so used to calling these change detectors “tests” that we don’t really think about the name anymore. But each time a new engineer learns about unit testing, it takes a little longer than it should, because the name comes with some preconceptions they need to unlearn.
“Did you test it?” sounds like you’re asking your colleague whether they’ve checked that their code works. But that’s the wrong question, and it’s not really what you’re asking. What you want to know is whether the code can be safely changed in future. A mechanism that’s good at providing safety by detecting unintended changes isn’t necessarily good at determining quality or reliability, and it doesn’t need to be.
Next time you get stuck trying to explain what unit tests are, try starting out with, “well, they’re not really tests…”
