Effective Technical Investigators

I was on a panel a number of months back and someone mentioned testers now have to be effective and efficient in their work. I forget the exact context but my first thought was this isn’t new. What might be new is the degree to which your work has exposed you to being an effective technical investigator.

I propose we embrace the idea that testing is an active, skilled technical investigation. Competent testers are investigators—clever, sometimes mischievous researchers—active learners who dig up information about a product or process just as that information is needed.

Cem Kaner in The Ongoing Revolution in Software Testing, 2007

From the moment I read this, I embraced it. Perhaps it’s the image of a Sherlock Holmes-type character trying to crack a case. Or a criminal investigator analyzing computer systems and networks for evidence of wrongdoing. Either way it’s an important concept because it encapsulates the pull and push of investigating things.

Skilled Investigations

Testing is an investigation into technology issues. We combine our knowledge of technology with experimentation, risk analysis, models, tools, critical thinking and many other techniques to understand the state of a product or system under test.

We organize the search, ask hard questions, make observations and look for evidence. We do research into how others might have solved similar problems. We look carefully at the data and parse the results. Finally we report the problems (if there are any) well and harshly enough to convince someone to fix them.

Effective Investigations

Much of our investigation is exploratory. We learn as we go and continually design new tests to reflect our increasing knowledge. Combined with time and complexity restraints, we explore until our results lead us somewhere interesting or to a dead end.

A big challenge is, how to make use of our limited time and resources? It’s a push and pull. We need time to investigate but we also need to be timely in producing insights. This is likely why Kaner later introduced the concept of being effective investigators.

Over the years I’ve developed a quick set of ideas for thinking through effectiveness:

1. We determine the scope of our investigation. Can we identify the smallest parts of the systems involved to help isolate where to work first? What do we need to investigate and can we break down those into mission statements?
2. We determine what skills might we need (even if we don't have them or they are currently beyond us).
3. We apply the appropriate techniques. There are many ways to design, run and interpret tests. Are we running the right ones? Some tests might be re-usable but most won't be, are we keeping the important ones?
4. We use or build tools as necessary to speed along the discovery process. What are our build systems doing? Can we write a script to filter out data we need to aggregate. Do we have useful automated tests?

Note: This list isn't meant to be exhaustive.

A Useful Concept

This essay isn't about how to be effective. It's about re-introducing and commenting on a valuable concept. Kaner's quote is from 2007 and I likely read it in 2014. It made sense because I was digging up information about a product just as it was needed. I became occupied with being effective and figuring out what it means to design the right kind of tests. Turns out I was being an effective technical investigator.