Testability Revisited

 

I read the below question on The Test Chat's Telegram group.

When you start working on a project, what steps do you take to establish the testability of the product?

This question is helpful in learning how we see the Testability of a product.  It is a common perception to see the product with testability and then to test the product using the testability.

But, in reality, the testability is associated more with the tests; the tests which are used to test a product being developed or developed.

So, when we talk about testability, we need to be more aware of the test that we will be designing to test the software.  This test should be quick and easy to execute with the help of the programmed or available testability factors and their attributes.

You can find more blog posts in and around testability here in Testing Garage.  Testability in software engineering and systems is one of my research areas.

Testability

I understand Testability as

• How easy it is easy to test by an engineer
• In a given context and skills of an engineer
• With the being used test approach and strategy

Note: The context can keep including factors as we add more and continue to test

It is not about if one can test the software or not.  It is all about software that is easily tested.  How easy?  That is one of the testability factors in software design and programming.

Test and Testability

Unless I know the test, I will not be certain about the Testability.  Testability does not drive tests.  It aids the execution of the test and it is a heuristic.  If the test is designed well to the context and if the testability is used well in the test's context, the execution of a test can be quick and easy.

The tests

• make use of available Testability
• helps to strengthen the Testability
• add more Testability in different seams/layers of the engineering and product

From here it will be two ways; the tests and testability will complement each other.  Further, it leads to developing and including more specific and deterministic tests and testability types in respective seam/layers.

Testability and Automatability

Testability can be classified further into several categories.  Based on the purpose and what to evaluate we will have to identify Testability in respective categories and need to use it.

As a software engineer one is bound to think testability with software programming and infrastructure.  But, testability in software engineering is not just bound to software programming.

The testability is diverse and available across engineering activities.  It is used in all engineering activities.  Maybe, for a software engineer who is hands-on with programming and testing, they infer Testability most times with programming and infrastructure.

I see, the Testability always exists to an extent.  But, can it be identified and used in the way I approach, design, execute and evaluate my test?  That is the point to explore.

If it is testable to some extent, then we are using some Testability attribute(s).

If there is Testability in a seam/layer, then there is an Automatbility in that seam/layer to an extent.

If it is automatable then there is some attribute of Testability in that seam/layer.  Again, the question comes to knowing and learning -- What am I testing and automating? Why? How? When? Where?

This discovers seams/layers to test and automate.  It leads to identifying the tests.  Then, to identify and build more Testability and Automatability.

A written program feature essentially will have an automatable characteristic and space.  If it is automatable, then it is making use of and extending the testability.

In summary,

• Know the test to know and identify the Testability better
• Know the Testability to automate better
• Know the Automatability to assist your testing better.

Context-Free Questions to Identify Testability

To know and have better Testability, here are a few things that I will want to know:

1. What is the test?
• What am I testing and what am I supposed to test?
• How is this test designed to learn and evaluate the system?
• What are the data, states, and events that I'm experimenting, exploring, and experiencing as I test this system with help of this test?
• How can I make this testing quick and easy?
• What should I use to make my testing quick and easy with this test?
• How should I use it to make my testing quick and easy with this test?
• When and where should I use it to make my testing quick and easy with this test?
2. Why am I testing this?
3. What happens if I test this and do not test this?
4. What is the value loss I will incur if not tested?
5. What is the value loss I will incur if I do not understand and learn the outcome of the test?
6. What changes the dimensions of my tests?
7. How can I learn the product better from this test?
8. What information am I learning from this test?
9. What information, heuristics, and Oracle help me and stakeholders to analyze and decide better?
10. Do I actually know the product from the perspectives of
• tech
• business
• user
• risks
• problems
• protocols
• guidelines
• environment
• money
• benefits
• exploitations
• team developing it
• and, more that I can add to the context of the product and project

To summarize, know the test and know how the test is designed.  It helps to identify better testability at the right layer/seam of the software system and engineering.  If there is no effective testability at that seam/layer, it helps to build one.  That way, the automatability also gets built in that seam/layer if the team collaborates well.