The term ‘Exploratory Testing’ is being used a lot these days to mean the testing done after the software or parts of it is deemed to be ready to be released to the customers. In this article, let’s take a look at this term in detail, and see how much of Software Testing is getting done by Exploratory Testing.
In the Continuous Integration and Continuous Deployment environment, parts of the product are released to the customers in an incremental fashion. In this context, testing is done on the final product that is about to be shipped to the customers. When the product is about to be shipped, Software Testing folks should take utmost care to make sure that the product is validated and verified. This is true for every iteration of the product that is being released, and every iteration has its own features or bug fixes that need to be tested.
The one difference between the CI/CD and the non-CI/CD scenario is that, in the non-CI/CD scenario, internal iterative builds are validated and verified by the test team, but the product does not reach the customer. In the CI/CD scenario, after testing, the product also reaches the customer. This cuts short the feedback time from the end-users or the customers, and hence, catering to the dynamic market needs as well as quick feedback on what is developed is what the customers want is made sure.
The intent of doing this kind product development and testing is great. But there are challenges, which I am highlighting here based on what I have seen.
Smoke Checks Done As Exploratory Testing
Yesterday, I had written about how the term ‘Exploratory Testing’ is misunderstood as cursory checks done before release is done. This type of testing checks only that the major features are working as expected as part of the acceptance criteria (a high level check to make sure none of the major features are broken because of the latest implementation). The term ‘Exploratory Testing’ is taken lightly by the practitioners and Software Testers are hurried to finish the testing so that the product can be on its way to the customer. We all understand the speed of the CI/CD pipeline, but to expect the same speed with the human effort is extreme. Sufficient time has to be given for testing in Staging.
Not Testing Non-functional Aspects Sufficiently
Exploratory Testing is sometimes conducted only as a functional testing activity, testing the functionality of the product developed, and non-functional aspects are either not tested, or pushed to automation as scripts to be executed. Non-functional experts can vouch on how much important it is to have the non-functional testing done with human oversight. There are not just one or two, but many non-functional aspects involved in the product, so testing has to cover the product from these various angles. For this too, sufficient time has to be allocated.
Is it really ‘Exploratory Testing’?
The term ‘Exploratory Testing’ is typically used when the product is being explored. This is apt for situations when you don’t know enough about the product. There might be several unknowns, and you explore the product to find out more about it. This typically happens when there is insufficient interconnect between developers, product owner, and the testers, and testers don’t have much clue on what is in their hands to test. In scenarios where the testers are constantly engaging with the developers, business analysts, and the product owner, and are well aware of what is being given to them for testing, it is less of ‘Exploratory Testing’ and more of testing in a whole, which is validation and verification.
If we claim that the level of testing that is being done at this stage is sufficient, then the only reason I can think of for using the term ‘Exploratory Testing’ for this testing, is to distinguish the testing that’s done by humans and the checks done by automation. Calling human-done testing as ‘Exploratory Testing’ for the lack of a better phrase is incorrect. Exploratory Testing is an appropriate word only for situations where testing is done as an exploration.
So, let’s get the Software Testing vocabulary right, and call testing as testing.
For Software Testing Process Consulting enquires, feel free to reach out to me.