The Truth About Functional Testing
I caused a bit of a stir on LinkedIn over the weekend.
How, might you ask?
By sharing a hard truth about functional testing.
I often post my thoughts on the testing and DevOps industry on LinkedIn. There is a great community of industry leaders on the platform, and we often have insightful discussions on the latest technology and trends of the day.
However, one recent LinkedIn discussion on functional testing got much more heated than I expected.
Functional Testing: A Contested Definition
On June 12, I wrote the following in response to an article about functional testing from Guru99, a popular resource in the world of software testing:
“In my humble opinion, regression testing or Sanity and Smoke testing are not the only types of functional testing. Teams should and are including performance testing, unit, and API testing inside their regressions and these are examples of nonfunctional testing.
I think that the definition of functional testing…is not accurate since it states that regression is functional testing while performance is nonfunctional, when…most recessions are and should include both types.”
Functional testing should not just be limited to sanity, regression, and smoke testing. To do so is very misleading and takes away from what really should matter to testers.
Instead, any working definition of functional testing should focus on the process of validating the functionality of a software application. There shouldn’t be limits to the various functionalities under test.
I put this out there to other testers in the industry to give their thoughts, and wow, did they deliver. Here are some of my favorite insights throughout this discussion that lead to a much more “truthful” understanding of functional testing.
Test Type vs. Test Scope in Functional Testing
One important point made during the discussion was that the Guru99 article completely blurred the distinction between the test ‘scope’ and a testing ‘type.’ While testing types can range from accessibility testing to stress testing, the scope refers to whether there are unit, integration, or systems tests.
As Eliezer Shindler of F5 noted:
“It’s completely feasible to have [a] unit test that is a performance test.”
He also pointed out his frustration that there are so many mixed up concepts out there, that they actually take away from the one goal of testing: ensuring the quality of the product.
Scope and type are not the same things. While not common, there are situations where you may require performance tests (type) at the unit level (scope). When you mix up important terminology, it makes it very difficult to build a sound and effective strategy for functional testing.
Functional vs. Nonfunctional Testing: Is It Necessary?
Others asked if it’s even necessary to make a distinction between functional and nonfunctional testing at all.
Paul Bruce suggested (and I agree) that the idea of functional vs. nonfunctional testing no longer makes sense. Instead of conflating terminology, testers should realize that each testing type has different tactics and goals.
Plus, there are ways to combine these testing types constructively, whether it’s functional regression testing or API performance testing.
Is Nonfunctional Testing Even A Thing?
Some commented on the fact that the term “nonfunctional testing” is a complete misnomer. Michael Bolton humorously compared nonfunctional testing to other terms that would make you scratch your head, such as ‘non-physical science’ or ‘non-edible food.’
Instead, he asserted that it makes more sense to refer to testing in the affirmative. In his LinkedIn comment, he suggested the following:
“Think of X testing as ‘testing focused on risk related to X.’ Functional testing focuses on the risk that functions in the product will have errors; performance testing focuses on performance-related risk; regression testing is testing focused on risk that the quality will regress (get worse) after a change. Every test focused on some risk also provides some peripheral, incidental, or accidental coverage of other kinds of risk, if we bother to pay attention.”
My colleague Jim Hazen also had important insights on the term “nonfunctional testing” (he was rightfully mad that I didn’t tag him in the original LinkedIn post, so I’m trying my best to make up for it here). He agrees that it doesn’t make sense to use the term “nonfunctional” to describe the work necessary to validate software and system readiness.
Instead, Hazen reminded everyone in the discussion that anything done in testing is either proving, disproving, or validating a hypothesis. For example, he noted that his performance tests check whether a system behaves according to some expected result. Therefore, performance testing is a type of “functional” testing.
Which leads me to the last (but not least) insight I am going to share from this LinkedIn thread:
Bottom Line: Testing Reqs Are Important, Test Labels Are Not
Alan Page summed up this conversation well by noting that we should not get bogged down too much by labels. Instead, he suggests that testers direct their energies toward identifying and exploring risk or failure points according to their specific context.
I wholeheartedly agree with this concept and encourage testers to build a testing strategy based on what is needed for your specific situation, not labels so you can check a proverbial ‘box.’
But as the conversation also showed, our industry publications need to do better when giving testers proper guidance.
Enjoyed the article? Follow me on Medium to stay up to date on all things test automation and application development.
Join FAUN: Website 💻|Podcast 🎙️|Twitter 🐦|Facebook 👥|Instagram 📷|Facebook Group 🗣️|Linkedin Group 💬| Slack 📱|Cloud Native News 📰|More.
If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇
