Part 2. How to approach API testing?
🤨 What is API Testing?
API testing is important for validating the functionality of the API and ensuring that it meets the functional requirements. It is critical for integration testing since APIs are used to communicate between different software systems. API testing helps to identify issues early in the development cycle and prevents costly bugs and errors in production. This process is designed to not only test the API’s functionality — but also its reliability, performance, and security.
🧪 Why should you care about API Testing?
- You can find bugs earlier and save money
Testing REST requests means you can find bugs earlier in the development process, sometimes even before the UI has been created!
FACT:
According to the Systems Sciences Institute at IBM, the cost to fix a bug found during implementation is about six times higher than one identified during design. The cost to fix an error found after product release is then four to five times as much as one uncovered during design, and up to 100 times more than one identified during the maintenance phase. In other words, the cost of a bug grows exponentially as the software progresses through the SDLC.
- You can find flaws before they are exploited
Malicious users know how to make REST request and can use them to exploit security flaws in your application by making requests the UI doesn’t allow; you’ll want to find and fix these flaws before they are exploited
- It is easy to automate
- Automation scripts run much faster than UI Automation
❌ Everything could go wrong!
When working with API, there set of risks and potential bugs that you might avoid to ensure the reliability and security of the application (not limited list of risks):
⚠️ Risk#1. We could extract personal / private information without proper authentication. It could lead to unauthorized access problems and data breaches.
🐞 Bugs: Missing or misconfigured authentication tokens, incorrect permission settings, or bypassing authorization checks.
⚠️ Risk#2. When a user sends wrong data in the wrong format, it could break the system with 500 errors.
⚠️ Risk#3. Improper input validation can lead to security vulnerabilities like SQL-injection cross-site scripting.
🐞 Bugs: not validation request parameters, not handling unexpected data formats properly
⚠️ Risk#4. Insecure data transmission. Transmitting data using unencrypted channels could lead to exposing sensitive information or interception.
🐞 Bugs: Not using HTTPS, ignoring SSL certification
⚠️ Risk#5. Poor error handling may lead to exposing sensitive information or make difficult diagnosing of issues
🐞 Bugs: returning too details error messages which are revealing implementation details or which are not providing necessary information to the user
⚠️ Risk#6. Performance issues. API doesn’t handle loads efficiently which can lead to performance degradation or outages.
🐞 Bugs: memory leaks, inefficient database queries, not optimized API response times.
This schema illustrates the types of questions that a tester can pose to ensure comprehensive API testing. This list is not limited.
💡 Let’s take a look at the API Testing in more detail
Introduced by Mike Cohn in his book Succeeding with Agile (2009), the pyramid is a metaphor for thinking about testing in software.
The testing pyramid is a concept in software testing that represents the ideal distribution of different types of tests in a software development process.
It emphasises having a larger number of lower-level tests and a smaller number of higher-level tests. The testing pyramid is a way to ensure a balanced and effective testing strategy.
I adjusted this pyramid to API Testing and what I’ve got:
Unit Testing
Unit tests, unit tests and unit tests once more. Everybody knows the benefits of unit tests: we should be able to identify any problems with the current components of APIs as soon as possible. The higher unit tests coverage, the better for you and your product.
Contract Testing
Assert that the specs have not changed. This type of testing is used to test the contracts or agreements established between various software modules, components, or services that communicate with each other via APIs. These contracts specify the expected inputs, outputs, data formats, error handling, and behaviours of the APIs.
JSON-schema is a contract that defines the expected data, types and formats of each field in the response and is used to verify the response.
Official Documentation: https://json-schema.org/
Functional Testing
The purpose of functional testing is to ensure that you can send a request and get back the anticipated response along with status. That includes positive and negative testing. Make sure to cover all of the possible data combinations.
Test Scenario categories:
- Happy Path (Positive test cases) checks basic information and if the main functionality met
- Positive test cases with optional parameters. With these test cases it is possible to extend positive test cases and include more extra checks
- Negative cases. Here we expect the application to gracefully handle problem scenarios with both valid user input (for example, trying to add an existing username) and invalid user input (trying to add a username which is null)
- Authorization, permission tests
How to start with Functional Testing?
- Read API documentation / specification / requirements carefully to understand its endpoints, request methods, authentication methods, status codes and expected responses.
- Based on the functionality you are going to test, outline positive and negative test scenarios which cover use cases and some edge cases as well. Revisit Functional API Testing section for more details.
- Setup test environment: create a dedicated test environment that mirrors the production environment.
- Select an appropriate tool (for example, Postman, Insomnia), frameworks (for example, pytest, JUnit, Mocha), technologies, programming languages (Python, Javascript, Java, etc.) with appropriate libraries for API Testing.
- Plan Test Data: It is always important to populate the environment with the appropriate data.
- Write Automation scripts: Automate repetitive test cases, like smoke, regression suites to ensure efficient and consistent testing. Validate responses against expected outcomes and assertions, checking for proper status codes, headers, and data content.
- Test the API’s error-handling mechanisms: Verify that the API responds appropriately with clear error messages and correct status codes.
- Document Test Results: Maintain detailed documentation of test cases, expected outcomes, actual results to make onboarding of new team members easier.
- Collaborate with developers: it is important to have consistent catch-ups with your team and stakeholders to review test results and address any identified issues.
- Continuous Improvement: Continuously refine and improve your testing process based on lessons learned from previous test cycles.
- Feedback Loop: Provide feedback to the development team regarding the API’s usability, performance, and any issues encountered during testing.
Non-Functional
Non-functional API testing is where the testers check the non-functional aspects of an application, like its performance, security, usability, and reliability. Simply put, the functional test focuses on whether API works, whereas non-functional tests focus on how well API works.
End-to-end testing
In general, end-to-end testing is the process of testing a piece of software from start to finish. We are checking it by mimicking user actions. If it comes to API, it is crucial to check if APIs can communicate properly by making call like a real client.
Exploratory testing
You’re not done testing until you’ve checked that the software meets expectations and you’ve explored whether there are additional risks. A comprehensive test strategy incorporates both approaches.
Elisabeth Hendrickson, book “Explore It!”
When all automation and scripted testing is performed, it is time to examine an API, interact and observe its behavior. This is a great way to learn and explore edge cases to uncover issues that automated or scripted testing would have missed.
There are two ways of doing it:
- When test engineer performs it individually, He/She needs to apply domain knowledge intuition, critical thinking and user-centric thinking.
- There is another way — pair testing which involves two people: driver and navigator. It is a time-boxed testing when the driver performs the actual testing while the navigator observes, provides guidance, and takes notes where necessary. This approach maximizes a level of creativity and encourages knowledge sharing and better collaboration between team members.
More information: https://www.agileconnection.com/article/two-sides-software-testing-checking-and-exploring
Book “Explore It!”: https://learning.oreilly.com/library/view/explore-it/9781941222584/
BONUS:
Health Check API: transition to the cloud and refactoring of the applications to microservices introduced new challenges in effective monitoring these microservices at scale. To standardise the process of validating the status of a service and its dependencies, it becomes helpful to introduce a health check API endpoint to a RESTful (micro) service. As part of the returned service status, a health check API can also include performance information, such as component execution times or downstream service connection times. Depending on the state of the dependencies, an appropriate HTTP return code and JSON object are returned.
🎬 Conclusion
In conclusion, mastering the art of API testing requires a good approach that includes strategic planning, and continuous improvement.
Remember, API testing is not a one-time effort, but an ongoing process that evolves alongside your software development lifecycle. Continuous improvement is key to refining your API testing strategy. Regularly review and update your test cases, incorporating changes due to new features, bug fixes, or code refactoring. Learn from exploratory testing output, identify areas of improvement by listening to your customer’s and team’s feedback.
