Overview
API testing is a type of integration test which is high value testing for SaaS (Server as a Service) products. As an integration test, the difficult challenge is component dependency. A good integration test case is well to isolate connections between multiple components or APIs. Besides, establishing hermetic environments is good to increase testing efficiency and confidence for integration tests.
This document is helping to build up a pytest architecture. The main models will be separated as Test case Module, Logic controller and Validation controller. Basically all controllers should be working with a main purpose. In order to help developers who should focus on each specific purpose to avoid illogical or mess during the test case design and coding.
Architecture
Test Case Module
A human readable interface to develop test steps into this module.
In this object (class) we are focusing on the user scenario, each test case contains clear steps with actions or validation. As testing framework, we can utilize “fixture” of pytest feature to design setup and tear_down mechanism during the test case module. Besides, you can define life cycle and scope with these mechanism as high flexibility.
The following example is showing you how is the Test Case Module looks liked
Logic Controller
This design pattern is similar to the page object pattern. If a web page is a unit as class module, An API is the same concept as the page object. Each API should contain specific methods and functions. Besides, we have to mitigate the dependencies between two APIs. The best practice is try to isolate each API. The advantages are following:
- Each API Logic controller won’t be impacted by other API interface.
- API Logic controller can be reused by multiple test case module.
- If any changes on specific APIs, the revising costs of logic controller is low.
- It’s easily to drop the logic controller, when API has been retired.
A standard restful API might having multiple http methods to use which likes GET, POST, PUT, PATCH, DELETE … etc. To construct a logic controller is based on the naming of routed URL which is likes:
https://api.github.com/users/weigo
According to last path in url (above), we will use “users” as name of logic controller, then we will construct two classes as class Users(Base) and class UsersAssertion(BaseAssertion).
class “users” is inherited by class “Base”. All of functions in class “users” are focusing on specific API function, data structure and any actions in the same API function.
class “Base” is created for general or basic API share functions, for example: request sender implementation, payload handler or response handler.
Validation controller
Ideally, every APIs have owned validation logic, therefore we expected each logic controller having a validation class. Besides, we have to clearly separate the function action layer and test case assertion layer to keep each controller clearly working on pure objective. in other hand, sometimes, API has a complex validation steps, if we implement them into the same place, it’s might not easy to maintain our test cases. By this design pattern, we can make developer having a clearly mindset to implement the right code into the right place.
A validation class structure is the same as logic controller. All of validation class are inherited by “BaseAssertion”, which class could embed general validations function or basic validation functions.
For example: assert for status code 200, status code 500, bad request 400 or content is not a Json format… etc.
Where is the best practice to implement assertion?
Assume each API response would having expected results and ideally each API should works standalone with clear input data. these validation logic would be placed in the “Validation controller”. and use pytest assert function to verify the results.
Sometimes, there are some unexpected response after sent through the function in logic controller, I suggest using the raise or try statement to raise unexpected behavior or ignore the noise, because the logic controller is focusing on the action, we don’t expected to many checks would be implemented in logic controller layer.
Make sure your API could be tested without any dependencies, to make your API input / output as standalone, there are 2 approaches:
- Mocked data
- Establish hermetic environment Reference : Google Testing Blog: Hermetic Servers (googleblog.com)
Test Data
In this practice, we are using “*.yml” to store pre-prepared testing data under
{root}/testdata/*.ymlwhen you trigger the pytest for a specific test suite, given a path as parameter in command line.
For example:
pytest ./testsuite/example_feature/test_get_users.py --env=STAGING --dataset={feature_folder}/test_data
Reference
You can download this Demo project to implement your owned API integration testing framework
Features:
- Support test environments
- Support api version changes : you can control which test case should be run by specific api version
- Support common utils module
- Support test data container
