Introduction to Contract Testing
My original blog on our company website (in Norwegian):
https://www.bouvet.no/bouvet-deler/introduksjon-til-contract-testing
What is Contract Testing and in which situations can Contract Testing help us?
Contract Testing is a method of software testing that focuses on verifying the interaction between different components, services or systems in a distributed architecture. This approach is beneficial in scenarios where multiple services or components are developed and maintained by separate teams, and it is critical to ensure that they communicate and work together correctly. In short, Contract Testing is a method to ensure that two separate systems (such as two microservices) are compatible and can communicate with each other.
From a previous role in a team that developed a backend system with multiple interfaces, both with the front-end and with other internal and external parties, I have gained a lot of experience with this type of testing. In order to be able to work together on the same product, but independently of each other, we created these contracts, so that we could agree on what we were going to develop before we developed it. This meant that both parties involved could create mock-ups based on the contract and build and test independently before the other party had finished its development.
What is Contract Testing?
In Contract Testing, a contract is a formal specification of expected behavior and communication rules between two or more components, services or systems. It defines inputs, outputs and interactions to ensure that the parties involved meet each other’s expectations. A contract usually consists of the following elements:
- Interface: The interface is the set of methods, functions, or API endpoints exposed by a service or component for other services to use. It specifies the names, parameters, and return types of these methods or endpoints.
- Data format: The data format defines the structure and types of data exchanged between the services or components. For example, in a RESTful API, the data format could be JSON or XML, and the contract would specify the schema for the expected request and response payloads.
- Prerequisites: The prerequisites are the conditions that must be met before a method, function or API endpoint can be called. These conditions may include input validation, authentication, or certain system states that must exist prior to a service interaction.
- Postconditions: Postconditions describe the expected state or result after a method, function, or API endpoint is executed. This may include the expected response format, the status of any data created or modified, and any side effects that should occur as a result of the interaction.
- Invariants: Invariants are the rules or conditions that always apply throughout the interaction between components or services. They define constraints on the system’s behavior that must be maintained during the execution of the contract.
- Error handling: The contract should specify how errors or exceptions are to be handled and communicated between the interacting services. This includes defining error codes, error messages and any expected recovery mechanisms.
By clearly defining the contract in Contract Testing, development teams can create tests to verify that each component or service adheres to the specified behavior and communication rules. This helps to ensure smooth integration and correct functioning of the overall system.
When you search the internet, you quickly find the abbreviations CDCT and PDCT, which stand for Consumer Driven Contract Testing and Provider Driver Contract Testing. Consumer-Driven means Contract Testing from the point of view of the interface consumer. The consuming party states its needs and expectations, and the supplier or publisher must ensure that these needs and expectations are met. In this case, the consumer is in the driver’s seat when it comes to defining the interface contract. When a Provider-Driven approach is used, the provider (often called the publisher) creates the contract, and consumers have to deal with what the provider offers.
The type of Contract Testing I have been involved in during my last assignment was one where both consumer and supplier communicated a lot to get the best and most efficient solution for both parties. Neither party was in the lead or pushed through demands.
In what situations can Contract Testing help us?
Imagine a large e-commerce platform that is developed using a microservices architecture. The platform consists of different services such as user management, inventory management, order processing, payment processing and shipment tracking, each managed by a separate development team. The services communicate with each other via APIs and have well-defined contracts that specify the input, output, and behavior of each interaction.
In this scenario, Contract Testing can be very beneficial:
- Ensures consistency: It helps ensure that each service adheres to the agreed contract, which guarantees the consistency of communication between services.
- Reduces integration problems: By verifying that each service fulfills its contract, contract testing can minimize integration problems that can arise due to mismatched expectations or faulty implementations.
- Simplifies maintenance: When a service is updated, contract testing can detect any unintended changes to the contract, helping to prevent potential problems in other dependent services.
- Enables parallel development: Since contracts act as a well-defined agreement between the services, separate teams can develop and test their services independently, relying on the contracts for integration.
- Promotes Continuous Integration and Delivery: Contract-based testing can be included in the CI/CD pipeline, enabling development teams to catch and fix issues early in the development process, before they reach production.
In summary, Contract Testing is beneficial in scenarios such as microservice architectures, where multiple services or components interact, and ensuring that these interactions work properly is critical to the reliability and performance of the overall system. However, it is not limited to microservices architectures. Any situation where teams need to create or update an interface can benefit from creating clear contracts. That way, all parties involved know what to expect, and changes that break the contract can be linked to the agreed contract.
Example contract
Imagine we have a simple Wallet API, with one endpoint where we can deposit money into our wallet. Our /deposit endpoint has the following JSON request body:
{
“amount”: 10.05
}
The “amount” parameter has a value of 10.05 and is therefore a decimal number with 2 digits of precision, so a “double” as it is called in OpenAPI 3.0 documentation.
We determine that the “amount” parameter is required and must be a positive number with a minimum value of 0.01. If this is not the case, an HTTP code 400 error should occur.
We also decide that the deposited amount should increase the wallet balance by the same amount.
The /deposit endpoint will respond with a 200 (success) code, and the following JSON response body:
{
“balance”: 10.05
}
The parameter “balance” also has an example value of 10.05 and will therefore also be defined as a “double” in our specification.
We decide that the wallet’s “balance” must never be a negative amount.
Below is an example of an OpenAPI 3.0 specification for this simple Wallet API. This API has a single endpoint (/deposit) that allows a user to deposit money into the wallet and return the updated wallet balance. I personally like to use swagger.io to model my contracts. The visual display of OpenAPI 3.0 specifications is also very useful for making the contract readable. Many tools (including Confluence) offer plug-ins for this type of documentation, allowing it to be displayed in its visual state, making the contract more readable for all parties involved.
Within this contract/specification we can define the following elements:
- Interface: The /deposit endpoint allows users to deposit money into their wallet.
- Prerequisite: The amount parameter must be specified, and it must be a positive number (minimum value of 0.01).
- Data Format: JSON is used as the data format for request and response payloads. The form of the request body is Deposit, and the form of the response body is Wallet.
- Post condition: After a successful deposit, the wallet balance is increased by the deposited amount.
- Invariant: The wallet balance must never be negative (indicated by the minimum value 0 in the Wallet form).
- Error handling: If the prerequisite is not met (eg the amount is missing or negative), the API returns a 400 Bad Request error.
This simple Wallet API demonstrates how to include prerequisites, data formats, postconditions, invariant conditions, and error handling in an OpenAPI 3.0 contract. If two teams were to implement this API (a publishing team that creates this API, and a consuming team that will use this API in their application), they can now work separately because the contract is defined and agreed upon. Both teams can individually create simulated requests and responses based on this contract and ensure that their part of the software works before connecting the actual components.
See below the more “readable” view of this contract, which is generated by the swagger.io editor.
How can we use contracts for Contract Testing?
What do we do with these contracts in terms of testing? One way to proceed is to manually check whether a request or response complies with the agreed contract. But this is a time-consuming and tedious way to do it, and doing this by hand is inefficient. Many tools now offer opportunities to check a request or response against a form. This exists for both SOAP and REST services. In addition, there are online tools where you can enter a request/response body and a form, where the tool validates whether the request/response complies with this form. But it’s even better to use this in your automated tests, for example adding them to your Postman test suites using, for example, Ajv form validation.
When using form validation, we see three possible results:
a. Both the request and response body comply with the agreed contract
b. Request body does not comply with the agreed contract
c. Response body does not comply with the agreed contract
In scenario b, the problem lies with the consumer of the service. The consumer should provide a request body that adheres to the agreed contract, because this is what the issuing party expects and analyzes to generate a response.
In scenario c, the problem lies with the publisher of the service. The response is created or generated by the publisher of the service and should always comply with the agreed contract.
In either case, however, it can be confusing when an error occurs with either the request or the response. Since both parties build software and react to each other, it will not be immediately clear which party is at fault. Let’s say a request is rejected by the API because the request body is incorrect. Then it could be that the consuming party made a mistake when setting up the request body, but it could also be that the issuing party made a mistake in the expected request body. The same applies to the response; it can be an error on the part of the issuing party by creating an incorrect response, but it can also be an error on the part of the consumer by misinterpreting the expected response. This is exactly why a contract is so important, because without a contract both parties have an equal say in what they thought or interpreted in terms of code that should be implemented.
I hope this blog will help you in your QA journey. In one of my next blogs, I’ll delve into how to use these contracts in your Postman tests, so you can perform contract-based testing with this tool!
About me
My name is Wessel Braakman and I’m a consultant at Bouvet in Norway. My aim is to share knowledge with others and to get others excited about QA, Low-Code and new technologies. If you have any questions about these topics, don’t hesitate to contact me through e-mail or LinkedIn!
