Advance Your API Testing Using JSON Schema on Your Test Framework

Published in

DANA Product & Tech

5 min readJan 3, 2022

Introduction

API testing has been played an important role to ensure the quality of the API. Nowadays, many API testing tools can be used to perform API testing like Rest-Assured, SOAP UI, Postman, JMeter, etc. Somehow, we need a powerful tool to analyze API responses in detail and determine whether the JSON matches with the correct JSON format. This can be accomplished by validating the schema, that’s why JSON schema came as a solution. Schema validation ensures that the response returns from the endpoint match with a predefined set of rules. We used Rest-Assured as a study case on JSON schema implementation, things to know are that the implementation of the JSON schema can be used in any test framework. This article presents an overview of JSON schema, which contains core terminology, how to create JSON schema, and implementation in the test framework.

Definition

JSON Schema is a JSON media type for defining the structure of JSON data. JSON schema is intended to define validation of JSON data which can be used to validate both responses and request JSON. In JSON Schema, we can validate the data type, whether the field is mandatory, minimum length or max length, etc.

Example

Let’s pretend we’re interacting with a JSON-based employee biodata. The employee biodata has a couple of information such as:

An identifier: employeeId
An employee name: employeeName
Age of the employee: employeeAge
The job of the employee: jobTitle
An optional set of employee hobbies: hobby

The example of the above JSON can leave a few questions:

What is employeeId ?
what is the maximum character length for employeeName ?
what is the maximum value for employeeAge ?
is jobTitle required?
how many hobby that we can put?

JSON schema can provide a contract that describes each field clearly, we will try to generate the default first, by submitting the JSON of employee biodata in JSON Schema Generator. , the JSON Schema will be generated as follow :

at first, glance looks confusing, right? but don’t worry, it’s as simple as you creating a “Hello world” program. Let’s take a look at each term :

The $schema keyword states that this schema is written according to a specific draft of the standard and used for a variety of reasons, primarily version control.
The $id keyword defines a URI for the schema and the base URI that other URI references within the schema are resolved against.
The title and description annotation keywords are descriptive only. They do not add constraints to the data being validated. The intent of the schema is stated with these two keywords.
The type validation keyword defines the first constraint on our JSON data and in this case, it has to be a JSON Object.

In JSON Schema terms we define each property:

The properties validation keyword.
The employeeId key.
description and type schema annotation is clearly defined in previous.
The required validation keywords listing are employeeId,employeeName,employeeAge,jobTitle,hobby .
The examples keyword means the examples of value or object that can be placed.
The default keyword means the default value for the field.

We can also provide additional properties such as minLength, maxLength,exclusiveMinimum , etc.

The age of employee can’t below 20, we can use exclusiveMinimum keyword validation
Minimum length of character for jobTitle is four-character, minLength keyword validation can be used to specify it.
The hobbies must provide uniquely, the uniqueItems keyword validation can be used.

JSON Schema Implementation on the test framework

There are so many JSON schema libraries, we used this library in our project. Let’s create a new maven project to implement JSON schema on our test framework. First, we need to add the required dependencies on the pom.xml file as shown below :

We have two java classes that will be used First, JsonSchemaUtils.java contains JSON schema validation, and the second is APITest.java which contains the API testing code. The project structure is shown as below picture :

This class contains a method (checkJsonSchema) with two-parameter

jsonSchemaPath path file for JSON schema that we used.
jsonSubject, the JSON which will be validated by JSON schema.

This is an example of the implementation checkJsonSchema method, as we can see in the following class APITest.java, the method checkJsonSchema was called in that java class, it used to validate the response that was received when API www.boredapi.com/api/activity/ called.

Let’s take a look, how the JSON schema work by running this test case

Run the test.
Wait for the result

As shown above picture the result is passed, it can be inferred that the JSON subject passed the JSON schema validation, as we can see there is no validation exception error message in the log.

Let’s assume there is a change from dev, a little bit changes for the key field, the dev team decided to add one more character length on that field.

before → “key”: “5262759”
after → “key”: “52627590”

We want to make sure key length can be validated by adding minLengthand maxLength key validation in keyproperties.

Rerun the test case with similar step likes previously, the result shown as below picture :

The test is failed, as shown in the above picture the error cause validation exception which means the length of keyvalue is not qualified based on JSON Schema. As a QA, we can report this incident to the dev team to resolve this problem.

On the other hand, we can also use JSON schema to validate the request which will be sent to an API. Currently, our team in DANA has been implementing this thing. We have a unified common message used internally which will translate the message to the following third-party API specs, and the result of the translation will be validated by the JSON schema.

Final Words

JSON schema was a versatile library that can help us in performing API testing, the ability to define the schema requirements with a JSON file show how powerful this library is. Hopefully, these examples give you some ideas of how you might use schema validation on your projects.