API Testing: Using the Fluent Interface Pattern for JSON Assertions

When you are working with APIs and especially testing them, then there’s usually also the step of building some JSON request and asserting the response. And it often means complicated JSON structures with nested elements, lists, and all kinds of scurviness. The focus of today’s article is on the Asserters.

Disclaimer: All samples are in Java code, but can most probably be transferred to most other languages as well.

Disclaimer 2: I haven’t seen this pattern documented or used anywhere else yet. So if you are aware that this is already somewhere described and shared, please let me know in the comments!

The Problem

Let’s take an example from a simple CRM system that manages customer contacts. Here we retrieve the contact information for the Sample Company with ID 123.

request:
GET /api/v1/contacts/company/123

response:
{
  "id": 123,
  "companyName": "Sample Company AB",
  "address": {
    "id": 4254,
    "address1": "Main Street 1",
    "address2": "2nd floor",
    "postCode": "12345",
    "city": "Sampletown",
    "createdAt": "2020-08-21T13:21:45.578",
    "updatedAt": "2022-07-23T07:14:25.987"
  },
  "contacts": [
    {
      "id": 423,
      "contactName": "Gary",
      "contactSurname": "Henderson",
      "role": "Sales",
      "email": "gary.henderson@samplecompany.com",
      "phone": [
        {
          "number": "+123456789-0",
          "type": "LANDLINE"
        },
        {
          "number": "+43123456789",
          "type": "MOBILE"
        }
      ],
      "createdAt": "2020-08-21T13:21:45.578",
      "updatedAt": "2022-07-23T07:14:25.987"
    },
    {
      "id": 378,
      "contactName": "Henriette",
      "contactSurname": "Garstone",
      "role": "CEO",
      "email": "henriette.garstone@samplecompany.com",
      "phone": [
        {
          "number": "+123456789-12",
          "type": "LANDLINE"
        }
      ],
      "createdAt": "2020-08-21T13:21:45.578",
      "updatedAt": "2022-07-23T07:14:25.987"
    }
  ],
  "createdAt": "2020-08-21T13:21:45.578",
  "updatedAt": "2022-07-23T07:14:25.987"
}

In my past 10 years since first getting into contact with APIs, I have seen a lot of test code that looks like this, and I have probably written my fair share of it.

@Test
public void firstTest() {
    Company company = MagicCRMRestRetriever.getContact(123);

    Assertions.assertEquals(123, company.id);
    Assertions.assertEquals("Sample Company AB", company.companyName);
    Assertions.assertEquals("Main Street 1", company.address.address1);
    Contact salesContact = company.contacts.first { it.role == "Sales" };
    Assertions.assertEquals("Gary", salesContact.contactName);
    Assertions.assertEquals("Henderson", salesContact.contactSurname);
    Phone salesContactMobile = salesContact.phone.first { it.type == PhoneType.MOBILE };
    Assertions.assertEquals("+43123456789", salesContactMobile.number);
}

And this is just a small example of an assertion orgy. When I first learned about the pattern I’m about to introduce to you, the project context was Airline industry, and the JSONs contained flight and ticket information. We talk about at least 100-150 entries, easily. Imagine the big blocks of assertions.

Those of you who say, just use a comparator, create the payload you expect and compare it with the one you get. Well, I don’t care about all fields in all cases. And that makes it easy to get distracted in the test you write. I want to focus on the fields that are of interest to me.
See for example the createdAt and updatedAt, and IDs of the contacts and address fields.

The Solution: Fluent Asserters

Already in 2005 Martin Fowler wrote about the FluentInterface. But as my main focus from 2003 – 2012 was on Oracle PLSQL, SAP ABAP, Excel VBA, and other non-sense I have never seen again since, my knowledge about design patterns was a bit outdated. One of the developers in the aforementioned project came up with the nice idea of introducing Fluent Asserters. Attention: This is not the same kind of fluent assertions that are possible with e.g. AssertJ, where you can chain assertions on the same variable.

First I will show you how sweet these Fluent Asserters can look in action, before I then scare the hell out of you about implementation details.
Here is the sample of the same test as above.

@Test
public void firstTest() {
    Company company = MagicRestRetriever.getContact(123);

    AssertCompany.instance(company)
      .id(123)
      .companyName("Sample Company AB")
      .address()
        .address1("Main Street 1")
        .done()
     .contacts()
       .byFilter(c -> c.role == "Sales")
         .contactName("Gary")
         .contactSurname("Henderson")
         .phone()
           .byType(MOBILE)
             .number("+43123456789")
             .done()
         .done()
       .done();
}

Doesn’t it look nice and neat, with all those indentations, that let it look a bit like a beautified JSON file.

If you like what you see, it might be worth to read on and get through all the nasty little details necessary to implement this kind of asserters. If you don’t like it, you better stop reading here, if you haven’t done already so.

The Asserter Interface and Chained Asserter Interface

The basic problem that needs to be solved is the chaining of DTOs. In our simple example above, the JSON is combining four different DTOs. And if you have a good API design it might be that certain DTOs are re-used all over the place. How do you distinguish which one to use. Well, you don’t have to, when all DTO asserter classes inherit from the Asserter Interface and Chained Asserter Interface.

public interface Asserter<TItem> {
    TItem get();
}

public interface ChainedAsserter <TItem, TParent extends Asserter> extends Asserter<TItem> {
    TParent done();
}

public abstract class AbstractChainedAsserter<TItem, TParent extends Asserter> implements ChainedAsserter<TItem, TParent>  {
    private final TItem item;
    private final TParent parent;

    protected AbstractChainedAsserter(final TItem item, final TParent parent) {
        this.item = item;
        this.parent = parent;
    }

    @Override
    public TItem get() {
        return item;
    }

    @Override
    public TParent done() {
        return parent;
    }
}

The ChainedAsserter is the heart piece, because it allows you to return back the parent (done()) in a way that the Asserter doesn’t need to know what type that parent is. This allows to embed the same DTO into several others without the need to distinguish between Asserters, because the parent is set on navigation through the DTO. As long as the parent is, of course, also an Asserter. The highest element must always be an Asserter.

The Implementation of the Asserter

When it comes to the actual implementation, the Asserter has to extend the AbstractChainedAsserter.

public class AssertCompanyDto<TParent extends Asserter<?>> extends AbstractChainedAsserter<CompanyDto, TParent> { ...

The TItem is the DTO to assert itself, and the TParent is another Asserter.
Every Asserter has two methods to instantiate it, instance(target) and chainedInstance(target).
instance is called in the actual test, when the Asserter is instantiated with the actual variable to investigate and assert.
The chainedInstance is called only by Asserters, when they hand over to another Asserter. Both methods call the constructor, instance is not setting a parent, as it represents the highest node of the JSON, and chainedInstance is setting a parent. The constructor also asserts that the handed over object is not null.

public static AssertCompanyDto<Asserter<CompanyDto>> instance(final CompanyDto target) {
    return new AssertCompanyDto<>(target, null);
}

public static <TP extends Asserter<?>> AssertCompanyDto<TP> chainedInstance(final CompanyDto target, TP parent) {
    return new AssertCompanyDto<>(target, parent);
}

private AssertCompanyDto(final CompanyDto target, TParent parent) {
    super(target, parent);
    assertThat(target).describedAs("Company must not be null").isNotNull();
}

Standard types like String, Int, Date, etc. can directly be asserted. And as it is a fluent interface type of implementation, the method has to return the Asserter itself (this).

public AssertCompanyDto<TParent> companyName(final String expected) {
    assertThat(get().getCompanyName()).describedAs("companyName").isEqualTo(expected);
    return this;
}

Chaining Asserters

When an attribute of the DTO is another DTO, then the according method is not expecting an object as comparison, but returns the Asserter for the nested DTO. While you can implement also a simple object comparison method that handles comparing the whole DTO at once, I have rarely used them, and hence dropped them in later projects.
Now we see the chainedInstance in action. The return type is defined with itself as parent. Also we directly check that the object is not null, so that we actually have something to assert.

public AssertAddressDto<AssertCompanyDto<TParent>> address() {
    final AddressDto target = get().getAddress();
    assertThat(target).describedAs("Address").isNotNull();
    return AssertAddressDto.chainedInstance(target, this);
}

List Asserters

Some of the embedded DTOs might even be lists. In our example that is contacts and the phone numbers of the contacts. Lists need to be treated specially. Because we want to navigate in lists, select specific elements (byIndex), e.g. when you know exactly which element should be on first position, or you want to search for an entry based on a known attribute (byFilter). Or simply know how big the list should be (size).

public class AssertContactDtoList<TParent extends Asserter<?>> extends AbstractChainedAsserter<List<ContactDto>, TParent> {
    public static AssertContactDtoList<Asserter<List<ContactDto>>> instance(final List<ContactDto> target) {
        return new AssertContactDtoList<>(target);
    }

    public static <TP extends Asserter<?>> AssertContactDtoList<TP> chainedInstance(final List<ContactDto> target, TP parent) {
        return new AssertContactDtoList<>(target, parent);
    }

    private AssertContactDtoList(final List<ContactDto> target) {
        super(target, null);
    }

    private AssertContactDtoList(final List<ContactDto> target, TParent parent) {
        super(target, parent);
    }

    public AssertContactDtoList<TParent> size(final int expectedSize) {
        assertThat(get().size()).describedAs("expected list size").isEqualTo(expectedSize);
        return this;
    }

    public AssertContactDto<AssertContactDtoList<TParent>> byIndex(final int index) {
        assertThat(0 <= index && index < get().size()).describedAs("index must be >= 0 and < " + get().size()).isTrue();
        return AssertContactDto.chainedInstance(get().get(index), this);
    }

    public AssertContactDto<AssertContactDtoList<TParent>> byFilter(final Predicate<ContactDto> predicate) {
        final ContactDto by = by(predicate);
        assertThat(by).describedAs("list entry matching given predicate not found").isNotNull();
        return AssertContactDto.chainedInstance(by, this);
    }

    private ContactDto by(final Predicate<ContactDto> predicate) {
        return get().stream().filter(predicate).findFirst().orElse(null);
    }
}

So, when dealing with a list element in your DTO, then you don’t directly return the Asserter of the DTO, but first its ListAsserter. This following method is part of the AssertCompanyDto class.

public AssertContactDtoList<AssertContactDto<TParent>> contactList() {
    final List<ContactDto> target = get().getContacts();
    assertThat(target).describedAs("Contacts must not be null").isNotNull();
    return AssertContactDtoList.chainedInstance(target, this);
}

Extend as you wish

And you can of course extend the Asserters with anything that you need often. An example is that often you might not want to compare a String exactly, but only check that it contains a certain substring. So why not add to your normal exact comparison method a “Contains” method.

public AssertCompanyDto<TParent> companyNameContains(final String expected) {
    assertThat(get().getCompanyName()).describedAs("companyName").contains(expected);
    return this;
}

The Secret to success: write a Generator

Disclaimer: I cannot share any code as of now, as the generators that I have produced so far are part of special code bases that I cannot publish. If interest is big enough though, I will try to re-create one in a public GitHub project. Leave me a comment, so that I know there’s interest.

The code described above is complicated and producing it manually is error-prone. And also it’s stupid work. Nobody likes stupid work, except maybe me sometimes. So the easiest way is to write a generator that analyzes the given DTO and creates the Asserters and ListAsserters auto-magically.

The generator is using the power of reflection by looking into the innards of the class, analyzing what it consists of.

The generator needs to do basically do the following things:

• scan the parent class provided as input
  • find all attributes, and in case of Java e.g. also determine the getter
    • find out if the type of the attribute is a standard type (String, Int, Double, etc)
    • find out if the type is a custom type (in our example above, e.g. AddressDto)
    • find out if it’s a list
    • maps also work, but I haven’t found a proper way to automate that yet
  • custom types need to be scanned and analyzed (yes, recursive call)
• manage the scanned classes, to avoid multi-scanning
  • you should also check which Asserters you already have to avoid unnecessary work
• after collecting the work that needs to be done, start creating the Asserters and ListAsserters
  • use a template engine to provide the standard content for each class
  • add for each attribute the methods you want to add
    • at least the 1:1 comparison
    • String contains
    • timestamp with a potential diff (so that you can compare timestamps you only know roughly)
    • number formats with smallerThan or largerThan
    • and so on, whatever makes most sense in your project to always provide out-of-the-box
  • write the files to folder

Moving the files to their target directory is then again a manual process, to check that everything worked as expected. Then you can immediately start writing your tests with beautiful fluent assertions as shown on top of this post.

In case any DTO changes, just re-run the generator and it will add or remove changes. Be careful though, when you added custom assertions to your Asserters. IDEs with proper version control support should be helpful here to restore the according methods easily.

Famous last words

I am very sorry. This was now a very quick and dirty description of what the generator needs to do. But going into detail would be way too much for this anyway way too long blog post.

I hope that you understood the basic concept of the fluent asserters and find them as useful as I do. I don’t want to miss them anymore in any project where JSON/DTOs are getting a little bit bigger. It makes the tests so much more readable, writeable and maintainable. It focuses the tests on what is really important and not clutters them with technical details.