Framework Agnostic UI Testing
WHAT? Compose implementation & framework agnostic tests in a “black box” manner
WHY? Increase test resilience to implementation & functional changes
HOW? Using test helpers & the driver pattern
When developed correctly, our tests need not be affected by implementation details, allowing us to test the UI in a black box manner.
I will demonstrate how the same functionality implemented using three different frameworks (React, Angular, Lit) is tested using the same test code.
We will be testing a simple application — a Pokémon catalog. We can browse through all Pokémons, or jump to a specific index.
You can find the live demos here: React, Angular, Lit
The application consists of a pokemon-catalog component which uses a service to send http requests to a backend server to fetch Pokémon data.
The pokemon-catalog component is composed from other components such as the pokemon-image component.
So three components: catalog, image, and Go!
Let’s start with E2E tests. Front End E2E tests refer to a helper robot that behaves like a user to click around the application and verify that it functions correctly.
We’ll start with the Angular implementation E2E tests.
What will we be testing?
- Prev button should be disabled when showing the first Pokémon
- Correct Pokémon name should be displayed
- Correct Pokémon index should be displayed
- Correct Pokémon image should be displayed
And then the same three tests again when we click Next, when we click Prev and when we type an index and click on Go
And how about E2E testing of the Lit (web components) implementation?
What are we testing? Exactly the same things.
And the same test suites will run for the React implementation
OK, tests suites look similar.
What does the test code look like?
describe("Pokemon e2e", () => {
const { when, get, beforeAndAfter } = new AppDriver();
beforeAndAfter();
beforeEach(function () {
when.visit("/");
when.waitUntil(() => get.elementByText("bulbasaur"));
});
it("should update index", () => {
then(get.pokemon.countText()).shouldStartWith("3 of");
});
it("should update pokemon image", () => {
then(get.pokemon.image.pictureSrc()).shouldEndWith("/3.gif");
});
it("prev button should be enabled", () => {
then(get.pokemon.prevButton()).shouldBeEnabled();
});
it("should render pokemon name", () => {
then(get.elementByText("venusaur")).shouldExist();
});
...
});Same test code for all three frameworks!
Things to note here:
- Each test tests one thing and tests it well
- No logic in test code
- If a test fails — just looking at the test name in the test report can tell us what went wrong
How About Integration Tests?
Front End Integration tests refer to running the Front End application while mocking the server response.
Let’s run some integration tests, we’ll start with Angular Integration Tests.
What are we testing?
- Prev and Next buttons should be disabled when the first/last Pokémon are showing
- Correct Pokémon image should be displayed
- When fetching a Pokémon by index, an outgoing http request with the correct index should be fired.
Same test suite will run for the Lit implementation
Same tests will run for the React implementation
But what does the test code look like?
describe("Pokemon Page integration Tests", () => {
const chance = new Chance();
const { when, given, get, beforeAndAfter } = new AppDriver();
beforeAndAfter();
describe("given a single pokemon", () => {
const pokemonList: PokemonList = Builder<PokemonList>()
.results([{ name: chance.word(), url: "1" }])
.count(1)
.build();
beforeEach(() => {
given.pokemon.fetchPokemonResponse(pokemonList);
given.pokemon.fetchImageResponse("default.png");
when.visit("/");
when.pokemon.waitForPokemonLastCall();
});
it("should disable next button once showing last pokemon", () => {
then(get.pokemon.nextButton()).shouldBeDisabled();
});
it("should disable prev button once showing first pokemon", () => {
then(get.pokemon.prevButton()).shouldBeDisabled();
});
it("should render correct image", () => {
then(get.pokemon.image.pictureSrc()).shouldEndWith("/1.gif");
});
it("should fetch pokemon by index", () => {
when.pokemon.pokemonGo.typePokemonIndex("78");
when.pokemon.pokemonGo.clickGo();
then(get.pokemon.fetchPokemonQueryParams()).shouldInclude({ offset: "77" });
});
});
});Which framework? All frameworks! Same test code.
Note how the test code is phrased like a user story: given <pre-condition>, when <something takes place>, expect <actual result> should <expected result>
OK, so given the API is the same, integration tests can be the same. But can I do it for component tests as well?
Component Tests
When testing a UI component in isolation — it is referred to as Component test.
Let’s run some component tests. These are the component tests of the pokemon-catalog component Angular implementation.
And same tests for the Lit pokemon-catalog component
And for the React pokemon-catalog component
Same Test Code?
Well, not exactly, as the component has to be mounted to the browser.
BUT, that is the ONLY difference!
The only difference between frameworks is how we mount the component.
describe("PokemonCatalogComponent Tests", () => {
const chance = new Chance();
const { when, given, get, beforeAndAfter } = new PokemonCatalogComponentDriver();
beforeAndAfter();
describe("given one of many pokemons", () => {
const name = chance.word();
const pokemon: PokemonList = Builder<PokemonList>()
.results([{ name, url: "2" }])
.count(3)
.next(chance.url())
.previous(chance.url())
.build();
beforeEach(() => {
given.pokemon(pokemon);
given.image.mockImageResponse("default.png");
given.onNextSpy();
given.onPrevSpy();
when.render(PokemonCatalog);
});
it("should show picture given pokemon provided as input", () => {
then(get.image.pictureSrc()).shouldEndWith("/2.gif");
});
it("should render pokemon name", () => {
then(get.nameText()).shouldEqual(name);
});
it("should render pokemon count", () => {
then(get.countText()).shouldEqual("2 of 3");
});
describe("when clicking next", () => {
beforeEach(() => {
when.clickNext();
});
it("should emit onNext", () => {
then(get.onNextSpy()).shouldHaveBeenCalledOnce();
});
it("should call getPokemon with the next pokemon's url", () => {
then(get.getPokemonSpy()).shouldHaveBeenCalledWith(pokemon.next);
});
});
describe("when clicking prev", () => {
beforeEach(() => {
when.clickPrev();
});
it("should emit onPrev", () => {
then(get.onPrevSpy()).shouldHaveBeenCalledOnce();
});
it("should call getPokemon with the prev pokemon's url", () => {
then(get.getPokemonSpy()).shouldHaveBeenCalledWith(pokemon.previous);
});
});
});
describe("given single pokemon", () => {
const name = chance.word();
const pokemon: PokemonList = Builder<PokemonList>()
.results([{ name, url: "1" }])
.count(1)
.build();
beforeEach(() => {
given.pokemon(pokemon);
given.image.mockImageResponse("default.png");
when.render(PokemonCatalog);
});
it("should show picture given pokemon provided as input", () => {
then(get.image.pictureSrc()).shouldEndWith("/1.gif");
});
it("should render pokemon count", () => {
then(get.countText()).shouldEqual("1 of 1");
});
it("next button should be disabled", () => {
then(get.nextButton()).shouldBeDisabled();
});
it("prev button should be disabled", () => {
then(get.prevButton()).shouldBeDisabled();
});
});
});What’s My Secret? Helpers & Drivers
Test Helpers
Test Helpers are designed to decouple our tests from our testing framework.
Test Helpers expose three public properties: given, when and get, Wrapping most useful commands, so our tests are as decoupled as possible from our testing framework.
Helpers also steer us in the right direction in terms of test readability — forcing us to use the “given, when, get” terms in our tests.
In this demo I am using CypressHelper, designed to be used in any test level, and holds common methods used in Cypress tests
I am also using CypressComponent helpers which are designed to be used in component tests and are NOT framework agnostic, as they have to mount the component to the browser
Assertable & then
The Assertable class wraps a subject we wish to assert something about, for example, assert that an element is disabled
Assertable wraps Cypress.Chainable so that your tests are as decoupled as possible from Cypress.
By using the Assertable class, you can use the same assertions in your tests, regardless of the testing framework you use.
All you need to do if you wish to replace Cypress with another testing framework and keep your tests, is to replace the implementation of the Assertable class.
You can also add assertions of your own, by extending Assertable class.
Here is an example of adding an assertion that asserts the subject is included in the style stored in local storage.
const styleFromWindow = (win: Window) => {
const styleId = win.localStorage.getItem("maputnik:latest_style");
const styleItem = win.localStorage.getItem(`maputnik:style:${styleId}`);
const obj = JSON.parse(styleItem || "");
return obj;
};
export class MaputnikAssertable<T> extends Assertable<T> {
shouldEqualToStoredStyle = () =>
then(
new CypressHelper().get.window().then((win) => {
const style = styleFromWindow(win);
then(this.chainable).shouldDeepNestedInclude(style);
})
);
}Test Drivers
The driver pattern basically means that we have an additional class which is responsible for “bridging the gap” between our test file and our component.
It will help our tests be unaware of the inner works of a component.
In addition, if we change something in the component which causes the test to fail, we will just have to amend the driver and the tests will just pass again when the same logic works.
Every Driver exposes three public properties
given — The given property will hold methods which will allow us to set pre-conditions before something takes place. This is a classic place to have methods which will set the inputs which are going to be passed down to our component.
when — The when property will hold methods of “events” which will take place like render, click, hover, etc.
get — The get property will hold methods which will give our tests access to the “output” of the component in a “black box” fashion
Example
The pokemon-image component displays a gif image, unless there is an error, then it shows a fallback png image.
Here is the Lit implementation:
@customElement("pokemon-image")
export class PokemonImageComponent extends LitElement {
@property({ type: Number })
pokemonIndex!: number;
@state()
showFallbackImage = false;
onImageError = event => {
this.showFallbackImage = true;
};
getPokemonImage = () =>
`https://raw.githubusercontent.com/PokeAPI/sprites/master/sprites/pokemon/versions/generation-v/black-white/animated/${this.pokemonIndex}.gif`;
getFallbackImage = () =>
`https://raw.githubusercontent.com/PokeAPI/sprites/master/sprites/pokemon/${this.pokemonIndex}.png`;
protected override render() {
return html`
<div>
${this.showFallbackImage
? html` <img
data-hook="pokemon-fallback-image"
src="${this.getFallbackImage()}"
class="pokemon-fallback"
alt="pokemon"
/>`
: html`<img
data-hook="pokemon-image"
src="${this.getPokemonImage()}"
class="pokemon"
alt="pokemon"
@error="${this.onImageError}"
/>`}
</div>
`;
}
}You can find the full component implementation of all three frameworks here: React, Angular, Lit.
What do we want to test?
We’ll have one test asserting the gif is displayed given there is no error.
it("given valid pokemon index should show gif", () => {
const pokemonIndex: number = chance.integer({ min: 1, max: 500 });
given.pokemonIndex(pokemonIndex);
given.mockImageResponse("default.png");
when.render(PokemonImageComponent);
then(get.pictureSrc()).shouldEndWith(`/${pokemonIndex}.gif`);
});Note:
- Using the driver’s given, when & get properties
- No logic in the test
- Test’s readability & maintainability
- Test’s name indicates what went wrong if test fails
And a test asserting the fallback png image is displayed, given the gif image is not found.
it("given image not found should show fallback image", () => {
const pokemonIndex: number = chance.integer({ min: 501, max: 1000 });
given.pokemonIndex(pokemonIndex);
given.missingImage();
when.render(PokemonImageComponent);
then(get.fallBackImage()).shouldExist();
});And what will the test driver look like?
export class PokemonImageComponentDriver {
private helper = new CypressHelper({ defaultDataAttribute: "data-hook" });
private componentHelper = new CypressComponentHelper();
private props = {
pokemonIndex: 0
};
beforeAndAfter = () => {
this.helper.beforeAndAfter();
};
given = {
pokemonIndex: (value: number) => {
this.props.pokemonIndex = value;
},
mockImageResponse: (fileName: string) =>
this.helper.given.interceptAndMockResponse({
url: "**/PokeAPI/sprites/master/sprites/pokemon/versions/generation-v/**",
response: { fixture: fileName }
}),
missingImage: () =>
this.helper.given.interceptAndMockResponse({
url: "**/PokeAPI/sprites/master/sprites/pokemon/versions/generation-v/**",
response: { headers: 404 }
})
};
when = {
render: (...) => {
this.componentHelper.when.mount(...);
}
};
get = {
pokemonImage: () => this.helper.get.elementByTestId("pokemon-image"),
fallBackImage: () => this.helper.get.elementByTestId("pokemon-fallback-image"),
pictureSrc: () => this.helper.get.elementsAttribute("pokemon-image", "src"),
fallbackPictureSrc: () => this.helper.get.elementsAttribute("pokemon-fallback-image", "src")
};
}As mentioned before, in order to mount the component, we need to use a component helper, which is not framework agnostic.
But the rest of the code is the same.
Note how the driver uses helper functions to implement the methods required to drive the test.
Theoretically, if we switch to another testing framework, the change will be relatively easy, as we will implement a helper for the new testing framework, keeping the driver very similar to it’s current implementation.
Driver Composition
Test drivers can be composed to drive the test of a component composed of multiple sub-components. This way, test drivers can be used as building blocks to compose more elaborated test drivers. And we can eventually create a driver for an entire page in our application, made of many components.
In our example, the pokemon-catalog driver will hold a pokemon-image driver and expose it’s methods. We do not want to repeat ourselves, we do not want to implement again the methods already implemented in the pokemon-image driver
Let’s look at the test driver’s code:
export class PokemonCatalogComponentDriver {
private helper = new CypressHelper({ defaultDataAttribute: "data-hook" });
private componentHelper = new CypressComponentHelper();
private pokemonImageDriver: PokemonImageComponentDriver = new PokemonImageComponentDriver();
private pokemonGoDriver: PokemonGoComponentDriver = new PokemonGoComponentDriver();
private pokemonServiceMock: IPokemonService = {
getPokemon: url => Promise.reject(),
getPokemonByOffset: offset => Promise.reject()
};
private props: IPokemonCatalogPros = {
onNext: () => {},
onPrev: () => {},
service: this.pokemonServiceMock
};
beforeAndAfter = () => {
this.helper.beforeAndAfter();
this.pokemonImageDriver.beforeAndAfter();
};
given = {
image: this.pokemonImageDriver.given,
pokemonGo: this.pokemonGoDriver.given,
onNextSpy: () => (this.props.onNext = this.helper.given.spy("onNext")),
onPrevSpy: () => (this.props.onPrev = this.helper.given.spy("onPrev")),
pokemon: (value: PokemonList) => {
this.pokemonServiceMock.getPokemon = this.helper.given
.stub()
.as(this.pokemonServiceMock.getPokemon!.name)
.returns(value);
this.pokemonServiceMock.getPokemonByOffset = this.helper.given
.stub()
.as(this.pokemonServiceMock.getPokemonByOffset!.name)
.returns(value);
}
};
when = {
image: this.pokemonImageDriver.when,
pokemonGo: this.pokemonGoDriver.when,
render: (...) => {
this.componentHelper.when.mount(...);
},
clickNext: () => this.helper.when.click("next"),
clickPrev: () => this.helper.when.click("prev")
};
get = {
image: this.pokemonImageDriver.get,
pokemonGo: this.pokemonGoDriver.get,
onNextSpy: () => this.helper.get.spy("onNext"),
onPrevSpy: () => this.helper.get.spy("onPrev"),
countText: () => this.helper.get.elementsText("count"),
nameText: () => this.helper.get.elementsText("pokemon-name"),
nextButton: () => this.helper.get.elementByTestId("next"),
prevButton: () => this.helper.get.elementByTestId("prev"),
getPokemonSpy: () => this.helper.get.spyFromFunction(this.pokemonServiceMock.getPokemon!),
getPokemonByOffsetSpy: () => this.helper.get.spyFromFunction(this.pokemonServiceMock.getPokemonByOffset!),
pokemonServiceMock: () => this.props.service
};
}You can see that pokemon-catalog driver exposes the methods of the pokemon-image driver. We have already implemented a method to get the image source of the pokemon-image, we do not wish to repeat ourselves, this way if there is a change, we only have to make it in one place.
Integration Driver
Integration Driver will be composed from all relevant component drivers ,in our case, the pokemon-catalog component driver, and will contain integration test related methods in addition, for example: methods for intercepting outgoing http requests and mocking responses.
export class PokemonPageDriver {
private helper: CypressHelper = new CypressHelper({ defaultDataAttribute: "data-hook" });
private pokemonDriver: PokemonCatalogComponentDriver = new PokemonCatalogComponentDriver();
beforeAndAfter = () => {
this.helper.beforeAndAfter();
this.pokemonDriver.beforeAndAfter();
};
given = {
fetchPokemonResponse: (response: PokemonList) =>
this.helper.given.interceptAndMockResponse({
url: "https://pokeapi.co/api/v2/pokemon**",
response,
alias: "pokemon"
}),
fetchImageResponse: (fileName: string) =>
this.helper.given.interceptAndMockResponse({
url: "/**/PokeAPI/sprites/**",
response: { fixture: fileName },
alias: "pokemon-image"
})
};
when = {
...this.pokemonDriver.when,
waitForPokemonLastCall: () => this.helper.when.waitForLastCall("pokemon")
};
get = {
...this.pokemonDriver.get,
fetchPokemonQueryParams: () => this.helper.get.requestQueryParams("pokemon")
};
}And as shown before, these are the integration tests — framework agnostic as promised.
describe("Pokemon Page integration Tests", () => {
const chance = new Chance();
const { when, given, get, beforeAndAfter } = new AppDriver();
beforeAndAfter();
describe("given a single pokemon", () => {
const pokemonList: PokemonList = Builder<PokemonList>()
.results([{ name: chance.word(), url: "1" }])
.count(1)
.build();
beforeEach(() => {
given.pokemon.fetchPokemonResponse(pokemonList);
given.pokemon.fetchImageResponse("default.png");
when.visit("/");
when.pokemon.waitForPokemonLastCall();
});
it("should disable next button once showing last pokemon", () => {
then(get.pokemon.nextButton()).shouldBeDisabled();
});
it("should disable prev button once showing first pokemon", () => {
then(get.pokemon.prevButton()).shouldBeDisabled();
});
it("should render correct image", () => {
then(get.pokemon.image.pictureSrc()).shouldEndWith("/1.gif");
});
it("should fetch pokemon by index", () => {
when.pokemon.pokemonGo.typePokemonIndex("78");
when.pokemon.pokemonGo.clickGo();
then(get.pokemon.fetchPokemonQueryParams()).shouldInclude({ offset: "77" });
});
});
});Note how most of the driver methods were already implement in the component driver.
And as shown before, these are the E2E tests:
describe("Pokemon e2e", () => {
const { when, get, beforeAndAfter } = new AppDriver();
beforeAndAfter();
beforeEach(function () {
when.visit("/");
when.waitUntil(() => get.elementByText("bulbasaur"));
});
it("prev button should be disabled", () => {
then(get.pokemon.prevButton()).shouldBeDisabled();
});
it("should render pokemon index", () => {
then(get.pokemon.countText()).shouldStartWith("1 of");
});
it("should render pokemon image", () => {
then(get.pokemon.image.pictureSrc()).shouldEndWith("/1.gif");
});
it("should render pokemon name", () => {
then(get.pokemon.nameText()).shouldEqual("bulbasaur");
});
...
});So we are using drivers as building blocks to create more elaborate test drivers using driver composition.
Summary
- The driver pattern enables decoupling of tests code and production code, so our tests can be unaware of the implementation details and focus on output.
- Using test drivers & helpers improves tests readability and maintainability.
- Test code uses the test driver, so that test code is unaware of the implementation details
- Test Driver uses test helper, so it is unaware of the testing framework
It may take some time to get used to using test helpers and test drivers. The best way to have an easy adoption is to engage in some hands-on practice.
You can find bellow the link to the repo with all three Pokémon apps, and you can practice a hands-on workshop as well, you’ll find the instructions in the repo.
Resources
Pokémon examples repo with workshop instructions
Happy Testing!