Visual Component Testing with Storybook
I have been actively involved in frontend development for over six years, primarily working with Angular, and I have also gained about a year of experience with React. Throughout this extensive period, we have employed Jasmine and Karma for Angular and Jest for React when testing our components. A common limitation shared by both of these testing libraries is that they do not provide a rendering capability that allows developers to visually inspect what they are testing or how the component behaves during the test execution. Consequently, I have observed instances where tests fail to assess the intended aspects, and in some cases, tests end up examining nothing at all.
Testing components has proven to be a significant challenge for our backend developers who wish to contribute to the frontend. While they are eager to engage in frontend development and complete end-to-end (E2E) user stories that span both back- and frontends, they often struggle with composing component tests. This difficulty leads even our frontend developers to test component logic within the context of E2E tests rather than isolating it, mainly because crafting E2E tests is more straightforward due to the visual feedback they provide.
Why Storybook?
This year, we initiated the development of module-federated web-components to seamlessly integrate with products from other teams in real-time. This necessitated the creation of comprehensive documentation for our web components, ensuring that other teams could integrate them without the need to contact us for clarifications. Given that Storybook was already a widely adopted tool within our organisation, we opted to continue using it for this purpose.
We were simultaneously in the process of introducing Playwright for end-to-end (E2E) testing. Interestingly, we discovered that Storybook incorporated visual component testing with Playwright as part of its functionalities, referred to as interaction testing. This serendipitous discovery aligned perfectly with our requirements.
At the time, we had also considered two other options for visual component testing: Cypress and Playwright. However, Cypress fell out of favour due to past experiences that I have detailed in another article. Additionally, Playwright’s component testing feature was still in an experimental phase, which made us apprehensive about introducing it into an enterprise-level application.
Ultimately, we concluded that it made more sense to combine web component documentation and testing. This approach ensured that no developer would overlook the documentation for new components, as it would naturally emerge as a by-product of the testing process.
Interaction testing benefits
Visual component testing with Storybook has proven to be remarkably straightforward and user-friendly. We have harnessed its capabilities for a wide spectrum of components, ranging from lean and straightforward UI elements to more substantial ones, which involve intricate UI and logic. During the past few months, we have reaped several advantages from this approach, which we would like to highlight:
- Ease of writing tests: Storybook renders the component in the same manner as a typical web application would, which simplifies the process of writing tests and use cases, referred to as “stories.” This approach allows for straightforward, isolated testing of each component using Jest syntax, akin to Playwright, all while providing real-time visualisation of the component’s behaviour.
- Component documentation: The input and output attributes of a component are well-documented, and they can be conveniently modified from the Storybook user interface. Any changes made take immediate effect, and the component is rendered based on its attribute inputs.
You have the flexibility to let Storybook auto-detect your component’s inputs and outputs, which leads to the automatic generation of the controls, as shown in the screenshot provided. We have adopted this approach for our internal components, which are exclusively used within our team. However, for web components used by other teams, we have opted to provide more extensive documentation. In such cases, Storybook enables you to add additional documentation to each control and even override its type, should that level of customisation be necessary.
- Live interaction: Since the attributes can be modified, and the component is rendered and can be interacted with, we also have a view of all events (outputs) that are triggered by the component.
- Debugging: Running through the test step-by-step is very easy, using the back-and-forth arrows provided in the
Interactionstab. We also have our normal browser developer tools available for debugging.
- Mocking APIs: You can choose to let API calls be executed as normal, or you can mock them. There are a few different libraries available to do this. We are currently using a library called
storybook-addon-mockthat shows what is being mocked as part of the UI; this provides a fantastic overview of your mocks. You can also modify the API response or HTTP code, the response delay, and enable/disable the mock via the UI. We mostly use this for our module-federated web components. In our case, these are complex components that perform HTTP calls.
- Headless execution: While it is convenient that all of these tests run on a browser while you are developing and debugging, you will most likely want to run your tests from the command line in your CI pipeline. The library
@storybook/test-runnerprovides everything you need. - Parallel runs: Storybook’s test-runner library supports parallelisation out-of-the-box. While I have not performed a full benchmark test on its performance, I can confidently say that our tests, at least, are almost twice as fast when we use double the workers in our pipeline (which our slow Jenkins seems to be able to handle).
- Test coverage: Storybook’s test-runner library generates
lcovreports for tests if you use the--coverageflag when running your tests. We then merge this report with our other test coverage reports to provide a unified report usingistanbul-merge. - Console logs: Storybook’s test-runner library shows all browser console logs in the terminal. This helps with debugging tests that may fail only in the pipeline.
Interaction testing shortcomings
Although Storybook has made significant strides, and we would not revert to our previous methods of crafting visual component tests, there are certain facets of Storybook that could benefit from further enhancement. Here are a few areas that could be improved:
- Error testing: Storybook currently silently swallows application exceptions. This means that if you would like to test an
ErrorBoundarybased on some exception being thrown, you are out of luck. We were also not able to test how our web components behave if an HTTP call fails (allowing the real HTTP call to occur); instead, we had to mock the response as shown on the screenshot below. This is because if we let the real HTTP call be executed and the backend were to respond with anything other than a 2xx HTTP status code, the request promise would never complete, and the test would fail with a timeout.
This is a fairly major issue for us because we would have liked to test the error use case integration with the backend.
- Storybook syntax: The syntax for creating stories, documentation, and tests in Storybook is quite distinctive and does not adhere to the more conventional
describe/itsyntax employed by other widely recognised testing libraries. It is essential to note that adapting to this novel syntax can be a learning curve and may take some time to become familiar with. This adjustment period should be factored into your evaluation when considering whether Storybook is a suitable tool for your team’s needs.
// ---------------------------------------------------------------------------
// Define decorators for your stories, this one adds your app theme to
// the components in Storybook
export const withTheme: Decorator = (StoryFn) => {
return (
<ThemeProvider theme={theme}>
<StoryFn />
</ThemeProvider>
);
};
// ---------------------------------------------------------------------------
// Define the story as the default export
// With this configuration, Storybook will auto-detect and create the controls
// based on the component props
export default {
title: 'Internal-Components/ThousandSeparatorNumberFormat',
component: ThousandSeparatorNumberFormat,
decorators: [withTheme],
} as Meta<typeof ThousandSeparatorNumberFormat>;
type Story = StoryObj<typeof ThousandSeparatorNumberFormat>;
// This is the template for all upcoming stories
const Template: StoryFn<typeof ThousandSeparatorNumberFormat> = (args) => (
<TextField
inputProps={{ ...(args as object) }}
InputProps={{
inputComponent: ThousandSeparatorNumberFormat as any,
}}
/>
);
// ---------------------------------------------------------------------------
// Use-case: thousands formatting with suffix without max value
export const WithoutMaxValue: Story = Template.bind({});
WithoutMaxValue.args = {
name: 'Thousands Number Formatter',
suffix: '$',
customTextIndent: '2px',
disableMaxValue: true,
customDataTestId: 'thousand-separator-number-format-input',
};
WithoutMaxValue.play = async ({ args, canvasElement, step }) => {
// your component will be rendered within the storybook canvas area
// if you are testing modals, this will not work and you will need to use
// screen.getByTestId(...) instead of canvas.getByTestId(...) where:
// import { screen } from '@storybook/testing-library';
const canvas = within(canvasElement);
// we use steps to group tests in small components like this one, or
// in more complex components, to group logical test steps
await step('Each digit triggers a change event', async () => {
userEvent.clear(canvas.getByTestId(args.customDataTestId ?? ''));
userEvent.type(canvas.getByTestId(args.customDataTestId ?? ''), '123456');
expect(canvas.getByDisplayValue('123,456 $')).toBeInTheDocument();
await waitFor(() => expect(args.onChange).toHaveBeenCalledTimes(6));
await waitFor(() =>
expect(args.onChange).toHaveBeenLastCalledWith({
target: {
name: args.name,
value: '123456',
},
})
);
});
};
- Lack of beforeEach/afterEach: Storybook lacks a built-in mechanism for executing code before and/or after each test. However, there are workarounds available that involve using decorators to encapsulate and customise your component under test, enabling you to implement code to run before each test as needed. It is important to note that while workarounds exist for executing code before each test, there is currently no known workaround for running code after each test in Storybook.
- Reporting: While Storybook’s test-runner does offer coverage reports and JUnit reports, these provide only limited support for debugging when failures occur in the CI pipeline. What sets Playwright apart is its comprehensive and detailed out-of-the-box report, encompassing screenshots, videos, and the complete DOM for effective debugging. Storybook currently lacks a report that consolidates all the ongoing tests, their execution times, and — most crucially — detailed information for failed test cases. This information could include screenshots, videos, access to all Storybook tabs, and more. Incorporating such a comprehensive reporting feature would greatly enhance the debugging capabilities of Storybook.
- Test run retry: Storybook does not provide a way of retrying individual failed tests. Both Jest and Playwright do provide configuration options for this, but Storybook does not.
- Speed: As anticipated, rendering the components in your tests does indeed result in increased test execution times. In our pipelines, we have observed that these tests take approximately three times longer to run compared to Jest tests, although we have not conducted in-depth benchmark tests and we are allowing HTTP calls to be executed in some of these Storybook tests. Despite the decrease in speed, our team finds that the advantages brought about by visualisation outweigh this performance trade-off.
- Headless run needs Storybook to be running: When executing Storybook tests via the command line, it is currently necessary to manually start the Storybook application and confirm that it is running smoothly before running the tests. It would be more convenient and efficient if the process of spawning Storybook were integrated into the runner command itself, simplifying the workflow for developers.
Conclusion
Testing UI components can be a challenging endeavour when you lack the ability to visually inspect their appearance at each stage of the testing process. This level of abstraction can lead to situations where developers inadvertently test aspects other than what they originally intended. The advent of visual component testing frameworks has significantly improved the developer experience when it comes to testing components. Even if these tests may run slower, the increased accessibility of testing motivates developers to write more comprehensive tests, enhancing the overall quality of the software.
We will continue to keep an eye on the progress in this field to see how it evolves, especially considering that the current visual component testing frameworks still have significant room for improvement. For now, we are content with Storybook and plan to persist with its adoption for our new frontend repositories, even though not all of them incorporate web components.
Note: This article refers to Storybook version 7.5
