Helpshift Engineering
Why testing CSS is challenging & how we did it
In the middle of difficulty lies opportunity
Recently, we were working on improving our WYSIWYG text editor, and while styling it, we noticed some existing rules conflicting with the new ones. A deeper look revealed a vital flaw in our CSS bundling process, which caused many repetitive styles, and thus rules with conflicts.
We started looking for the root cause of this flaw & found it soon. One primary SCSS mixin — used by almost all of our CSS files — takes two arguments. One is the class name & the other is a boolean, controlling if CSS reset rules should be included in the current class scope. The default value of the second parameter was, True — which caused this issue.
I know what you are thinking. CSS reset styles are included once somewhere at the start! Why do component & page CSS files need to reset rules?
The answer, like most hard & unexplainable questions in software development — is for legacy reasons. A few years ago, we started a CSS revamp for our system. Until that process was complete, we had to keep two CSS style systems. In that case, there was a need for some container elements to have reset styles, so styles from older files won’t interfere.
We decide to fix the root cause & remove this duplication of styles. We hoped to get some reduction in CSS file size, in turn improving load performance.
The Fix — Easy part
We found that we could safely set the default value for the parameter in question to False. Wherever required, it was explicitly passed as True. To our surprise, this small change reduced our final CSS bundle by a third. In absolute terms, the total CSS size was reduced from 1.64 MB to 1.12 MB — approximately 32% savings. We had to add a few additional rules in style to ensure nothing was broken, but those were minor. This was a significant improvement, with a minor effort, and the solution was ready to go live. And here comes the problem…
We had to test this. It was an app-level change, so a round of thorough testing was needed. Although we have both automated & manual testing processes, those were seemingly not viable for this. The reasons are, that there are no good tools to test the output of CSS modifications, and manual testing CSS is very challenging.
Testing CSS — Hard Part
There are two primary reasons why testing a CSS change is hard.
The first is found in its name, CSS — Cascading Style Sheets.
Cascading means final styles for any elements are the result of all styles from that element & all its ancestor (containers) elements, combined with the browser user agent style rules. Thus, a text element can get its font family from the body, color from the parent paragraph, and alignment from the parent section.
Another issue is that CSS deals with styles — the visual aspect of web pages. And not all of these changes are easily detectable. The human eye can easily miss if one of the text elements changes font size from 16px to 14px in a busy admin dashboard or landing page. Detecting different shades of the same color is even more difficult.
We need to test the final result — the rendered output — to ensure our CSS is working as expected using some automated tool. It is worth noting that we are not replacing any existing testing tool, just adding one more layer of visual testing.
The Tests — Happy part
When we realized how huge & difficult testing for this optimization was, we were dumbstruck. We achieved such a massive improvement at a relatively low cost, but could not deploy it soon. We use Cypress for our UI automation tests, but the solution using Cypress was not very impressive. Cypress recommends Percy, but it is a cloud-based tool & we do not favor that way.
Luckily for us, around the same time when we faced this roadblock, our QA team discovered Playwright & had started using it. When we explored Playwright to see if it fits our use case, we were delighted to find that it was the solution we were looking for.
We quickly created a working POC for UI screenshot testing. The final result was a testing flow, in which we had to type just one command & it’d take screenshots from two different branches & compare those. Below is a rough schema diagram of how it works:
When screenshot testing in fired in the dev branch which you want to test against the master, it’ll check out to the master branch. There, it’ll prepare a static build, which will be served in the runtime browser launched by testing utilities. The next step is to take screenshots of all views specified in config & save those to a temp directory. Once this is done, it switches back to the dev branch & repeats the same process there. Once screenshots for both branches are available, a comparison is performed & the final result is printed to the console.
Future
We plan to refine this process & provide this as autil in our build tools. Likewise, combine it with our release flow. Details are yet to be decided, but we plan to use this to ensure CSS sanity in the final build. This will also provide us with the confidence to make any major improvements to our CSS codebase in the future.
