Making Test Frameworks Readable: The Domain Specific Language
What is a Domain Specific Language you ask? In the context of a test framework, a Domain Specific Language is a readable language that uses keywords for quick test development. This allows anyone to write automated tests without any prior programming knowledge.
Why I want to talk about this? Well, recently I dedicated my time to develop in a custom test framework of a big Belgian bank. And I noticed they used a custom DSL.
This Domain Specific Language allowed any third party to use the test framework within their own project and without the need for any prior programming knowledge.
This changed my view on how testing should be performed in medium to large size companies. It is not the first time I heard of it. Nor had I seen it implemented for the first time. The difference was, it was done in a smart way.
The idea of using a DSL in the test framework didn’t came out of thin air. There was a problem to be solved. Mainly a vast amount of people testing the applications within the company that had many different talents of which not all were programming related.
These people need to test the internal applications and public applications, without knowing anything about the used programming language in the test automation framework.
At that point a solution came into play: The Domain Specific Language.
For the company the use of a custom DSL made sure that anyone can get started with programming automated tests for their project. In our case it made sure that the non-programmers could start creating their own test scripts.
It also allowed employing people that are talented in different fields and not only programming or test (automation) focussed.
After telling you all this an example really shows what I’m talking about.
It is important to undestand the difference between a normal test script without DSL and one that uses DSL.
Giving an example in pseudo-code is the best way to show how it works.
A test without DSL: Ugly and hard to read
import applicationDriver
import applicationClicker
import AssertapplicationDriver.create()
theWindow = applicationDriver.getWindow("Welcome Screen")
buttonResult = applicationClicker.leftClickOnButton(theWindow, "theButtonName")
applicationDriver.close()Assert(buttonResult).isNotEmpty()
A test with custom DSL: Great readability and quick to maintain
import dsl
import theWindow
import AsserttheWindow {
perform leftClick on theButton
clickResult = getResult()
Assert(clickResult).isNotEmpty()
}
Is the above example with DSL the cleanest of examples? No not at all. With DSL it is all about abstraction to achieve readability.
So even more more abstraction layers can be added? Yes! For example an abstraction layer for Assert can be added. Changing the assertion to something like:
Assert clickResult toBe isNotEmpty
With the above explanation and the examples. You probably got an idea of what DSL is. Just to make sure you fully grasp it. I also have to explain what it isn’t.
A DSL is not equal to the well known “Gherkin” used in many test frameworks. That’s because Gherkin fits within the DSL spectrum. With our example we showed that a DSL can consist of personally defined keywords rather than for example only following the Gherkin syntax of “Given”, “When” and “Then” keywords.
DSL also doesn’t completely get rid of programming woes. It still requires a bit of programming understanding from the DSL user and does require some maintaining and updating. Depending on the project size and its dependencies on different platforms, these tasks can get quite big.
The use of a DSL is not something that just instantly checks the Behaviour Driven Development (BDD) checkbox for your project. BDD is also a mindset. Using a DSL still means you have to create the test cases from which you can create the DSL-based test scripts. A DSL also comes in handy in conjunction with popular tools like Cucumber to write your test scenarios.
I hope, by example, you understood what all the above meant and what kind of results it could have within your company.
Let’s sum up some of the positives you get by using a DSL for your project. After which we also go over the drawbacks.
To sum up some positives:
- DSL makes the code way more readable thanks to abstraction.
- Anyone without programming knowledge can start automating tests. This allows for more creative input in what to test.
- Tests are very quick to set up.
- No steep learning curve for any new tester.
- Opens up hiring different talents within the company that can also test.
A tester mindset is still needed though(!) - Using a custom DSL gives a lot of freedom, so think twice before just choosing any DSL syntax (like Gherkin via Cucumber).
The main drawback is that a DSL requires quite an amount of extra development. Each and every main function of your framework and its implementations of the software to be tested, should be rewritten and interface with a DSL form.
Now this doesn’t mean that 100% of the supported software needs to be 100% rewritten for interfacing. Almost certain there is efficiency to be gained in some way or another. Metrics in this case show you exactly which libraries need to be supported by a DSL.
Another drawback is that, while it lowers the threshold for a new tester, it still involves setting up the framework at the end-user side. This comes with the needed support as well. Along with that a IDE for code/test development could scare off a test engineer.
Also the maintenance of the implemented DSL can become cumbersome. This is because DSL itself is an abstraction that doesn’t directly show what the underlying code is doing.
Lastly. It is possible that some features of your DSL are not used by any test engineer. In that case it is highly recommend to add the gathering of metrics in the test framework. This way you exactly know which functions are used the most and which are not. For example; This might show you that the test engineers aren’t using a password security feature, that might be necessary for operations or required per business or law standards.
After reading all the above, you probably understood what DSL can mean for yourself, your team and company.
DSL does have drawbacks, but compared to the benefits, it still is a great route to follow for your test automation approach. Atleast for me a test automation project for a medium to big sized company isn’t complete without including a DSL.
The main benefit I take from using a DSL is the fact that it allows writing automated test scripts very quickly and makes them readable. It will not only make me happy when I program, but also my future successor. As there is no better feeling than instantly grasp the concept of a test automation project because the current and previous developers made the right choice from the start.
I hope you learned something through my article. If you have any more questions or doubts, hit me up. We could grab a drink together in Belgium or The Netherlands, and talk about what keeps us busy. I would love to hear from you!
