Karate: a tool which pretends BDD but in fact is against BDD principles
WHAT IS KARATE FRAMEWORK?
Karate Framework is an open-source tool for testing web services and APIs that was developed by Peter Thomas. This framework is built on top of the Cucumber framework and Java programming language. Karate Framework is intended to simplifu the automation of testing for web services and APIs by providing a simple, yet powerful syntax for expressing test cases. Sounds promising. Right?
HOW DOES KARATE FRAMEWORK WORK?
Karate Framework works by providing a domain-specific language (DSL) for writing tests. This DSL is designed to be easy to read and write. The syntax of Karate Framework is similar to that of Cucumber.
Here is an example of Karate feature file.
Feature: Hello World
Background:
Given url baseUrl
Given path '/api/hello'
Scenario: Hello world
When method GET
Then status 200
And match $ == "Hello world!"
Scenario: Hello with a param
Given param name = 'Daas'
When method GET
Then status 200
And match $ == "Hello Daas!"IS IT A GOOD TOOL?
Karate may seem interesting but is it a good tool? I work in a company where there is a lot of tests on API layer. Actually currently I work only with such tests and to be honest I totally do not feel if that tool could help. In real test cases we do not have one simple call on API layer. We have whole scenarios with specific and rather complicated business logic involving communication with various services (not only internal). Here Karate will not help.
Ok, that was my personal view. What about more general aspects? Still I am strongly against using Karate. It pretends that BDD is applied by using Gherkin-like syntax. But Gherkin it is not the main value of BDD. This value is focusing on business logic when designing test steps. What we see in Karate? Something completely different. Karate encourages focusing on technical details.
KARATE UI? IT IS EVEN WORSE!
For Karate framework’s enthusiasts there were created also Karate UI for automating the web pages. Here is an example of it.
Feature: Move mouse
Scenario: move mouse
* configure driver = { type: 'chrome' }
Given driver 'https://karatelabs.github.io/karate/karate-core/'
* delay(2000)
* driver.maximize()
* delay(7000)
And click("(//a//code[text()='driver'])[1]")
* delay(7000)For me seeing examples of Karate UI is a nightmare. When I am going to make communication with business side easier, someone is trying to make it harder and bother non-technical people with totally irrelevant (from business logic point of view) matters as e.g. And click("(//a//code[text()='driver])[1]"). This does not sound simple. This is totally against Cucumber logic.
I wrote one text about the logic of separation of layers in Cucumber.
WHAT SAYS EXPERT?
This approach is also totally against of well known experts recommendation. Let me quote John Ferguson Smart.
Good BDD scenarios talk about real-world problems, in domain language. […]
Well-written BDD scenarios are highly effective, but creating them isn’t always easy. It takes practice, a little skill, and a lot of collaboration, to get them right.
Because they are not test scripts (which is the way most folk use them…and get into trouble with).
And they aren’t old-school requirements specifications, written by BAs or product owners because some misguided agile coach or trainer told them that’s how they had to write their “agile” requirements.
But when you get them right (and what “right” is will vary a little for each team and each project), they really do have the potential to slash defect counts and streamline delivery.
[…]
And another quote:
[…]
Not that you can’t describe UI interactions in BDD scenarios. You can.
But if you use Gherkin to describe a UI test script (the way 95% of courses teach Gherkin, and the way a majority of testers learn it), you will end up with automated test that:
❌ Are fragile — tightly coupled to the user interface means any change to the UI will break a multitude of scenarios
❌ Are slow to write and hard to read and maintain
❌ Do not true reflect business needs (“I want to click on this button then wait until the spinner dissapears” — said no user ever!)
BDD is about describing examples of requirements and business rules, in clean, crisp business language. No clicks or waits. No JSON payloads or 200 response codes. Just pure, business requirements.
Finally, let us look on the following diagram.
BDD should be a bridge between requirements gathering and user acceptance tests. BDD tools should not replace other tools on lower levels. Why anyone should do so? What for?
WHAT TO USE INSTEAD OF IT?
What to use instead of Karate? In my opinion pure Cucumber with some API library (e.g. restassured) or Serenity BDD. Or if you consider that you don’t need business logic layer the better option is using pure Java with restassured. For making some order in your code you can add some comments to separate arrange, act and assert parts. That’s all. Do not complicate your life.
@Test public void
lotto_resource_returns_200_with_expected_id_and_winners() {
when().
get("/lotto/{id}", 5).
then().
statusCode(200).
body("lotto.lottoId", equalTo(5),
"lotto.winners.winnerId", hasItems(23, 54));
}Do not believe in magic. “Given”, “When”, “Then” are not incantations causing that test understanding will be better. These words are just some small part of tool in larger process. They are useful, but they cannot replace the whole process.
One can say that examples in my course are also focused on technical details. This is right. These examples are not real cases. They are written intentionally to show how to set up a project. The sample page used in my examples has almost no business logic. In real case BDD should focus on peoples needs. This is starting point. To test fulfilling a requirement I need to know what someone wants to achieve.
