Creating Kotlin DSLs

Kotlin has many nice things going for it - be it null safety, data classes, extension functions, immutable collections or many other things. For me one additional thing is very interesting: The ability to create domain specific languages (DSLs) easily. Easily, because creating a DSL with Kotlin means that you simply choose to combine several core features of the language.

What is a DSL

But first things first: What actually is a domain specific language?

A domain specific language is a construct that allows you to create things and set them up in a way that feels more natural and usually comes closer to proper language usage than normal code.

As the name implies, DSLs are only useful in a constrained context - the domain at hand. In contrast to general purpose languages (of which Kotlin of course is one) DSLs apply to a limited set of use cases only.

What then is the benefit of using a DSL over normal constructs? The main benefit is, that domain specific languages provide a more concise and at the same time much better readable API for a specific purpose, compared to the normal general purpose way to achieve the same.

External DSLs

DSLs are nothing new actually. They have been available in other languages for quite a while (on the JVM for example in Groovy and Scala). Actually most DSLs are language independent. Famous examples are SQL, HTML, Gherkin or all those YAML files that are all the rage nowadays.

You can write code using those DSLs in any text editor and later the DSL is then used and parsed by some other code. For example a RDBMS parses SQL, Cucumber parses your Gherkin code, a browser the HTML code and so on. In theory a parser for this could be written in any language. The DSL is external to the language with which it is parsed. Those DSLs are external DSLs. Frederico Tomassetti has compiled a list of more examples of these.

Internal DSLs

But there are also internal DSLs. They are written in the same general purpose language as the DSL itself. The DSL is just a specific way to use the host language. Gradle is an example of an internal DSL - in this case in Groovy.¹⁾

When I’m speaking of Kotlin DSLs, I’m speaking of internal DSLs, of DSLs that make use of specific Kotlin features to achieve their simplicity and benefits.

Examples of Kotlin DSLs

Let’s see some examples of DSLs with Kotlin to show what DSLs are about and why I like them.

KotlinTest

KotlinTest is a library to write better unit tests. One benefit of KotlinTest is the way it allows you to structure your test code as you can see in the following sample:

"when user entered credentials" {
    "and clicked the login button" {
        "and the credentials were valid, then the app" {
            should("switch to main flow") {
                // testing logic goes here :-)
            }
            // more should blocks ...
        }
        // more contexts ...
    }
}

Having done quite some work with Javascript (mostly for the Google Assistant), I started to like Jasmine and gripe about what I used to use on the JVM. Happily with KotlinTest better structured and more easily readable tests for Kotlin projects are no longer an issue.

Anko

I think the Kotlin DSL most well known among Android developers is Anko, especially Anko layouts. If you like to write layouts in code (like in Flutter or React) this can be pretty interesting for you.

override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    verticalLayout {
        val name = editText()
        button("Say Hello") {
            onClick { toast("Hello, ${name.text}!") }
        }
    }
}

I personally happen to prefer the XML way for layouts and as such am no Anko user. But it’s a well liked project with 13.000 stars on github, so I might be an outlier when it comes to that.²⁾

Koin

The final example to show the value of DSLs is Koin. Koin is a dependency injection framework ³⁾ specific for Kotlin. And it integrates nicely with Android (though it works on standalone JVM projects as well).

Since Koin 2.0 will be even better than the current version, I use samples of the upcoming version. It’s going to be released soon enough for this blog to highlight Koin 2.0.

With Koin you define your dependencies like this:

val appModule = module {
    single { AndroidLogger() as Logger }
    single(name = "mainDispatcher") { Dispatchers.Main as CoroutineDispatcher }
    //...
}
// and somewhere else:
koinApplication {
    useAndroidContext(this@SomeApp)
    loadModules(appModule, loginModule, ...)
}

Again the important thing is, that you can understand easily what is going on here.

There are of course many more examples. Actually DSLs can enhance and ease the use of many libraries, so I expect them to pop up more and more.

But what is common to all these and other samples, is that DSLs are nicely readable. They make the intent of the code obvious and as such they help us achieve understandability and maintainability of our code. That's why I like DSLs - if appropriately designed and judiciously used 🙂

Creating your own DSL

Now that we’ve seen sample of DSLs, let’s cover how to actually create one. I will go through some more snippets of code from the many libraries that already offer DSLs and show how they made use of simple Kotlin concepts to achieve the desired result.

Designing a DSL

A DSL is part of your public API that you expose to clients. And since DSLs are part of the overall API, you should also follow common design approaches for APIs.

The first design consideration is, that you always should design with the client usage on your mind. It’s not your job to make the creation of your DSL easy for you. It’s your job to make the usage of your DSL as easy and as intuitive as possible for your client.

The overall guiding principle is, that the proper use of your API should be easy, while any potential misuse should be as hard to make as possible. This is one of the many advices by Joshua Bloch in his presentation on the topic of API design. This one recommendation is probably the most important one, but I strongly recommend to have a look at all of his slides.

One way to ease client development is to shield the client from any technical obstacles. Do not hassle the client with technical details that are of no importance to her.

But what to do if your client might be interested in them. For example information about what went wrong when trying to do a remote call. In that case you could offer a reasonable default behavior for all those that do not care about the details and only make those details available to those clients that want to act differently depending on the cause of the problem.

A DSL specific part of API design is the proper naming of things and of choosing a good syntax. When it comes to naming things, solutions should be kind of obvious. After all you are creating a domain specific language. And as such you should follow the naming conventions of the domain at hand. There also might exist established patterns for naming entities or actions on them. If so, you should follow along to meet client expectations.

With these thoughts in mind you can start designing your DSL. The following paragraphs concentrate on the various elements of the Kotlin language that can help you achieve a nice DSL.

Trailing lambdas

Let’s start with trailing lambdas since they are so common in Kotlin, that you are bound to have seen them.

A trailing lambda is simply syntactic sugar for using function types as arguments to your higher order functions. For example the way you use the should spec style in KotlinTest is made possible by using trailing lambdas. This is a simplified should spec function:

should("show an error message when something bad happened") {
    presenter.doSomething()
    verify {
        view.showError("Some error")
    }
}

And this is how the should method is defined in the KotlinTest code base:

fun should(name: String, test: () -> Unit): TestCase {
    val testCase = TestCase(
            suite = current, name = "should $name", test = test, config = defaultTestCaseConfig)
    current.addTestCase(testCase)
    return testCase
}

As you can see, the argument list of this function expects another function as the last argument, the test argument. And the language designers of Kotlin made it possible to simplify passing the lambdas in when they are the last argument by allowing you to close the argument parenthesis before starting the lambda.

Without trailing lambdas, the should spec above would have looked like that:

should("show an error message when something bad happened", {
    presenter.doSomething()
    verify {
        view.showError("Some error")
    }
})

This is, of course, valid code, but the placements of the parentheses doesn’t look that nice. By allowing you to put the closing parenthesis in front of the lambda argument, the code becomes more readable.

Extension functions

Extension functions are a common pattern that is widely used in existing Kotlin DSLs. They allow you to extend existing classes and to make those extensions only valid within the scope of your DSL.

The next code snippet shows how to write an extension function for the String class. It simply returns a new String in which the original characters are turned into dots. Think of replacing clear text passwords in logs and so on.

fun String.toDots() =
        this.replace(".".toRegex(), "●")

In the Spring routing DSL extension functions on Strings are used to specify handlers for certain paths, to nest different parts of the path or to combine a path with another request predicate. For example to combine a path with the requirement for a certain content type you can use this code:

("/api" and contentType(MediaType.APPLICATION_JSON)) { 
    userHandler.findAll(it) 
}

In this case the and is an extension function on String:

infix fun String.and(other: RequestPredicate): RequestPredicate = path(this).and(other)

We will cover the infix part pretty soon. For now we concentrate on the fact that by providing this extension function, the routing DSL allows us to write code that is readable and quickly understandable.

If you come from Java, you might wonder about this, since String is final and you cannot inherited from it. That’s true, and you still would be able to write extension functions for Java’s String class.

That’s working since the Kotlin compiler transforms extension functions into static functions that take the object in question, in this case the string, as an additional first parameter. As such the code is not really extending the String class, it’s just pretending to do so.

Infix calls

Now the sample just used was still a bit weird since it didn’t look like a normal function call. That’s because the and function is an infix function, denoted by the infix keyword.

Infix functions are functions with exactly one parameter and the infix keyword makes it possible to separate the object and it’s single parameter by the function name. Which is ideal for DSLs since it makes your code more naturally readable.

Instead of having to write the call “somePath”.and(somePredicate) you are able to use "somePath" and somePredicate instead. Much better.

Another sample for infix calls is the following code:

every { view.showNewState(any()) } just runs

The just method tells MockK that a code, that doesn't return any value, must nevertheless run at least once when calling the object under test.

The way to write it like shown above, is only possible because of the use of the infix keyword:

infix fun MockKStubScope.just(runs: Runs) 
        = answers(ConstantAnswer(Unit))

Without the infix keyword one would have to write the MockK definition like this:

every { view.showNewState(any()) }.just(runs)

So we would have to add the dot between the object and it's function and we would have to put the argument within parentheses. I guess you agree that the version that actually uses the infix syntax is much easier to read.

Empty objects

Now let’s have another look at the MockK sample we've just seen. This time we don’t look at the just but at the runs. What is that actually.

every { view.showNewState(any()) } just runs

It turns out that it is an empty object:

object Runs
typealias runs = Runs

An object without any behaviour that exists only to make this infix syntax possible at all.

Now, be careful with this and don’t overdo it. Very often other ways might be better. In this case, though, I like it.

Conventions

You have already seen how extension functions as well as trailing lambdas can help you when writing a DSL, but the following code still looks a bit weird:

"should show an error message when something bad happened" {
    presenter.doSomething()
    verify {
        view.showError("Some error")
    }
}

It is nearly the same test method you have seen a few paragraphs before, but now instead of using the should spec style it's using a string spec style.

This code snippet suggests that you can pass a lambda to an object. As if the object were a function itself. But of course it is not 🙂

There is a convention in Kotlin, though, that allows you to treat objects as if they were functions. At least in some very restricted context.

If the object in question has a method named invoke and this function acts as a higher-order funtion, that accepts a function as argument, you can call it like shown above.

A look at the definition of this method makes it easier to understand:

operator fun String.invoke(test: () -> Unit): TestCase {
    val tc = TestCase(suite = rootTestSuite, name = this, test = test, config = defaultTestCaseConfig)
    rootTestSuite.addTestCase(tc)
    return tc
}

You can have multiple invoke methods. And you can use them as extension functions to enhance existing classes, like the String class in this sample.

Lambdas with receivers

You have already seen what trailing lambdas are. But in many cases a normal lambda is not enough. Since DSLs are especially useful for creating and setting up objects or even fully fledged object graphs, we need a way to do exactly that. This is where lambdas with receivers (or function literals with receivers) enter the scene. They allow a domain specific language to create a specific object for you while expecting the configuration of that object in the trailing lambda. But how?

Let's see an example first:

val appModule = module {
    single { AndroidLogger() as Logger }
    single(name = "mainDispatcher") { Dispatchers.Main as CoroutineDispatcher }
    //...
}

What you see here is the Koin 2.0 way to initialize the module graph. Nowhere in this code do you actually create module objects. So the methods must do that for you.

The same mechanism is also what allows Anko to create the view hierarchy for your screens. Let's review the code shown initially in the DSL sample section:

verticalLayout {
    val name = editText()
    button("Say Hello") {
        onClick { toast("Hello, ${name.text}!") }
    }
}

The verticallayout and button blocks are where objects get created and in the respective trailing lambdas you define how to set up those objects.

Within the lambdas you have access to the newly created object as this and thus there is no implicit argument named it.⁴⁾

Now let's see how a trailing lambda looks like from the perspective of the DSL creator. The above modules method definition looks like this:

typealias ModuleDeclaration = Module.() -> Unit

fun module(createdAtStart: Boolean = false, override: Boolean = false, moduleDeclaration: ModuleDeclaration): Module {
    return Module(createdAtStart, override).apply(moduleDeclaration)
}

You can think of function literals with receivers as of local extension functions that are valid only within the scope of the defining function.

As you can see in above's typealias the function type is an extension function on Module. The only thing missing from a normal extension function definition is the name of the function. And this is exactly the same as the name of the argument. So in this case moduleDeclaration (the last argument to the module function on line 3).

On line 4 the function type that was passed into the module function is then passed on to the apply function. But the Module object itself, on which the local extension function is defined, is not passed into the function. Instead it is created within the function right after the return keyword.

Lambdas with receivers might be hard to grasp initially. But they are an important Kotlin concept - not only within the context of DSLs. You can read more about function literals with receivers on Simon Wirtz' blog.

DSLMarker

If you have lambdas with receivers the lambda can access the receiving object with this. But if you have nested structures this can apply to many objects in the hierarchy. And using a method on an object might be wrong outside of the given scope.

That's where the DSLMarker annotation comes into play. If you have multiple annotations the function closest to the call wins. Thus, given the Anko sample above, any methods specific to verticallayout are not accessible within the trailing lambda of the button.

This is great in preventing accidental misuse of your DSL. As you might recall, making your API hard to misuse is one of the fundamental aspects of API design. Luckily Kotlin has us covered!

Let's see it in action in Anko:

inline fun ViewManager.button(text: CharSequence?, init: (@AnkoViewDslMarker android.widget.Button).() -> Unit): android.widget.Button {
    return ankoView(`$$Anko$Factories$Sdk25View`.BUTTON, theme = 0) {
        init()
        setText(text)
    }
}
//... somewhere else:
inline fun Activity.verticalLayout(theme: Int = 0, init: (@AnkoViewDslMarker _LinearLayout).() -> Unit): LinearLayout {
    return ankoView(`$$Anko$Factories$CustomViews`.VERTICAL_LAYOUT_FACTORY, theme, init)
}

And here's the definition of the annotation:

@DslMarker
@Target(AnnotationTarget.TYPE)
annotation class AnkoViewDslMarker

What also might help your DSL

Kotlin allows you to define type aliases, which are a nice way to make your code more explicit. In the MockK example you have seen a type alias for the object Runs which made it possible to use the word in a more natural way.

Another thing to consider is operator overloading. Actually invoke - shown above - is an operator. But there might be others that can come in handy, for example plus or unaryPlus. The library kotlinx.html for example makes use of the unaryPlus operator:

html {
    body {
        div {
            +"a trivial example"
        }
    }
}

Finally I recommend to consider deviating from naming conventions. The routing DSL in Spring for example uses methods to route incoming traffic to certain handler methods. And since they are dealing with HTTP verbs they name those methods in all upper case like GET, PUT, POST and so on. I think in this case the deviation from normal naming conventions is justified and makes the intent of the code more obvious than if the methods were in lower case:

"/api".nest {
    accept(APPLICATION_JSON).nest {
        GET("/users", userHandler::findAll)
    }
}

You can find a more complete example for this in this gist.

Is it worth it?

Now that you have seen how to create a DSL, you still have to consider whether to go for it. And again, there are two aspects to it:

• Should you use a DSL if one exists?
• Should you create a DSL for your API?

Should you use a DSL?

The answer whether to use one or not is pretty easy: of course you should. But, then again, it’s not that easy.

You might have to decide which library to use for a specific problem. And a DSL might help you choose one over the other. But, of course, it’s not the only thing to take into consideration when selecting a library. You still need to be sure that the API fulfills your overall needs, that it is likely to be maintained in the long run (or easy enough to fork and maintain yourself in case the original maintainer(s) wander off).

If two or more libraries fulfill your needs, are more or less equally likely to be maintained in the long run and have an overall nicely designed API I would choose the one that offers the best readability of the client code. If one of those libraries uses a DSL, it’s likely that this one is the lib whose client code is best to maintain and easiest to read. In such a case, a DSL can be the reason to choose one library over the other.

Should you create a DSL?

As nice as a DSL might be, it still comes at a cost. It’s simply more costly to create one. While any API has to be designed carefully, a DSL puts some extra workload on your shoulders. Thus you have to ask yourself whether you should go for it or not.

This depends largely on what your use case is. Is your API for inhouse clients? Is your API used very often or only every now and then?

For libraries things are a tad different. Here the most important questions to ask yourself is: Does your client really benefit of a DSL? Does the domain in question lend itself easily to a DSL and does it offer enough and intuitive concepts to create one? If so, you probably should create one. It will help your library gain traction over those, that do not provide one.

This is even true for Java based libraries. Consider to offer special constructs for Kotlin. The people at Pivotal did so while creating Spring 5. They offer a DSL for Kotlin users even though most of the overall Spring code is written in Java. For me this was one big selling point when considering which Java lib to use for my latest App backend.

It all boils down to this: Is the client code easier to read and write if you provide a DSL?

If yes, by all means go for it. And if not, it’s just not worth it.

But even if you decide not to create a DSL, all the things above might still help you create a nicely designed API that is easy to use. Especially trailing lambdas and function literals with receivers are a very common pattern in Kotlin that might be a good fit for your API - DSL or not.

When’s a DSL a DSL?

By now you hopefully are convinced that DSLs make code more readable and can be worth the effort. You really want to create one, but you’re not sure, if what you’re thinking about is actually a DSL or not. Actually that’s a very tough question, since there are not hard and fast rules that make some code a DSL and some other code not.

In my opinion it all boils down to the question whether the domain offers enough concepts to actually create a meaningful DSL out of it. And if you can create a proper language out of the concepts. After all it’s no DSL if either the domain or the language part of it are missing.

In the end it doesn’t really matter that much. Because the important thing is, that your overall API is easy to use. Whether it’s a proper DSL or just a nicely designed API is not important. Important is, that your API takes the general recommendations for good API design into consideration. That it takes the client front and center while designing the API and that it thus makes the use of your API intuitive and easy and the misuse hard: That’s the problem you have to tackle. And if you do so, your client’s will like to use your API and the resulting code will be easy to maintain.

Want to know more?

There are some good blog posts and some information on the Kotlin website that also cover Kotlin DSLs.

In addition to that I highly recommend reading the book Kotlin in Action. This book is outstanding - and it has a chapter on DSLs. It might be worth to check if it's available as a deal on Mannings site in their year's end sale.

It's always important not to go overboard with new features. That's why in my talks about Kotlin DSLs I always recommend having a look at Márton Braun's repo that discusses the pros and cons of different approaches and provides sample code for all of them. He has nicely summarized the different DSL styles.

Happy coding!

Wolfram Rittmeyer lives in Germany and has been developing with Java for many years.

He has been interested in Android for quite a while and has been blogging about all kind of topics around Android.

You can find him on Google+ and Twitter.