ABAP Test Double Framework
As many of you may know I have been playing around a lot with TDD last year, doing it in real life and speaking about the subject all over Europe.
In this blog I want to tell you a little story about “Test Doubles”. Are you sitting comfortably? Then I’ll begin. This is the story about how I became very puzzled about the so called “ATDF” and how I found the answers.
Just in case you don’t know what a Test Double is I would recommend the Open SAP Course on Unit Testing.
Here is a blog I wrote about the week they covered Test Doubles:-
https://blogs.sap.com/2018/04/21/sap-open-course-unit-testing-week-6-working-with-existing-code/
I have known for a long time there was such a thing as the “ABAP Test Double Framework” and have given examples in my ABAP books as far back as 2106, but in real life at work thus far all our test doubles were actual test double classes we created ourselves manually.
One day I woke up and thought now would be a good time to start playing with the ABAP test double framework in anger. At first glance it would seem to be a perfect fit with the FACTORY / INJECTION thing. For an explanation of what that is, see another blog I wrote about the course:-
https://blogs.sap.com/2018/04/14/sap-open-course-unit-testing-week-5-dependency-lookup/
In the In the SETUP method of the unit test I would have the following:-
"Create Test Double
mock_pers_layer ?= cl_abap_testdouble=>create( 'ZIF_MONSTER_PERS_LAYER' ).
DATA(injector) = NEW zcl_monster_injector( ).
injector->inject_pers_layer( mock_pers_layer ).
And then in my real code I would use a factory to get the persistency layer object, so during a test the double would be used.
So far, so good. Then the trouble started, In the ATDF I have always thought the crazy thing is that you configure the result before you say what method will give that result.
cl_abap_testdouble=>configure_call( mock_pers_layer )->returning( no_of_monster_heads ).
At this point I have not even created the method that is going to be called in the interface and yet the compiler is fine with everything because of course I have not yet said what method it is I am going to be mocking.
Before the ADTF framework existed as a standard SAP solution there was an equivalent ABAP Open Source project called ZMOCKA by Uwe Kunath which did pretty much the same thing, and in fact I strongly suspect SAP based their solution upon his. However the syntax in ZMOCKA made a lot more sense to me:-
lo_mocker->method( 'CALCULATE_SCARINESS'
)->with( i_p1 = ms_input_data
)->returns( 'REALLY SCARY' ).
You say what method is going to be mocked and what result comes back for what input data. With the ATDF you do this in two parts, the result first, and then in another call specify the input data and the method name to be mocked. That is backwards as far as I can see.
Even worse when you do configure what method you want in the ATDF it looks like this:-
mock_monster_simulator->calculate_scariness( is_bom_input_data = ms_input_data ).
Now to me that looks just like an actual method call to an instance of a class, but it is not, and then later on when the exact same statement is executed again (with the correct signature) it IS an actual method call albeit to some sort of generated construct impersonating an instance of the persistency layer class.
OK fair enough I know the syntax. So in my GIVEN method I want to set up some hard coded data for the test.
Right away I saw a problem staring me in the face. My proposed method being tested would read data from TWO different database tables and then do some calculations based on the result – in this case one table tells me how many heads a given type of monster has, the other table tells me how many monsters of each type are under my bed. As always this example is based on an actual example from work with some minor changes
The trouble is for a test double it looked like – from all the example on the internet, from every single on – that you could only test one method at a time. All the examples did the unit test on a single method directly after configuring the test double, sometimes they did a large number of variations, but always only one method was called during the test.
So, it seemed to me based on the examples and the blogs on the internet, that if my production method has two calls on the persistency layer object i.e. calls two methods that each read a database table, then it cannot be tested using the test double framework. At least that is how it looked.
So off I went looking up the blogs I had read when the ADTF first came out to see if I was missing something really obvious.
The standout blog was by Sandra Rossi.
https://blogs.sap.com/2018/04/03/short-examples-of-cl_abap_testdouble/
It did not help me with my exact problem but it certainly refreshed my memory. The most interesting thing about the blog however was the “latest news” at the end.
===> LATEST NEWS, March 16th, 2019: hey, what is the real point of using the ATD Framework, if you can simply create your own local class implementing the interface and code the rules you want… (discussion here:
https://blogs.sap.com/2015/01/05/abap-test-double-framework-an-introduction/#comment-452240). By the way I moved the code to Github.
I found that interesting. Had she become disillusioned with the ATDF and decided it was pointless, which, I admit was pretty much what I was feeling at that point.
So I followed the link to the comment:-
Enno Wulff
February 25, 2019 at 4:37 pm
Thanks Prajul Meyana for clarifying the use of SAP’s test doubles framework.
Yes, I know, the post is older than four years now, but I think that this approach hasn’t changed at all?!
IMHO the cl_abap_testdouble is totally nonsense!
As Christian Drumm already mentioned: As long as I need an interface, it’s so much easier to simply create a new class where I can hard code the desired values. If I am at the point where I can use an interface, because the application thankfully supports dependency injection for the right methods, then it is easier, less complicated, clearer and uses less code to implement a test-double-class for my test cases.
Oh! I thought. Enno Wulff. I know him, I have met him at various SAP Inside Track Events (he even attended a speech I gave on Test Driven Development) and read a load of his blogs which are excellent.
So a lot of people I respect are saying the whole thing is nonsense based on various criteria, the case against the ATDF (compared to using manually created test double classes) being it was:-
- Slower
- You could only have 36 different interfaces in a test run (not usually a problem I would think)
- Needed more code
- Incredibly over-complicated and difficult to understand
- Possibly others but that is all I can recall off the top of my head
Was the whole thing nonsense? By this point I had been pretty much convinced it was as you can see in that blog as I replied trying to be objective but coming down on the side of the ATDF being over-complicated.
Meanwhile I had been reaching out to SAP expert Jim McDonough in the USA to see what he thought. He was a lot more positive about the ATDF, he had used it in real life and it seemed OK to him. He went over the basics again to me (by that point this is what I really needed as I was starting to lose the plot) and more importantly said just try doing two configurations on your ATDF object and calling the test with two different methods and see what happens.
OK I thought, if I get that working then I will have two example programs, one using a local test double class, one using ATDF and I will see which is the best as far as I can tell.
It did work, which was a Good Thing.
METHOD atdf_experiment.
DATA: mock_pers_layer TYPE REF TO zif_monster_pers_layer.
"Create Test Double
mock_pers_layer ?= cl_abap_testdouble=>create( 'ZIF_MONSTER_PERS_LAYER' ).
"Inject it into the factory
DATA(injector) = NEW zcl_monster_injector( ).
injector->inject_pers_layer( mock_pers_layer ).
"Configure the results for the first method within the CUT method to be tested
cl_abap_testdouble=>configure_call( mock_pers_layer )->returning(
VALUE zcl_monster_head_model=>mtt_monsters_per_bed(
( zz_bed = 'MINE' zz_monster_type = 'EATER' zz_count = 5 ) ) ).
"Now say what method will be invoked to get that result
mock_pers_layer->derive_monsters_per_bed( 'MINE' ).
"Now do the same thing for the second method to be called
cl_abap_testdouble=>configure_call( mock_pers_layer )->returning( 3 ).
mock_pers_layer->derive_heads_per_monster( 'EATER' ).
"Call Method Under Test
when_head_nos_are_calculated( ).
"Assertion
then_no_of_heads_should_be( 15 ).
ENDMETHOD.
That compiled OK and the test passed, so you can have as many different methods as you want and as far as I can see I am the first person ever on the internet to post such an example i.e. using the ATDF to test a method with calls to different methods of the same test double class. I am probably not, but as I said, I could not find any and I looked really hard.
So how to the two approaches compare?
In regards to performance Eclipse tells you how long a test run takes. In my experiment the ATDF framework takes approximately four times as long to run as having a local test double class, presumably because it has to generate code and the like, and dynamic programming is always slower than fully typed programming (I think).
Now, leaving the 36 thing out, the other arguments come down to clarity and ease of creation. One thing is for sure, with Eclipse creating a local test double class is the easiest thing in the world.
As I said in a comment to one of the above blogs in Eclipse I try to create a test double of a local class that does not yet exist yet starts with LTD_ then the quick fix is clever enough to work out I want to create a test double class and does the definition and implementation all for me. It is not clever enough yet to work out that because I have typed the variable I am trying to create based on an interface, that the test double should implement that interface, but that is trivial.
Ten seconds later I have (blank) implementations for all the interface methods. How in the world can this be considered tedious? It would be tedious in SE80 – of that there is no doubt!
Now let us look at clarity:-
I think I have got to the stage where I understand HOW to code the ADTF such that it does what I want. I just think such code looks bonkers and seems to be in diametric opposition to the “clean code” initiated currently underway.
https://github.com/SAP/styleguides/blob/master/clean-abap/CleanABAP.md
Which is trying to say that code should always be 100% clear about what it is actually doing, and it should be obvious to a reader, to make it easy to maintain going forwards.
The irony of course is that very document says to use the ATDF as opposed to manual TD classes based on the following logic:-
Consider to use the tool ABAP test double
DATA(customizing_reader) = CAST /clean/customizing_reader( cl_abap_testdouble=>create( '/clean/default_custom_reader' ) ).
cl_abap_testdouble=>configure_call( customizing_reader )->returning( sub_claim_customizing ).
customizing_reader->read( 'SOME_ID' ).
Shorter and easier to understand than custom test doubles:
" anti-pattern
CLASS /dirty/default_custom_reader DEFINITION FOR TESTING CREATE PUBLIC.
PUBLIC SECTION.
INTERFACES /dirty/customizing_reader.
DATA customizing TYPE /dirty/customizing_table.
ENDCLASS.
CLASS /dirty/default_custom_reader IMPLEMENTATION.
METHOD /dirty/customizing_reader~read.
result = customizing.
ENDMETHOD.
ENDCLASS.
METHOD test_something.
DATA(customizing_reader) = NEW /dirty/customizing_reader( ).
customizing_reader->customizing = sub_claim_customizing.
ENDMETHOD.
Shorter – yes. Easier to understand? Not in a million billion years as far as I am concerned, and my local test double class is a lot easier to understand than the “dirty” example.
Conclusion
At this point I have got the ATDF doing what I want. I have two different versions of the program, one with a local test double class, one using the ATDF.
The one with the ATDF is a lot shorter in lines of code, but takes four times as long to run and is incredibly difficult to understand what in the world is going on.
In any case I hide the complexity away:-
The good thing about using the GIVEN / WHEN / THEN pattern is that I can hide the nonsense code needed by the ATDF inside a GIVEN method. Here is one of my test methods:_
METHOD calculate_no_of_heads_needed.
given_customizing_that_says( for_monster_type = 'EATER'
no_of_heads_needed = 3 ).
given_monster_numbers( in_bed = 'MINE'
of_type = 'EATER'
there_are = 5 ).
when_head_nos_are_calculated( ).
then_no_of_heads_should_be( 15 ).
ENDMETHOD.
There is no way to tell if I am using the ATDF or not, the code in that method is identical in both versions of my program, and that is good as this method is supposed to explain WHAT is being tested, not how a unit testing framework works.
This is an incredibly emotive subject no doubt, so I would love to hear everyone’s opinion on the matter.
Cheersy Cheers
Paul
