Make Your Testing Groovy

©ASERT2006-2016
Make your testing Groovy
Dr Paul King
Groovy Lead for Object Computing Inc.
@paulk_asert
http:/slideshare.net/paulk_asert/make-tests-groovy
https://github.com/paulk-asert/MakeTestingGroovy

©ASERT2006-2016
Why test with Groovy?

Why test with Groovy?
• Wrong first question!
• … but great second question? 
• Consider first the task at hand and
the organization and people fit!
©ASERT2006-2016

“People Fit” / “Organization Fit”
• People
– Developers (familiarity with languages)
– Testers (tools language familiarity)
– Responsibility to create/run/maintain
– BAs/SMEs (ability to read technical tests)
– Consider both development and maintenance
– Expected feedback from tests (JUnit/Reports)
• Organization
– Maturity level
– Degree of automation
– Tools in use
– Need for adaptability
©ASERT2006-2016

Why Test With Groovy?
• Advantages
– Easy to learn
– Particularly good for Java shops
– Easy to plug and play different testing tools
– Good community & tools for professional agile
• Disadvantages
– Requires JVM
– Less advantages if your developers using
Python, .Net, PHP
– Maybe your testers already know Ruby
©ASERT2006-2016

Scripting/Dynamic Languages
• Advantages
– Lend themselves to succinct code/DSLs
– Powerful
– Increased Refactoring
– Increased Reuse
– Less prone to Brittleness
– Flexibility for tool integration
– Open APIs provide extensibility
• Disadvantages
– Can be too low level
– Sometimes less tool support
©ASERT2006-2016

Test Characteristics…
• Coverage/Traceability
– Requirement coverage/traceability
– Code coverage: line, branch, path, state
– Transactional Tracing
• Purpose
– Unit, Integration, System, Customer
• Manageability
– Removing duplication
– Managing lifecycle: shared setUp/tearDown
(@Before @After)
©ASERT2006-2016

…Test Characteristics
• Handling test data
– Data-driven, databases, Spreadsheets, tables,
keywords, random, model-driven
– Large Test Volumes
– Speed of feedback, performance testing
• Tool evolution, longevity, cost,
support, documentation
– Open Source/Commercial, Critical
mass/popularity
• Combinability
©ASERT2006-2016

Key Testing Practices…
• Use testing DSL’s
• Look to move up the testing stack
– It used to be all about the driver
– Now the driver is hidden in the
framework or tool stack
• Apply good testing practices
– Pareto analysis, bug clusters, mutation
testing, test early, all pairs/equivalence
partitions/orthogonal array testing, risk-based
test selection, coding for testability, use CI,
boundary value analysis, defensive
programming
©ASERT2006-2016

…Key Testing Practices
• Plug and play testing tools
– Run different drivers with different runners
and different tools
• Complement automated tests with
exploration
• Expand testing scope
– Test environment readiness, test
deployments
©ASERT2006-2016

Groovy and Testing Tool Spectrum*
©ASERT2006-2016
* Tools/libraries/frameworks don't always neatly fall into one category – still useful conceptually

©ASERT2006-2016
What is Groovy?
“Groovy is like a super version of Java.
It leverages Java features but
adds productivity features and
provides great flexibility and
extensibility.”
Groovy = Java – boiler plate code
+ closures (1st class FP)
+ extensible type system
+ runtime & compile-time
metaprogramming
+ flexible grammar (DSLs)
+ scripting & GDK library

Groovy starter
©ASERT2006-2016
System.out.println("Hello, World!"); // supports Java syntax
println 'Hello, World!' // but can remove some syntax
String name = 'Guillaume' // Explicit typing/awareness
println "$name, I'll get the car." // Gstring (interpolation)
def longer = """${name}, the car
is in the next row.""" // multi-line, implicit type
assert 0.5 == 1/2 // BigDecimal equals()
assert 0.1 + 0.2 == 0.3 // and arithmetic
def printSize(obj) { // implicit/duck typing
print obj?.size() // safe dereferencing
}
def pets = ['ant', 'bee', 'cat'] // literal list syntax
pets.each { pet -> // closure support
assert pet < 'dog' // overloading '<' on String
} // or: for (pet in pets)...

Target audience
• Testing for developers
– Organising tests
– Running tests
– Data generation
– Mocking
– What can be tested
– Code coverage
• Testing beyond developers
– Structuring
– Better reporting
– Data-driven
– Domain specific language (DSL)
– Test coverage

Testing Frameworks
• None
• JUnit 3
• JUnit 4
• JUnit 5
• TestNG
• Spock

Testing Frameworks
• None ???
• JUnit 3
• JUnit 4
• JUnit 5
• TestNG
• Spock

Testing Frameworks
• None ???
– Groovy deems testing so important
that it comes with built-in testing:
– Built-in asserts, mocks
– Built-in JUnit 3 GroovyTestCase and utilities
– Built-in runners for tests
– Bundled JUnit 4
• JUnit 3
• JUnit 4
• JUnit 5
• TestNG
• Spock

Built-in assertions
©ASERT2006-2016
import static Converter.celsius
assert 20 == celsius(68)
assert -17 == celsius(0).toInteger()
class Converter {
static celsius (fahrenheit) {
(fahrenheit - 32) * 5 / 9
}
}

Built-in assertions
• But what about errors?
• Groovy’s power Assert mechanism gives
a friendly description of what went wrong
©ASERT2006-2016

GroovyTestCase (extends JUnit 3)
©ASERT2006-2016
import org.junit.Assert
import static org.junit.Assert.assertEquals
class SimpleGroovyTest extends GroovyTestCase {
void testShouldConvert() {
assert Converter.celsius(68) == 20
assert Converter.celsius(212) == 100, "Should convert boiling"
assertEquals("Should convert freezing", 0.0, celsius(32.0))
assertEquals("Should convert nearly freezing", 0.0, celsius(32.1), 0.1)
}
}

JUnit4
©ASERT2006-2016
import org.junit.Test
import static org.junit.Assert.assertEquals
class SimpleJUnit4Test {
@Test
void shouldConvert() {
assert celsius(68) == 20
assert celsius(212) == 100, "Should convert boiling"
assertEquals("Should convert freezing", 0.0, celsius(32.0))
assertEquals("Also for nearly freezing", 0.0, celsius(32.1), 0.1)
}
}

JUnit5
©ASERT2006-2016
@Grab('org.junit.platform:junit-platform-runner:1.0.0-M2')
@Grab('org.junit.jupiter:junit-jupiter-engine:5.0.0-M2')
import org.junit.jupiter.api.Test
import org.junit.platform.runner.JUnitPlatform
import org.junit.runner.RunWith
@RunWith(JUnitPlatform)
class ConverterJUnit5Tests {
@Test
void freezing() {
}
@Test
void boiling() {
}
}

Spock - BDD style
©ASERT2006-2016
@Grab('org.spockframework:spock-core:1.0-groovy-2.4')
import spock.lang.Specification
class GivenWhenThenSpec extends Specification {
def "test adding a new item to a set"() {
given:
def items = [4, 6, 3, 2] as Set
when:
items << 1
then:
items.size() == 5
}
}

Parameterized
©ASERT2006-2016
import org.junit.Test
import org.junit.runner.RunWith
import org.junit.runners.Parameterized
import org.junit.runners.Parameterized.Parameters
@RunWith(Parameterized)
class DataDrivenJUnitTest {
private c, f, scenario
@Parameters static scenarios() {[
[0, 32, 'Freezing'],
[20, 68, 'Garden party conditions'],
[35, 95, 'Beach conditions'],
[100, 212, 'Boiling']
]*.toArray()}
DataDrivenJUnitTest(c, f, scenario)
this.c = c
this.f = f
this.scenario = scenario
}
@Test void convert() {
def actual = celsius(f)
def msg = "$scenario: ${f}°F should convert into ${c}°C"
assert c == actual, msg
}
}

Spock
©ASERT2006-2016
import spock.lang.*
class SpockDataDriven extends Specification {
def "test temperature scenarios"() {
expect:
celsius(tempF) == tempC
where:
scenario | tempF || tempC
'Freezing' | 32 || 0
'Garden party conditions' | 68 || 20
'Beach conditions' | 95 || 35
'Boiling' | 212 || 100
}
}

Spock - Celsius
©ASERT2006-2016
import spock.lang.*
class SpockDataDriven extends Specification {
@Unroll
def "Scenario #scenario: #tempFºF should convert to #tempCºC"() {
expect:
celsius(tempF) == tempC
where:
scenario | tempF || tempC
'Freezing' | 32 || 0
'Garden party conditions' | 68 || 20
'Beach conditions' | 95 || 34
'Boiling' | 212 || 100
}
}

Spock - Mocks
©ASERT2006-2016
class SpockMock extends Specification {
def "buy ticket for a movie theater"() {
given:
def purchase = new Purchase("Lord of the Rings", 2)
MovieTheater theater = Mock()
theater.hasSeatsAvailable("Lord of the Rings", 2) >> true
when:
purchase.fill(theater)
then:
purchase.completed
1 * theater.purchaseTicket("Lord of the Rings", 2)
}
}

Spock - Mocks
©ASERT2006-2016
class SpockMockWildcards extends Specification {
def "cannot buy a ticket when the movie is sold out"() {
given:
def purchase = new Purchase("Lord of the rings", 2)
when:
theater.hasSeatsAvailable(_, _) >> false
then:
!purchase.completed
0 * theater.purchaseTicket(_, _)
}
}

Spock - Mocks
©ASERT2006-2016
class SpockMockClosureChecks extends Specification {
def "on couples night tickets are sold in pairs"() {
given:
def purchase = new Purchase("Lord of the Rings", 2)
theater.hasSeatsAvailable("Lord of the Rings", 2) >> true
when:
then:
1 * theater.purchaseTicket(_, { it % 2 == 0 })
}
}

Assertion frameworks
• None
• JUnit 3 assertions
• Hamcrest
• FEST
• Google truth
assertThat(googleColors).containsNoneOf(PINK, BLACK, WHITE, ORANGE)
assert ![PINK, BLACK, WHITE, ORANGE].any {
color -> color in googleColors }

Property-based testing
• Agile testing game (TDD)
– Minimum test code to steer design of minimal
production code with desired business functionality
but 100% code coverage
– “Grey box” testing
– Rarely used with functional programming
©ASERT2006-2016

• Agile testing game (TDD)
– Minimum test code to steer design of minimal
production code with desired business functionality
but 100% code coverage
– “Grey box” testing
– Rarely used with functional programming
– Instead validate certain properties
©ASERT2006-2016

©ASERT2006-2016
@Grab('net.java.quickcheck:quickcheck:0.6')
import static net.java.quickcheck.generator.PrimitiveGenerators.*
import static java.lang.Math.round
def gen = integers(-40, 240)
def liquidC = 0..100
def liquidF = 32..212
100.times {
int f = gen.next()
int c = round(celsius(f))
assert c <= f
assert c in liquidC == f in liquidF
}

Property-based testing: spock genesis
©ASERT2006-2016
@Grab('com.nagternal:spock-genesis:0.6.0')
@GrabExclude('org.codehaus.groovy:groovy-all')
import spock.genesis.transform.Iterations
import static spock.genesis.Gen.integer
class ConverterSpec extends Specification {
@Iterations(100)
def "test phase maintained"() {
given:
int tempF = integer(-40..240).iterator().next()
when:
int tempC = round(celsius(tempF))
then:
tempC <= tempF
tempC in liquidC == tempF in liquidF
}
…

Property-based testing: spock genesis
©ASERT2006-2016
@Grab('com.nagternal:spock-genesis:0.6.0')
@GrabExclude('org.codehaus.groovy:groovy-all')
import spock.genesis.transform.Iterations
import static spock.genesis.Gen.integer
class ConverterSpec extends Specification {
@Iterations(100)
def "test phase maintained"() {
given:
int tempF = integer(-40..240).iterator().next()
when:
int tempC = round(celsius(tempF))
then:
tempC <= tempF
tempC in liquidC == tempF in liquidF
}
…
…
@Iterations(100)
def "test order maintained"() {
given:
int tempF1 = integer(-273..999).iterator().next()
int tempF2 = integer(-273..999).iterator().next()
when:
int tempC1 = round(celsius(tempF1))
int tempC2 = round(celsius(tempF2))
then:
(tempF1 <=> tempF2) == (tempC1 <=> tempC2)
}
}

GPars
• Library classes and DSL allowing
you to handle tasks concurrently:
– Data Parallelism map, filter, reduce functionality
in parallel with parallel array support
– Asynchronous functions extend the Java
executor services to enable multi-threaded
closure processing
– Dataflow Concurrency supports natural
shared-memory concurrency model, using
single-assignment variables
– Actors provide Erlang/Scala-like actors
including "remote" actors on other machines
– Safe Agents provide a non-blocking mt-safe
reference to mutable state; like "agents" in Clojure
8

Constraint/Logic Programming
• Description
– Style of programming where relations between
variables are stated in the form of constraints
– First made popular by logic programming languages
such as Prolog but the style is now also used outside
logic programming specific languages
– Constraints differ from the common primitives of
other programming languages in that they do not
specify one or more steps to execute but rather the
properties of a solution to be found
– Popular libraries used with Groovy supporting
constraint programming include Gecode/J, Choco
and tuProlog
– We'll look at Choco as an example

Case Study with Constraint Programming
• You have been asked to set up some test
cases representing the Simpsons’ weekly
blogging habits
• After some careful study you observe the
following strange behavior
– They never blog on the same day
– Marge blogs only on a Saturday or Sunday
– Maggie blogs only on a Tuesday or Thursday
– Lisa blogs only on a Monday, Wednesday or Friday
– Bart blogs only on the day after Lisa
– Homer only blogs if noone else blogged the previous
day and doesn't allow anyone to blog the next day

Case Study with Constraint Programming

Testing DSLs
• Low-level DSL
• Higher-level DSL
post blog from Bart with title "Bart rulz!" and category School and content "Cowabunga Dude!"

More Information: Groovy in Action

Make Your Testing Groovy

Recommended

Recommended

More Related Content

Similar to Make Your Testing Groovy

Similar to Make Your Testing Groovy (20)

More from Paul King

More from Paul King (19)

Recently uploaded

Recently uploaded (20)

Make Your Testing Groovy