The Limits of Example-Based Testing
Traditional unit tests check specific examples: given one input, expect one output. This approach has a fundamental blind spot: you can only test the cases you think of. If your mental model of the code's behavior has a gap, your tests will have the same gap.
Property-based testing inverts this approach. Instead of specifying individual examples, you define properties that should hold for all valid inputs, and the framework generates the inputs. By default Kotest runs each property 1,000 times. When it finds a failing input, it shrinks it toward the minimal reproducible case.
The inputs are not purely random. Kotest's generators mix random samples with a hard-coded set of edge cases: for integers that is 0, 1, -1,
Int.MAX_VALUE and Int.MIN_VALUE, and by default 2% of the values a test sees are edge cases. Those are exactly the values a hand-written example suite forgets.Setting Up Kotest Property Testing
Kotest ships its property testing as a separate module,
kotest-property, which is supported on all Kotlin targets. You do not need to adopt Kotest as your test framework to use it, though the examples here do. The current release line is 6.2.
There are two entry points. forAll takes a function that returns a Boolean and passes when it returns true for every input. checkAll takes a function that returns nothing and passes when no assertion throws, which lets you use ordinary matchers. The examples below use checkAll.kotlin
// build.gradle.kts
dependencies {
testImplementation("io.kotest:kotest-runner-junit5:6.2.5")
testImplementation("io.kotest:kotest-property:6.2.5")
testImplementation("io.kotest:kotest-assertions-core:6.2.5")
}
// In your test class
class PriceCalculatorPropertyTest : FunSpec({
test("discount never exceeds original price") {
checkAll(
Arb.double(1.0, 10000.0), // price
Arb.int(0, 100) // discount percentage
) { price, discountPercent ->
val discounted = calculateDiscount(price, discountPercent)
discounted shouldBeGreaterThanOrEqual 0.0
discounted shouldBeLessThanOrEqual price
}
}
test("price is always non-negative after tax") {
checkAll(
Arb.double(0.01, 99999.99), // base price
Arb.double(0.0, 0.25) // tax rate
) { basePrice, taxRate ->
val total = applyTax(basePrice, taxRate)
total shouldBeGreaterThanOrEqual basePrice
}
}
})Which Properties Should You Test?
The hardest part of property-based testing is identifying properties. Here are patterns that apply broadly:
Roundtrip properties: If you encode then decode, you should get the original value back. Serialization, encryption, and compression all have this property. Our Kotlin Serialization guide builds the kind of model classes a roundtrip test protects.
Invariant properties: Some condition that must always be true. A sorted list's length equals the original list's length. A balanced tree's height is O(log n).
Oracle properties: Compare your implementation against a simpler, obviously correct one. Your optimized search should return the same results as a brute-force scan.
Idempotency: Applying an operation twice should give the same result as applying it once. Formatting already-formatted text, or deduplicating an already-deduplicated list.
kotlin
class SerializationPropertyTest : FunSpec({
// Roundtrip: serialize then deserialize = identity
test("User roundtrip through JSON") {
checkAll(userArb) { user ->
val json = Json.encodeToString(user)
val restored = Json.decodeFromString<User>(json)
restored shouldBe user
}
}
// Invariant: sorting preserves elements
test("sorting preserves list size and elements") {
checkAll(Arb.list(Arb.int())) { list ->
val sorted = list.sorted()
sorted.size shouldBe list.size
sorted.toSet() shouldBe list.toSet()
}
}
// Idempotency: trimming whitespace twice = once
test("sanitizeInput is idempotent") {
checkAll(Arb.string()) { input ->
val once = sanitizeInput(input)
val twice = sanitizeInput(once)
twice shouldBe once
}
}
})
// Custom generator for a domain object. Arb.bind combines the shrinkers of
// its parts, so a failing User is shrunk field by field.
val userArb = Arb.bind(
Arb.long(1, Long.MAX_VALUE), // id
Arb.string(1..50), // name
Arb.email(), // email
Arb.int(13, 120), // age
::User,
)How Does Shrinking Find the Minimal Failure?
When a property test finds a failing input, the raw generated value is often large and confusing. Kotest shrinks it: it retries with progressively simpler values and keeps the simplest one that still fails. By default it makes up to 1,000 shrink attempts, and
Prefer
ShrinkingMode.Unbounded removes the cap.
If your function fails on a list of 847 elements, the shrinker tries smaller lists until it finds that the failure needs only 2 elements in a specific arrangement. That case is far easier to debug than the original.
How a custom generator is built decides whether it shrinks at all, and it is easy to get wrong. An earlier version of this article did.
| How the generator is built | Shrinking |
|---|---|
Arb.bind(a, b, ::Type) | Full: every field is shrunk independently |
a.flatMap { ... } | Partial: only the outermost arbitrary's shrinker applies |
arbitrary { ... } | None, unless you pass a Shrinker explicitly |
Arb.bind for domain objects. An arbitrary { } block reads nicely and then reports the raw failing value, unshrunk.How Do You Reproduce a Failing Run?
Random inputs raise the obvious objection: a failure you cannot reproduce. Kotest answers it with seeds. Every run draws its values from a random source created from a seed, and whenever a property test fails, Kotest outputs the seed that was used. It also writes that seed to
~/.kotest/seeds/<spec>/<testname> and picks it up on the next run, so a red test stays red on your machine until you fix it.
To reproduce a failure from CI, where that file does not exist, pass the seed yourself. An explicit seed takes precedence over a saved one.kotlin
test("concatenation adds the lengths") {
// The seed Kotest printed when this property failed.
checkAll<String, String>(PropTestConfig(seed = 127305235)) { a, b ->
(a + b).length shouldBe a.length + b.length
}
}Integrating with Android Testing
Property-based testing works well with Android components that have well-defined input/output contracts. ViewModels, repositories, mappers, and utility functions are all good candidates. You can run these alongside standard JUnit 5 tests without any special configuration.
Avoid property-testing UI components directly: the input space is too large and the properties are hard to define. Test the logic layer that drives the UI instead, and use the approaches in our Compose testing strategies for the UI itself.
kotlin
class CurrencyFormatterPropertyTest : FunSpec({
test("formatted currency always has 2 decimal places") {
checkAll(Arb.double(0.0, 1_000_000.0)) { amount ->
val formatted = formatCurrency(amount, "USD")
// Should match pattern like "$1,234.56"
formatted shouldMatch Regex("""\$[\d,]+\.\d{2}""")
}
}
test("parsed amount matches original within rounding") {
checkAll(Arb.double(0.01, 999_999.99)) { amount ->
val rounded = (amount * 100).roundToInt() / 100.0
val formatted = formatCurrency(rounded, "USD")
val parsed = parseCurrency(formatted)
parsed shouldBe (rounded plusOrMinus 0.01)
}
}
})Key Takeaways
- 1Property-based tests define rules that hold for all inputs, catching edge cases examples miss.
- 2Four key property types: roundtrip, invariant, oracle, and idempotency.
- 3Kotest runs each property 1,000 times by default and mixes in edge cases such as 0, -1 and Int.MAX_VALUE.
- 4Build domain generators with Arb.bind, which shrinks every field. An arbitrary { } block does not shrink without an explicit Shrinker.
- 5A failed run prints its seed; pass PropTestConfig(seed = ...) to replay it.
- 6Test logic layers (ViewModels, repos, mappers) rather than UI components.
Frequently Asked
When should I use property-based testing?
Use it for pure functions with clear invariants: data transformations, serialization, parsers, mappers and arithmetic. Those have properties you can state in one line, such as a roundtrip or an idempotency rule. It is less useful for UI code or heavily stateful systems, where the input space is large and the property is hard to pin down.
How is this different from fuzzing?
Both generate inputs, but a property test checks a rule you wrote and then shrinks a failure to a minimal case, while fuzzing is usually aimed at crashes. Kotest also mixes in fixed edge cases such as 0, -1 and Int.MAX_VALUE, runs 1,000 iterations by default, and prints the seed so a failure can be replayed.
Are property-based tests flaky?
They are random, not flaky. Each run uses a new seed, so a property can pass many times and then fail on an input it had not tried before. Kotest prints the seed on failure and saves it under ~/.kotest/seeds, so the next local run replays the same failure until the code is fixed.
Ready to architect your next Android app?
ANDROID-ARCHITECT generates production-ready Kotlin code, architecture blueprints, and CI/CD configurations from plain-language descriptions. Start building for free.
ANDROID-ARCHITECT · CONSOLE
S/N · AX-1A-00001
STRATEGIA-X · ENGINEERED · IN · CALIFORNIA