fix: benchmark infrastructure — JVM heap, Liquibase, MockProducer leak#48
Merged
Conversation
Three independent fixes that make ./gradlew :okapi-benchmarks:jmh complete cleanly. Before these, the JMH run OOMs partway through. 1. Bump JMH JVM heap from -Xmx2g to -Xmx8g. Throughput-mode microbenchmarks call deliver() at ~1M ops/s; each call allocates Jackson + Kotlin reflection state for deserialization. At 2GB the allocation rate exceeds GC throughput and OOMs within the first measurement iteration. 2. Pass -Dliquibase.duplicateFileMode=WARN as JMH JVM arg. okapi-postgres.jar and the fat JMH jar both ship the changelog at the same classpath path. Liquibase 4.x treats duplicate resources as an error by default, which aborts PostgresBenchmarkSupport setup. The two files are identical (same jar source on the classpath twice), so WARN is safe. 3. Subclass MockProducer in DelivererMicroBenchmark to clear() history after every send(). MockProducer.history (internal `sent` list) retains every record sent for inspection — there is no eviction. In throughput mode at ~1M ops/s for 30s × forks × iterations that list grew to GBs and OOMed the JVM regardless of heap size. Discarding per call is safe because microbench doesn't inspect what was sent — only timing. All three issues exist on main today; running the benchmark suite without these fixes will fail. No test or production code is touched.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three independent fixes that make `./gradlew :okapi-benchmarks:jmh` complete cleanly. Before these, the JMH run OOMs partway through. All three issues exist on main today; running the benchmark suite without these fixes will fail.
No test or production code is touched — pure benchmark infrastructure.
Fixes
1. Bump JMH JVM heap to `-Xmx8g`
Throughput-mode microbenchmarks call `deliver()` at ~1M ops/s; each call allocates Jackson + Kotlin reflection state for JSON deserialization. At the previous default `-Xmx2g` the allocation rate exceeds GC throughput and OOMs within the first measurement iteration.
2. Pass `-Dliquibase.duplicateFileMode=WARN` as JMH JVM arg
`okapi-postgres.jar` and the fat JMH jar both ship the changelog at the same classpath path (`com/softwaremill/okapi/db/postgres/changelog.xml`). Liquibase 4.x treats duplicate resources as an error by default, which aborts `PostgresBenchmarkSupport` setup. The two files are identical (same jar source on the classpath twice), so `WARN` is safe.
3. Subclass `MockProducer` in `DelivererMicroBenchmark` to `clear()` history after every `send()`
`MockProducer.history` (internal `sent` list) retains every record sent for inspection — there is no eviction. In throughput mode at ~1M ops/s for 30s × forks × iterations that list grew to GBs and OOMed the JVM regardless of heap size. Discarding per call is safe because microbench doesn't inspect what was sent — only timing.
With this fix, `DelivererMicroBenchmark.kafkaDeliver` now produces meaningful numbers (~2.3M ops/s ± <1%) instead of `error > score`.
Files
Why a separate PR
These are pure infrastructure fixes — completely independent of any specific benchmark or transport implementation. Carved out from PR #46 (KOJAK-82) so they can land on main right away, without waiting for the Kafka deliverBatch (#40) review cycle. PR #46 will then contain only the refreshed JMH numbers.
Test plan