Files

T

Zimin A.N. c11044c32e feat(bench): JMH microbenchmarks module + manual CI job

CEO plan v1 polish — закрывает gap "JMH performance benchmarks в CI для
baseline + Redis-enabled scenarios".

New module:
- ordinis-bench/ — JMH 1.37 microbenchmarks. Gated behind `bench` Maven
  profile в parent pom — НЕ строится при обычном `mvn package` (default
  CI остаётся быстрым).
- maven-shade-plugin под `bench` profile собирает uber-jar
  `target/benchmarks.jar` (86 MB, ~12 sec build).
- LineageBenchmark covers hot paths из dict-relationships-v2:
  * parseRef_simple / withSchemaButNoOnClose / withOnClose
  * onCloseAction_block / cascade / null
  * schemaTraversal_findRefs (40 props, 5 refs — realistic shape)

Baseline numbers (M3 MBP, JDK 25, single thread):
- onCloseAction_*: 0.6–12 ns/op
- parseRef_*: 20–93 ns/op
- schemaTraversal: 417 ns/op (~83 ns per ref на 40 props)
Cumulative budget for findSchemaDependents на 40 dicts × 40 props × 5 refs:
~17 µs. p99 endpoint SLO = 200 ms — запас 4 порядка. README documents
budget + 2x regression alarm.

CI integration:
- New job maven-bench (stage: test). Manual trigger only:
  * RUN_BENCH=true variable, OR
  * web pipeline source с manual click.
- Default CI (push/merge) НЕ запускает bench — slow + noisy в history.
- Output artifacts: bench-results.json + bench-results.txt (5 day expire).
- Quick smoke config: 1 fork, 2 warmups, 3 measurements × 2s = ~5 min total.

Side effect: ReferenceValidator.parseRef static methods переведены из
package-private в public. Они и так часть public ParsedRef contract —
visible to bench module без изменения design intent.

Verify:
- mvn -P bench -pl ordinis-bench -am package: BUILD SUCCESS (12s).
- java -jar benchmarks.jar -f 1 -wi 1 -i 2 Lineage: all 7 benchmarks
  exec'ятся, valid ns/op numbers.
- mvn -pl ordinis-rest-api -am test: 106/106 PASS unchanged.
- glab ci lint: clean.

README в ordinis-bench/README.md документирует:
- baseline numbers + 2x regression budget
- local run options (smoke / full / GC profile / JSON output)
- when CI bench runs (manual web trigger)
- difference vs k6/wrk (HTTP load tests are different layer, both needed)
- how to add new benchmark

2026-05-08 12:22:18 +03:00

3.6 KiB

Raw Permalink Blame History

ordinis-bench

JMH microbenchmarks для hot paths Ordinis. Не deployable artifact — только для ad-hoc и CI manual runs.

Цель

Catch регрессии алгоритмической сложности на критичных горячих путях:

Schema traversal в LineageIndexService.findSchemaDependents — iterate properties, find x-references, parse каждый.
ReferenceValidator.parseRef — schema reference parsing (вызывается на bundle import + per-record CRUD validation).
OnCloseAction.fromString — case-insensitive enum lookup внутри parseRef.

Когда обновлять baseline

Базовые цифры (M3 MacBook Pro, JDK 25, single thread, 1 fork × 5 measurements):

Benchmark	ns/op	Notes
`onCloseAction_null`	< 1	trivial null branch
`onCloseAction_block`	~5	first enum match
`onCloseAction_cascade`	~12	third enum match (linear values() scan)
`parseRef_simple`	~20	без property schema
`parseRef_withSchemaButNoOnClose`	~25	+ JsonNode lookup, no on-close
`parseRef_withOnClose`	~93	+ on-close parsing
`schemaTraversal_findRefs` (40 props)	~417	~83 ns per ref

Если PR делает любой из этих в 2x медленнее — investigate. Cumulative budget for findSchemaDependents на 40 dicts × 40 props × 5 refs: 40 * 417 ns ≈ 17 µs. p99 endpoint SLO = 200 ms — запас 4 порядка.

Run locally

Build uber-jar (~12 sec, includes shade):

mvn -P bench -pl ordinis-bench -am package -DskipTests

Run all benchmarks (~5-7 min total):

java -jar ordinis-bench/target/benchmarks.jar

Filter by name:

java -jar ordinis-bench/target/benchmarks.jar Lineage
java -jar ordinis-bench/target/benchmarks.jar parseRef

Quick smoke (1 fork, 1 warmup, 2 measurements × 1 sec — finishes in ~10 sec):

java -jar ordinis-bench/target/benchmarks.jar -f 1 -wi 1 -i 2 -w 1 -r 1 Lineage

Profile с GC:

java -jar ordinis-bench/target/benchmarks.jar -prof gc Lineage

JSON output для дальнейшего парсинга:

java -jar ordinis-bench/target/benchmarks.jar -rf json -rff bench-results.json

Run в CI

Bench job — manual trigger only через web source в GitLab. По default не fire'ит на push (slow + noisy в CI history).

Trigger:

https://git.nstart.cloud/2-6/2-6-4/terravault/ordinis/-/pipelines/new
Variable: RUN_BENCH=true
Run pipeline

Output: artifact bench-results.json + bench-results.txt (5 day expire).

Adding new benchmark

Создать класс в cloud.nstart.terravault.ordinis.bench package.
Annotate с @BenchmarkMode(Mode.AverageTime) + @OutputTimeUnit(TimeUnit.NANOSECONDS).
Добавить @Warmup, @Measurement, @Fork, @State(Scope.Benchmark).
JMH annotation processor автоматически генерирует runner classes на compile.
Update baseline numbers в этом README после первого fork × 5 run'а.

Why JMH and not k6 / wrk

JMH = JVM-level microbenchmarks. Catches regressions в алгоритмах (parseRef становится O(n²), regex compilation per call, allocation hot spot).
k6 / wrk = HTTP load tests. Тестируют end-to-end SLO — это другой layer (см. Phase 4 Prometheus alerts: OrdinisDependentsLatencyHigh, OrdinisCascadeCloseTimeouts).
Оба нужны: JMH для hot paths сразу при PR review, k6 для periodic SLO validation против staging/prod.

3.6 KiB Raw Permalink Blame History Unescape Escape