Zimin A.N.
c11044c32e
CEO plan v1 polish — закрывает gap "JMH performance benchmarks в CI для baseline + Redis-enabled scenarios". New module: - ordinis-bench/ — JMH 1.37 microbenchmarks. Gated behind `bench` Maven profile в parent pom — НЕ строится при обычном `mvn package` (default CI остаётся быстрым). - maven-shade-plugin под `bench` profile собирает uber-jar `target/benchmarks.jar` (86 MB, ~12 sec build). - LineageBenchmark covers hot paths из dict-relationships-v2: * parseRef_simple / withSchemaButNoOnClose / withOnClose * onCloseAction_block / cascade / null * schemaTraversal_findRefs (40 props, 5 refs — realistic shape) Baseline numbers (M3 MBP, JDK 25, single thread): - onCloseAction_*: 0.6–12 ns/op - parseRef_*: 20–93 ns/op - schemaTraversal: 417 ns/op (~83 ns per ref на 40 props) Cumulative budget for findSchemaDependents на 40 dicts × 40 props × 5 refs: ~17 µs. p99 endpoint SLO = 200 ms — запас 4 порядка. README documents budget + 2x regression alarm. CI integration: - New job maven-bench (stage: test). Manual trigger only: * RUN_BENCH=true variable, OR * web pipeline source с manual click. - Default CI (push/merge) НЕ запускает bench — slow + noisy в history. - Output artifacts: bench-results.json + bench-results.txt (5 day expire). - Quick smoke config: 1 fork, 2 warmups, 3 measurements × 2s = ~5 min total. Side effect: ReferenceValidator.parseRef static methods переведены из package-private в public. Они и так часть public ParsedRef contract — visible to bench module без изменения design intent. Verify: - mvn -P bench -pl ordinis-bench -am package: BUILD SUCCESS (12s). - java -jar benchmarks.jar -f 1 -wi 1 -i 2 Lineage: all 7 benchmarks exec'ятся, valid ns/op numbers. - mvn -pl ordinis-rest-api -am test: 106/106 PASS unchanged. - glab ci lint: clean. README в ordinis-bench/README.md документирует: - baseline numbers + 2x regression budget - local run options (smoke / full / GC profile / JSON output) - when CI bench runs (manual web trigger) - difference vs k6/wrk (HTTP load tests are different layer, both needed) - how to add new benchmark
102 lines
3.6 KiB
Markdown
102 lines
3.6 KiB
Markdown
# ordinis-bench
|
||
|
||
JMH microbenchmarks для hot paths Ordinis. Не deployable artifact —
|
||
только для ad-hoc и CI manual runs.
|
||
|
||
## Цель
|
||
|
||
Catch регрессии алгоритмической сложности на критичных горячих путях:
|
||
|
||
- **Schema traversal** в `LineageIndexService.findSchemaDependents` —
|
||
iterate properties, find x-references, parse каждый.
|
||
- **`ReferenceValidator.parseRef`** — schema reference parsing (вызывается
|
||
на bundle import + per-record CRUD validation).
|
||
- **`OnCloseAction.fromString`** — case-insensitive enum lookup внутри parseRef.
|
||
|
||
## Когда обновлять baseline
|
||
|
||
Базовые цифры (M3 MacBook Pro, JDK 25, single thread, 1 fork × 5 measurements):
|
||
|
||
| Benchmark | ns/op | Notes |
|
||
|---|---|---|
|
||
| `onCloseAction_null` | < 1 | trivial null branch |
|
||
| `onCloseAction_block` | ~5 | first enum match |
|
||
| `onCloseAction_cascade` | ~12 | third enum match (linear values() scan) |
|
||
| `parseRef_simple` | ~20 | без property schema |
|
||
| `parseRef_withSchemaButNoOnClose` | ~25 | + JsonNode lookup, no on-close |
|
||
| `parseRef_withOnClose` | ~93 | + on-close parsing |
|
||
| `schemaTraversal_findRefs` (40 props) | ~417 | ~83 ns per ref |
|
||
|
||
Если PR делает любой из этих в 2x медленнее — investigate.
|
||
Cumulative budget for `findSchemaDependents` на 40 dicts × 40 props × 5 refs:
|
||
40 * 417 ns ≈ 17 µs. p99 endpoint SLO = 200 ms — запас 4 порядка.
|
||
|
||
## Run locally
|
||
|
||
Build uber-jar (~12 sec, includes shade):
|
||
|
||
```bash
|
||
mvn -P bench -pl ordinis-bench -am package -DskipTests
|
||
```
|
||
|
||
Run all benchmarks (~5-7 min total):
|
||
|
||
```bash
|
||
java -jar ordinis-bench/target/benchmarks.jar
|
||
```
|
||
|
||
Filter by name:
|
||
|
||
```bash
|
||
java -jar ordinis-bench/target/benchmarks.jar Lineage
|
||
java -jar ordinis-bench/target/benchmarks.jar parseRef
|
||
```
|
||
|
||
Quick smoke (1 fork, 1 warmup, 2 measurements × 1 sec — finishes in ~10 sec):
|
||
|
||
```bash
|
||
java -jar ordinis-bench/target/benchmarks.jar -f 1 -wi 1 -i 2 -w 1 -r 1 Lineage
|
||
```
|
||
|
||
Profile с GC:
|
||
|
||
```bash
|
||
java -jar ordinis-bench/target/benchmarks.jar -prof gc Lineage
|
||
```
|
||
|
||
JSON output для дальнейшего парсинга:
|
||
|
||
```bash
|
||
java -jar ordinis-bench/target/benchmarks.jar -rf json -rff bench-results.json
|
||
```
|
||
|
||
## Run в CI
|
||
|
||
Bench job — **manual trigger only** через `web` source в GitLab.
|
||
По default не fire'ит на push (slow + noisy в CI history).
|
||
|
||
Trigger:
|
||
1. https://git.nstart.cloud/2-6/2-6-4/terravault/ordinis/-/pipelines/new
|
||
2. Variable: `RUN_BENCH=true`
|
||
3. Run pipeline
|
||
|
||
Output: artifact `bench-results.json` + `bench-results.txt` (5 day expire).
|
||
|
||
## Adding new benchmark
|
||
|
||
1. Создать класс в `cloud.nstart.terravault.ordinis.bench` package.
|
||
2. Annotate с `@BenchmarkMode(Mode.AverageTime)` + `@OutputTimeUnit(TimeUnit.NANOSECONDS)`.
|
||
3. Добавить `@Warmup`, `@Measurement`, `@Fork`, `@State(Scope.Benchmark)`.
|
||
4. JMH annotation processor автоматически генерирует runner classes на compile.
|
||
5. Update baseline numbers в этом README после первого fork × 5 run'а.
|
||
|
||
## Why JMH and not k6 / wrk
|
||
|
||
- JMH = JVM-level microbenchmarks. Catches regressions в алгоритмах
|
||
(parseRef становится O(n²), regex compilation per call, allocation hot spot).
|
||
- k6 / wrk = HTTP load tests. Тестируют end-to-end SLO — это другой layer
|
||
(см. Phase 4 Prometheus alerts: `OrdinisDependentsLatencyHigh`,
|
||
`OrdinisCascadeCloseTimeouts`).
|
||
- Оба нужны: JMH для hot paths сразу при PR review, k6 для periodic SLO
|
||
validation против staging/prod.
|