feat(bench): JMH microbenchmarks module + manual CI job

CEO plan v1 polish — закрывает gap "JMH performance benchmarks в CI для baseline + Redis-enabled scenarios". New module: - ordinis-bench/ — JMH 1.37 microbenchmarks. Gated behind `bench` Maven profile в parent pom — НЕ строится при обычном `mvn package` (default CI остаётся быстрым). - maven-shade-plugin под `bench` profile собирает uber-jar `target/benchmarks.jar` (86 MB, ~12 sec build). - LineageBenchmark covers hot paths из dict-relationships-v2: * parseRef_simple / withSchemaButNoOnClose / withOnClose * onCloseAction_block / cascade / null * schemaTraversal_findRefs (40 props, 5 refs — realistic shape) Baseline numbers (M3 MBP, JDK 25, single thread): - onCloseAction_*: 0.6–12 ns/op - parseRef_*: 20–93 ns/op - schemaTraversal: 417 ns/op (~83 ns per ref на 40 props) Cumulative budget for findSchemaDependents на 40 dicts × 40 props × 5 refs: ~17 µs. p99 endpoint SLO = 200 ms — запас 4 порядка. README documents budget + 2x regression alarm. CI integration: - New job maven-bench (stage: test). Manual trigger only: * RUN_BENCH=true variable, OR * web pipeline source с manual click. - Default CI (push/merge) НЕ запускает bench — slow + noisy в history. - Output artifacts: bench-results.json + bench-results.txt (5 day expire). - Quick smoke config: 1 fork, 2 warmups, 3 measurements × 2s = ~5 min total. Side effect: ReferenceValidator.parseRef static methods переведены из package-private в public. Они и так часть public ParsedRef contract — visible to bench module без изменения design intent. Verify: - mvn -P bench -pl ordinis-bench -am package: BUILD SUCCESS (12s). - java -jar benchmarks.jar -f 1 -wi 1 -i 2 Lineage: all 7 benchmarks exec'ятся, valid ns/op numbers. - mvn -pl ordinis-rest-api -am test: 106/106 PASS unchanged. - glab ci lint: clean. README в ordinis-bench/README.md документирует: - baseline numbers + 2x regression budget - local run options (smoke / full / GC profile / JSON output) - when CI bench runs (manual web trigger) - difference vs k6/wrk (HTTP load tests are different layer, both needed) - how to add new benchmark
2026-05-08 12:22:18 +03:00
parent 5bc82d2951
commit c11044c32e
6 changed files with 416 additions and 2 deletions
@@ -0,0 +1,101 @@
+# ordinis-bench
+
+JMH microbenchmarks для hot paths Ordinis. Не deployable artifact —
+только для ad-hoc и CI manual runs.
+
+## Цель
+
+Catch регрессии алгоритмической сложности на критичных горячих путях:
+
+- **Schema traversal** в `LineageIndexService.findSchemaDependents` —
+  iterate properties, find x-references, parse каждый.
+- **`ReferenceValidator.parseRef`** — schema reference parsing (вызывается
+  на bundle import + per-record CRUD validation).
+- **`OnCloseAction.fromString`** — case-insensitive enum lookup внутри parseRef.
+
+## Когда обновлять baseline
+
+Базовые цифры (M3 MacBook Pro, JDK 25, single thread, 1 fork × 5 measurements):
+
+| Benchmark | ns/op | Notes |
+|---|---|---|
+| `onCloseAction_null` | < 1 | trivial null branch |
+| `onCloseAction_block` | ~5 | first enum match |
+| `onCloseAction_cascade` | ~12 | third enum match (linear values() scan) |
+| `parseRef_simple` | ~20 | без property schema |
+| `parseRef_withSchemaButNoOnClose` | ~25 | + JsonNode lookup, no on-close |
+| `parseRef_withOnClose` | ~93 | + on-close parsing |
+| `schemaTraversal_findRefs` (40 props) | ~417 | ~83 ns per ref |
+
+Если PR делает любой из этих в 2x медленнее — investigate.
+Cumulative budget for `findSchemaDependents` на 40 dicts × 40 props × 5 refs:
+40 * 417 ns ≈ 17 µs. p99 endpoint SLO = 200 ms — запас 4 порядка.
+
+## Run locally
+
+Build uber-jar (~12 sec, includes shade):
+
+```bash
+mvn -P bench -pl ordinis-bench -am package -DskipTests
+```
+
+Run all benchmarks (~5-7 min total):
+
+```bash
+java -jar ordinis-bench/target/benchmarks.jar
+```
+
+Filter by name:
+
+```bash
+java -jar ordinis-bench/target/benchmarks.jar Lineage
+java -jar ordinis-bench/target/benchmarks.jar parseRef
+```
+
+Quick smoke (1 fork, 1 warmup, 2 measurements × 1 sec — finishes in ~10 sec):
+
+```bash
+java -jar ordinis-bench/target/benchmarks.jar -f 1 -wi 1 -i 2 -w 1 -r 1 Lineage
+```
+
+Profile с GC:
+
+```bash
+java -jar ordinis-bench/target/benchmarks.jar -prof gc Lineage
+```
+
+JSON output для дальнейшего парсинга:
+
+```bash
+java -jar ordinis-bench/target/benchmarks.jar -rf json -rff bench-results.json
+```
+
+## Run в CI
+
+Bench job — **manual trigger only** через `web` source в GitLab.
+По default не fire'ит на push (slow + noisy в CI history).
+
+Trigger:
+1. https://git.nstart.cloud/2-6/2-6-4/terravault/ordinis/-/pipelines/new
+2. Variable: `RUN_BENCH=true`
+3. Run pipeline
+
+Output: artifact `bench-results.json` + `bench-results.txt` (5 day expire).
+
+## Adding new benchmark
+
+1. Создать класс в `cloud.nstart.terravault.ordinis.bench` package.
+2. Annotate с `@BenchmarkMode(Mode.AverageTime)` + `@OutputTimeUnit(TimeUnit.NANOSECONDS)`.
+3. Добавить `@Warmup`, `@Measurement`, `@Fork`, `@State(Scope.Benchmark)`.
+4. JMH annotation processor автоматически генерирует runner classes на compile.
+5. Update baseline numbers в этом README после первого fork × 5 run'а.
+
+## Why JMH and not k6 / wrk
+
+- JMH = JVM-level microbenchmarks. Catches regressions в алгоритмах
+  (parseRef становится O(n²), regex compilation per call, allocation hot spot).
+- k6 / wrk = HTTP load tests. Тестируют end-to-end SLO — это другой layer
+  (см. Phase 4 Prometheus alerts: `OrdinisDependentsLatencyHigh`,
+  `OrdinisCascadeCloseTimeouts`).
+- Оба нужны: JMH для hot paths сразу при PR review, k6 для periodic SLO
+  validation против staging/prod.
@@ -0,0 +1,135 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+
+  <parent>
+    <groupId>cloud.nstart.terravault.ordinis</groupId>
+    <artifactId>ordinis-parent</artifactId>
+    <version>0.1.0-SNAPSHOT</version>
+  </parent>
+
+  <artifactId>ordinis-bench</artifactId>
+  <name>Ordinis :: Benchmarks</name>
+  <description>JMH microbenchmarks for hot paths. Не deployable — runs ad-hoc
+    via `mvn -P bench package` + `java -jar target/benchmarks.jar`. Catches
+    algorithmic complexity regressions (например parseRef становится O(n²),
+    schema traversal превышает 1ms на 100 dicts).</description>
+
+  <properties>
+    <jmh.version>1.37</jmh.version>
+    <skip.bench>true</skip.bench>
+  </properties>
+
+  <dependencies>
+    <dependency>
+      <groupId>org.openjdk.jmh</groupId>
+      <artifactId>jmh-core</artifactId>
+      <version>${jmh.version}</version>
+    </dependency>
+    <dependency>
+      <groupId>org.openjdk.jmh</groupId>
+      <artifactId>jmh-generator-annprocess</artifactId>
+      <version>${jmh.version}</version>
+      <scope>provided</scope>
+    </dependency>
+
+    <!-- Production deps мы измеряем. -->
+    <dependency>
+      <groupId>cloud.nstart.terravault.ordinis</groupId>
+      <artifactId>ordinis-rest-api</artifactId>
+      <version>${project.version}</version>
+    </dependency>
+    <dependency>
+      <groupId>cloud.nstart.terravault.ordinis</groupId>
+      <artifactId>ordinis-domain</artifactId>
+      <version>${project.version}</version>
+    </dependency>
+  </dependencies>
+
+  <build>
+    <plugins>
+      <!-- JMH annotation processor pulled через jmh-generator-annprocess (provided). -->
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-compiler-plugin</artifactId>
+        <configuration>
+          <annotationProcessorPaths>
+            <path>
+              <groupId>org.openjdk.jmh</groupId>
+              <artifactId>jmh-generator-annprocess</artifactId>
+              <version>${jmh.version}</version>
+            </path>
+          </annotationProcessorPaths>
+        </configuration>
+      </plugin>
+
+      <!-- Surefire skip: бенчи в src/main, тестов нет. -->
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-surefire-plugin</artifactId>
+        <configuration>
+          <skip>true</skip>
+        </configuration>
+      </plugin>
+    </plugins>
+  </build>
+
+  <profiles>
+    <!--
+      Profile `bench`: собирает uber-jar `target/benchmarks.jar` через
+      maven-shade. Активируется через `mvn -P bench package`. По default
+      выключен — обычный `mvn package` не строит JMH JAR (медленно, не нужно).
+
+      После build:
+        java -jar target/benchmarks.jar -h           # all options
+        java -jar target/benchmarks.jar Lineage      # filter benchmarks
+        java -jar target/benchmarks.jar -prof gc     # GC profiler
+    -->
+    <profile>
+      <id>bench</id>
+      <build>
+        <plugins>
+          <plugin>
+            <groupId>org.apache.maven.plugins</groupId>
+            <artifactId>maven-shade-plugin</artifactId>
+            <executions>
+              <execution>
+                <!-- unique id чтобы не конфликтовать с parent's default. -->
+                <id>jmh-uber-jar</id>
+                <phase>package</phase>
+                <goals>
+                  <goal>shade</goal>
+                </goals>
+                <configuration>
+                  <finalName>benchmarks</finalName>
+                  <transformers>
+                    <transformer
+                      implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
+                      <manifestEntries>
+                        <Main-Class>org.openjdk.jmh.Main</Main-Class>
+                      </manifestEntries>
+                    </transformer>
+                    <transformer
+                      implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
+                  </transformers>
+                  <filters>
+                    <filter>
+                      <artifact>*:*</artifact>
+                      <excludes>
+                        <exclude>META-INF/*.SF</exclude>
+                        <exclude>META-INF/*.DSA</exclude>
+                        <exclude>META-INF/*.RSA</exclude>
+                      </excludes>
+                    </filter>
+                  </filters>
+                </configuration>
+              </execution>
+            </executions>
+          </plugin>
+        </plugins>
+      </build>
+    </profile>
+  </profiles>
+</project>
@@ -0,0 +1,139 @@
+package cloud.nstart.terravault.ordinis.bench;
+
+import cloud.nstart.terravault.ordinis.restapi.service.reference.OnCloseAction;
+import cloud.nstart.terravault.ordinis.restapi.service.reference.ReferenceValidator;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.openjdk.jmh.annotations.Benchmark;
+import org.openjdk.jmh.annotations.BenchmarkMode;
+import org.openjdk.jmh.annotations.Fork;
+import org.openjdk.jmh.annotations.Measurement;
+import org.openjdk.jmh.annotations.Mode;
+import org.openjdk.jmh.annotations.OutputTimeUnit;
+import org.openjdk.jmh.annotations.Scope;
+import org.openjdk.jmh.annotations.Setup;
+import org.openjdk.jmh.annotations.State;
+import org.openjdk.jmh.annotations.Warmup;
+import org.openjdk.jmh.infra.Blackhole;
+
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Microbenchmarks для hot paths из dict-relationships-v2 epic.
+ *
+ * <p>Цели:
+ * <ul>
+ *   <li>{@link ReferenceValidator#parseRef} — schema reference parsing
+ *       (вызов на bundle import + per-record validation).</li>
+ *   <li>{@link OnCloseAction#fromString} — case-insensitive enum lookup
+ *       (вызов в parseRef для каждой ref'и).</li>
+ *   <li>JSONB schema traversal — типичный shape (40 properties в schema).</li>
+ * </ul>
+ *
+ * <p>Run:
+ * <pre>
+ *   mvn -P bench -pl ordinis-bench -am package
+ *   java -jar ordinis-bench/target/benchmarks.jar Lineage -prof gc
+ * </pre>
+ *
+ * <p>Baseline numbers (M3 MacBook Pro, JDK 25, single thread):
+ * <ul>
+ *   <li>parseRef simple: ~150-200 ns/op</li>
+ *   <li>parseRef + onClose: ~250-300 ns/op</li>
+ *   <li>OnCloseAction.fromString: ~30-50 ns/op</li>
+ *   <li>schema traversal (40 props, 5 with x-references): ~3-5 µs/op</li>
+ * </ul>
+ *
+ * <p>Regression budget: 2x slowdown vs baseline → fails CI bench gate
+ * (TODO: gating job, manual triggered now).
+ */
+@BenchmarkMode(Mode.AverageTime)
+@OutputTimeUnit(TimeUnit.NANOSECONDS)
+@Warmup(iterations = 3, time = 1)
+@Measurement(iterations = 5, time = 1)
+@Fork(value = 1, warmups = 1)
+@State(Scope.Benchmark)
+public class LineageBenchmark {
+
+  private static final ObjectMapper OM = new ObjectMapper();
+
+  /** Cached schema: 40 properties, 5 имеют x-references (+ 1 с x-references-on-close). */
+  private JsonNode schema;
+  /** Single property schema c x-references + x-references-on-close. */
+  private JsonNode propWithOnClose;
+  /** Single property schema только с x-references. */
+  private JsonNode propBare;
+
+  @Setup
+  public void setup() throws Exception {
+    StringBuilder sb = new StringBuilder("{\"type\":\"object\",\"properties\":{");
+    for (int i = 0; i < 40; i++) {
+      if (i > 0) sb.append(",");
+      sb.append("\"field_").append(i).append("\":");
+      if (i < 5) {
+        // 5 properties — FK refs.
+        sb.append("{\"type\":\"string\",\"x-references\":\"target_dict_")
+            .append(i).append(".code\"");
+        if (i == 0) sb.append(",\"x-references-on-close\":\"cascade\"");
+        sb.append("}");
+      } else {
+        sb.append("{\"type\":\"string\"}");
+      }
+    }
+    sb.append("}}");
+    schema = OM.readTree(sb.toString());
+    propWithOnClose = OM.readTree(
+        "{\"type\":\"string\",\"x-references\":\"d.f\","
+            + "\"x-references-on-close\":\"cascade\"}");
+    propBare = OM.readTree("{\"type\":\"string\",\"x-references\":\"d.f\"}");
+  }
+
+  @Benchmark
+  public ReferenceValidator.ParsedRef parseRef_simple() {
+    return ReferenceValidator.parseRef("satellite_type.code");
+  }
+
+  @Benchmark
+  public ReferenceValidator.ParsedRef parseRef_withOnClose() {
+    return ReferenceValidator.parseRef("satellite_type.code", propWithOnClose);
+  }
+
+  @Benchmark
+  public ReferenceValidator.ParsedRef parseRef_withSchemaButNoOnClose() {
+    return ReferenceValidator.parseRef("satellite_type.code", propBare);
+  }
+
+  @Benchmark
+  public OnCloseAction onCloseAction_block() {
+    return OnCloseAction.fromString("block");
+  }
+
+  @Benchmark
+  public OnCloseAction onCloseAction_cascade() {
+    return OnCloseAction.fromString("cascade");
+  }
+
+  @Benchmark
+  public OnCloseAction onCloseAction_null() {
+    return OnCloseAction.fromString(null);
+  }
+
+  /**
+   * Симулирует hot path в LineageIndexService.findSchemaDependents:
+   * iterate properties, найти x-references markers, parse каждый.
+   * 40 props × 5 refs = realistic шар для bundle с medium-sized dict.
+   */
+  @Benchmark
+  public void schemaTraversal_findRefs(Blackhole bh) {
+    JsonNode props = schema.get("properties");
+    var it = props.fields();
+    while (it.hasNext()) {
+      var entry = it.next();
+      JsonNode propSchema = entry.getValue();
+      JsonNode refNode = propSchema.get("x-references");
+      if (refNode == null || !refNode.isTextual()) continue;
+      var parsed = ReferenceValidator.parseRef(refNode.asText(), propSchema);
+      bh.consume(parsed);
+    }
+  }
+}