docs(design): AI schema assist + dictionary marketplace v1 proposals

2026-05-12 15:10:22 +00:00
parent 4aa5fd97d1
commit cef5f49bf7
2 changed files with 678 additions and 0 deletions
@@ -0,0 +1,399 @@
+# Design: Dictionary Bundle Marketplace
+
+**Author:** zimin.an
+**Date:** 2026-05-12
+**Status:** PROPOSED v1 — needs `/office-hours` для premise validation + `/plan-eng-review`
+**Scope:** ДЗЗ domain, multi-tenant ready (corp private bundles + curated public ДЗЗ catalog)
+**Sprint estimate:** **~1.5-2 спринта** (10-14 days CC), v1 internal-only (no public catalog UI)
+**Blockers:** none, но рекомендация defer'a до v2.14.0 prod stable + 2 weeks dogfood
+
+---
+
+## TL;DR
+
+Сейчас каждый customer install Ordinis строит свой `ordinis-cuod-bundle` Maven module from scratch. ЦУОД bundle = 40 dictionaries (КА, типы, ground stations, частотные диапазоны, операторы и т.д.) — это ~2-3 недели работы дублируется при onboarding нового customer'а. **Нет shared catalog**, нет «бери готовое».
+
+**Предложение:** Bundle marketplace. Авторы публикуют bundle versions в реестр (corp Nexus → потенциально public ДЗЗ catalog). Admin в running instance видит каталог, install'нет нужный bundle (с optional sample data seed), получает 40 готовых справочников за минуту.
+
+**Win:** Customer onboarding 2-3 недели → 1 день. Sales: «у нас готовая ДЗЗ-библиотека из 40+ справочников». Compounding value: каждый install обогащает catalog.
+
+**Risk:** schema conflicts при install, malicious bundles, version drift breaking consumer integrations. Mitigations через signed manifests + dry-run preview + namespace prefix.
+
+---
+
+## Current state
+
+Сейчас в Ordinis:
+- `ordinis-cuod-bundle` Maven module → ЦУОД-specific dictionaries + sample data в `src/main/resources/bundles/cuod/`
+- Build-time bundle: компилируется в jar, deploy'ится с app
+- Single-bundle per installation (multi-bundle architectural future, не shipped)
+- No runtime install / browse / discover
+
+```
+[ Customer A install ]              [ Customer B install ]
+  ↓ docker pull ordinis:vX            ↓ docker pull ordinis:vX
+  + ordinis-cuod-bundle               + ordinis-customer-b-bundle
+  (build-time, requires Maven build)  (нет — customer пишет свой Maven module)
+```
+
+Каждый новый customer = **new Maven module от руки**.
+
+## Что хочется
+
+```
+[ Customer admin opens /catalog ]
+  ↓
+  Browse: ЦУОД ДЗЗ Bundle v1.4.0 [40 dicts]
+         Гидрометео ДЗЗ Bundle v0.8.0 [12 dicts]
+         Базовый справочник стран v2.0.0 [1 dict]
+  ↓
+  [ Install ] → dry-run preview → confirm
+  ↓
+  POST /api/v1/bundles/install { bundleId, version, options }
+  ↓
+  40 dictionaries created in DB через standard `DictionaryDefinitionService`
+  + optional sample data seeded
+  + bundle metadata recorded в `installed_bundles` table
+  ↓
+  Done in ~30 sec
+```
+
+---
+
+## Architecture
+
+```
+                          Bundle Registry
+                          (Nexus / GitLab Package Registry)
+                          - immutable versioned artifacts
+                          - signed bundles (corp PKI)
+                          ▲
+                          │ npm/maven-like resolve
+                          │
+       ┌──────────────────────────────────────────┐
+       │  ordinis-rest-api (NEW endpoints)         │
+       │  BundleCatalogController                  │
+       │    GET  /api/v1/bundles/available         │
+       │    GET  /api/v1/bundles/{id}/versions     │
+       │  BundleInstallController                  │
+       │    POST /api/v1/bundles/install (dry-run +│
+       │                                  apply)   │
+       │    GET  /api/v1/bundles/installed         │
+       │    POST /api/v1/bundles/{id}/uninstall    │
+       └────────────────┬──────────────────────────┘
+                        │
+       ┌────────────────▼──────────────────────────┐
+       │  BundleResolver (NEW component)           │
+       │   - fetch manifest.yaml from registry     │
+       │   - verify signature (corp PKI ed25519)   │
+       │   - parse dictionary defs + sample data   │
+       │   - check conflicts with existing dicts   │
+       └────────────────┬──────────────────────────┘
+                        │
+       ┌────────────────▼──────────────────────────┐
+       │  Standard DictionaryDefinitionService     │
+       │   - create dict (per-dict CREATE event)   │
+       │   - seed sample records (if opted in)     │
+       │   - outbox → Kafka → downstream consumers │
+       └───────────────────────────────────────────┘
+                        ▲
+       ┌────────────────┴──────────────────────────┐
+       │  ordinis-admin-ui                         │
+       │  BundleCatalogPage.tsx (NEW route)        │
+       │  BundleInstallModal.tsx (dry-run preview) │
+       │  InstalledBundlesTab (manage existing)    │
+       └───────────────────────────────────────────┘
+```
+
+### Bundle manifest format
+
+```yaml
+# bundle.yaml — published artifact metadata
+apiVersion: ordinis.io/v1
+kind: Bundle
+metadata:
+  id: ru.cuod.dzz-ground-segment
+  name: ЦУОД ДЗЗ — наземный сегмент
+  version: 1.4.0
+  description: |
+    40 справочников для управления наземным сегментом ДЗЗ:
+    КА, типы КА, наземные станции, антенны, частотные диапазоны,
+    операторы, форматы данных, уровни обработки.
+  domain: dzz
+  scope: PUBLIC  # or INTERNAL / RESTRICTED
+  author: ЦУОД team
+  license: proprietary
+  signature: ed25519:base64...
+
+dependencies:
+  # Reference other bundles (e.g. shared "countries" dict)
+  - id: ru.shared.countries
+    version: ^2.0.0
+  - id: ru.shared.iso-units
+    version: ^1.0.0
+
+dictionaries:
+  - name: spacecraft
+    schemaVersion: 1.0.0
+    scope: PUBLIC
+    schemaJson:
+      $schema: http://json-schema.org/draft-07/schema#
+      type: object
+      properties:
+        code: {type: string, x-unique: true}
+        type: {type: string, x-references: "satellite_type.code"}
+        # ...
+    sampleData:
+      - businessKey: ISS
+        data: {code: ISS, type: OPERATIONAL, ...}
+  # ... 39 more dictionaries
+
+# Migration hints for upgrades
+migrations:
+  - from: 1.3.x
+    to: 1.4.0
+    summary: добавлено поле x-frequencies в antenna
+    breaking: false
+    sql: null  # null = no DB migration, schema add only
+```
+
+### Install flow (dry-run + apply)
+
+```
+1. Admin clicks Install в catalog
+2. Frontend: POST /api/v1/bundles/install
+   { bundleId, version, dryRun: true, options: { seedSampleData: true } }
+3. Backend BundleResolver:
+   - Download manifest.yaml from Nexus
+   - Verify ed25519 signature against trusted authors registry
+   - Parse 40 dictionary defs
+   - For each: check conflicts (dict name уже exists? schema compatible?)
+   - Compute diff: новые dicts, modifications, conflicts
+   - Return: { willCreate: [...], willUpdate: [...], conflicts: [...], warnings: [...] }
+4. Frontend shows preview modal:
+   - 40 dicts to create
+   - 0 conflicts
+   - 312 sample records to seed
+   - [Cancel] [Install for real]
+5. POST /api/v1/bundles/install { ..., dryRun: false }
+6. Backend opens transaction:
+   - For each dict: DictionaryDefinitionService.create() (existing logic)
+   - For sample data: bulk insert via existing DictionaryRecordService
+   - INSERT INTO installed_bundles (id, version, installed_at, installed_by, manifest_sha256)
+   - Commit
+7. Standard outbox → Kafka → downstream consumers получают NewDictionaryCreated events
+   (40 events за raz, batched в outbox)
+8. Frontend: success toast + redirect к catalog (теперь installed_bundles tab показывает entry)
+```
+
+### Conflict resolution strategies
+
+При install bundle X v1.4.0, если dict `spacecraft` уже существует:
+
+| Strategy | Behavior |
+|---|---|
+| **`abort`** (default) | Return 409, admin вручную resolve |
+| **`namespace`** | Create as `dzz_ground_segment__spacecraft` (prefix bundle id) |
+| **`merge`** | Skip dict if schema совместима (existing has all bundle fields); fail if incompatible |
+| **`override`** | Replace existing schema (DANGEROUS — only с explicit confirmation + admin role RESTRICTED) |
+
+v1: только `abort` + `namespace`. `merge` / `override` — v2 после dogfooding.
+
+---
+
+## Components (new)
+
+| Component | Module | LOC est |
+|---|---|---|
+| `BundleResolver` (fetch + verify + parse) | ordinis-app or new ordinis-bundles module | ~300 |
+| `BundleCatalogController` (read endpoints) | ordinis-rest-api | ~150 |
+| `BundleInstallController` (dry-run + apply) | ordinis-rest-api | ~200 |
+| `BundleSignatureVerifier` (ed25519) | ordinis-bundles | ~100 |
+| `InstalledBundle` JPA entity + repo | ordinis-domain | ~80 |
+| Migration 00XX: `installed_bundles` table | ordinis-migrations | ~30 |
+| `BundleCatalogPage.tsx` (UI) | ordinis-admin-ui | ~250 |
+| `BundleInstallModal.tsx` (dry-run preview) | ordinis-admin-ui | ~200 |
+| `InstalledBundlesTab.tsx` | ordinis-admin-ui | ~150 |
+| Bundle CLI publisher (corp Maven plugin / scripts) | new ordinis-bundle-publisher | ~250 |
+
+Total: ~1700 LOC. Plus ~600 LOC tests.
+
+---
+
+## Registry choice
+
+### Option A: Reuse corp Nexus (RECOMMENDED для v1)
+
+- Pros: уже есть в инфре (corp packages), self-hosted, signed artifacts
+- Cons: Nexus ориентирован на Maven/npm, нужна custom REST query layer
+- Path: `nexus.corp/repository/ordinis-bundles/{bundle-id}/{version}/manifest.yaml + sampledata.tar.gz`
+
+### Option B: GitLab Package Registry
+
+- Pros: уже используем GitLab, native CI publish from bundle source repos
+- Cons: less flexible queries, GitLab CE может иметь limits
+
+### Option C: Custom registry service (NEW)
+
+- Pros: tailored to bundles (search, semver, dependencies)
+- Cons: yet another service to ops, +2 weeks effort
+
+**Recommendation:** A (Nexus) для v1. C если customer growth >10 bundles published.
+
+---
+
+## Multi-tenancy
+
+Каждый customer install ordinis имеет свой namespace в registry:
+- `nexus.corp/ordinis-bundles/private/{customer-id}/` — corp private bundles
+- `nexus.corp/ordinis-bundles/public/dzz/` — curated public ДЗЗ catalog
+
+Customer A не видит customer B's private bundles. Public catalog visible всем authenticated installs.
+
+Trust model:
+- Private bundles: signed customer's own key
+- Public ДЗЗ catalog: signed ЦУОД editorial team
+- Admin UI shows badge: «✅ Verified ЦУОД» / «🏢 Private (your org)» / «⚠️ Unverified»
+
+---
+
+## Non-goals (v1)
+
+- ❌ Public internet-facing marketplace UI (browser) — only corp Nexus discoverable through admin UI
+- ❌ Paid bundles / billing
+- ❌ Bundle rating / reviews / comments
+- ❌ Automatic bundle updates (admin manually triggers upgrade)
+- ❌ Bundle composition (compose multiple bundles into super-bundle)
+- ❌ Bundle export from running instance back to registry (one-way: registry → install)
+- ❌ Migration scripts execution (DB schema changes) — v1 только additive (new dicts, new fields), no DB migration
+
+---
+
+## Risks
+
+1. **Malicious bundle uploaded в Nexus** — bundle с `x-id-source` указывающим на JNDI или другой attack vector. Mitigation:
+   - All bundles require ed25519 signature от trusted author
+   - Schema validator strict mode (no eval, no JNDI, allowlist `x-*` annotations)
+   - Sandboxed sampleData parse (no executable code, JSON only)
+
+2. **Bundle version conflict** — bundle X v1.4 deps require shared/countries v2.x, but другой bundle Y deps require shared/countries v1.x. Mitigation:
+   - Resolver detect conflict at dry-run, return 409 conflict с explanation
+   - Recommend admin: deinstall Y or wait until Y updates
+
+3. **Breaking schema change в new version** — bundle v2.0 removes field `mass_kg`, existing records have it. Mitigation:
+   - `migrations` block в manifest documents breaking changes
+   - Upgrade endpoint requires `acknowledgeBreaking: true` flag
+   - Audit log records upgrade event with migration summary
+
+4. **Registry down при install** — admin clicks install, Nexus 503. Mitigation:
+   - User-friendly error «Catalog временно недоступен, повторите через минуту»
+   - Local cache: recently viewed manifests cached 1h
+   - Circuit breaker на Resolver level
+
+5. **Schema-level FK references to non-existing dicts** — bundle dict A references dict B, B installed позже. Mitigation:
+   - Resolver topologically sort install order
+   - Atomic transaction (all-or-nothing)
+   - Cross-bundle FK: explicit dependency declaration в manifest, resolver enforces
+
+6. **Sample data сбивает existing records** — install bundle с sample data, customer уже имеет records с теми же businessKeys. Mitigation:
+   - Default: `seedSampleData: false`
+   - If true: dry-run shows count of records, admin confirms
+   - Skip records existing с тем же businessKey (idempotent)
+
+---
+
+## Test plan
+
+| # | Test | Type |
+|---|---|---|
+| 1 | Happy path: install bundle с 5 dicts → все 5 created, sample data seeded | integration |
+| 2 | Dry-run mode returns preview without DB changes | integration |
+| 3 | Conflict detection: install bundle с existing dict name → 409 | integration |
+| 4 | Namespace strategy: prefix всех dict names с bundle id | integration |
+| 5 | Signature verification: tampered manifest → 422 | integration |
+| 6 | Signature verification: unknown author → 422 with «author not trusted» | integration |
+| 7 | Dependency resolution: install bundle X depending on Y → 409 если Y absent | integration |
+| 8 | Topological install order: bundle с inter-dict FK → outer FK created first | integration |
+| 9 | Atomic transaction: failure mid-install rolls back ALL changes | integration |
+| 10 | Outbox events: install 5 dicts → 5 NewDictionaryCreated events in outbox | integration |
+| 11 | Registry timeout: Nexus 503 → user-friendly 502 + retry | integration |
+| 12 | Circuit breaker: 10 fails → 5min cool-down | integration |
+| 13 | Bundle uninstall: deletes dicts + records + emits delete events | integration |
+| 14 | Uninstall blocked если dicts have records authored locally (non-sample) | integration |
+| 15 | Frontend BundleCatalogPage shows verified badge | RTL |
+| 16 | Frontend BundleInstallModal — preview correctly counts new/updated/conflicts | RTL |
+| 17 | Multi-tenancy: customer A не видит customer B private bundles | integration |
+
+---
+
+## Effort
+
+| Step | Effort (CC) | Notes |
+|---|---|---|
+| 1. `installed_bundles` table migration | 1h | Standard Liquibase |
+| 2. `InstalledBundle` JPA entity + repo | 1h | Standard JPA |
+| 3. `BundleResolver` (fetch + parse) | 6h | HTTP client + YAML + JSON parse |
+| 4. `BundleSignatureVerifier` (ed25519) | 4h | BouncyCastle JCA |
+| 5. `BundleCatalogController` (read endpoints) | 3h | Standard REST |
+| 6. `BundleInstallController` (dry-run + apply) | 8h | Transactional, conflict detection, atomic |
+| 7. Topological install order + dep resolution | 4h | Kahn's algorithm на dict deps |
+| 8. Frontend `BundleCatalogPage.tsx` | 6h | Standard list + search + filter |
+| 9. Frontend `BundleInstallModal.tsx` (dry-run preview) | 6h | Preview UI, diff visualization |
+| 10. Frontend `InstalledBundlesTab.tsx` | 4h | List + uninstall action |
+| 11. Bundle publisher CLI (Maven plugin) | 8h | Sign + upload to Nexus |
+| 12. i18n keys (~25 strings ru/en) | 1h | Standard |
+| 13. Tests (17 cases) | 16h | testcontainers + RTL |
+| 14. Docs (admin guide + publisher guide + ops runbook) | 6h | docs/user-guide/marketplace.md |
+| **Total** | **~74h (~10 рабочих дней)** | within 1.5 sprints |
+
+---
+
+## Open questions
+
+1. **Где будут жить public ДЗЗ catalog editors?** Кто signs verified bundles от имени community? Suggestion: ЦУОД core team initially, expand by invitation later.
+
+2. **Bundle CI/CD pipeline для авторов?** Authors write bundle.yaml + sample data → CI signs + uploads to Nexus? Yes — provide Maven plugin (`ordinis-bundle-publisher`) which integrates с GitLab CI. ~1 day extra effort.
+
+3. **Versioning policy?** SemVer (major.minor.patch). Breaking schema changes = major bump. Recommend documenting в `docs/user-guide/bundle-authoring.md`.
+
+4. **Sample data ownership после install?** Records seeded из bundle — owned by «system» user, audit log helps trace. Customer-edited records → owned by customer.
+
+5. **Cross-bundle FK?** Bundle X dict X-A references bundle Y dict Y-B. Allowed? Suggestion: yes, declare explicitly в `dependencies`, resolver enforces install order.
+
+6. **Bundle update flow?** v1: deinstall + install new version (loses customer-edited records). v2: incremental upgrade с migration. Add to backlog.
+
+7. **Integration with AI schema assist?** AI suggestion → save as bundle → publish? Это **product-market-fit gold**: customer создаёт schema с AI, publish'нет в catalog, monetize'нет / share'нет. v2 idea, defer.
+
+---
+
+## Combined value with `ai-schema-assist.md`
+
+Эти два дизайна имеют **massive synergy**:
+
+| Без AI + без marketplace | + AI | + Marketplace | + Оба |
+|---|---|---|---|
+| Customer пишет 40 schemas вручную ~3 недели | Customer пишет 40 schemas с AI ~3 дня | Customer берёт готовый bundle ~30 сек | Customer описывает домен («гидрометеостанции») → AI генерит bundle → publishes → другие installer'ят |
+
+**Compounding value loop:** AI lowers bar to create bundles → marketplace enables sharing → more bundles → more demand → more AI usage → more bundles.
+
+---
+
+## Recommendation
+
+**Defer until после v2.14.0 prod stable + 2 weeks dogfood, + AI design `/office-hours`'d first** (поскольку AI affects bundle authoring flow).
+
+**Sequencing:**
+1. v2.14.0 prod stable + dogfood ~2 weeks (current)
+2. `ai-schema-assist.md` — `/office-hours` → `/plan-eng-review` → implement (5-7 days)
+3. Use AI assist для production, gather feedback ~1 month
+4. `dictionary-marketplace.md` — `/office-hours` → `/plan-eng-review` → implement (10-14 days)
+5. Public catalog launch — Q3-Q4 2026
+
+**Это потенциально strongest product differentiator** для МДМ-продукта на рынке — combine AI + Marketplace + ДЗЗ domain expertise = **product nobody else has** в RU/CIS МДМ-сегменте.
+
+---
+
+## See also
+
+- Companion: `ai-schema-assist.md` — AI-assisted authoring (sequencing prerequisite)
+- Roadmap mention: `docs-internal/status/2026-05-05-state.md` § "v2 (governance / data quality / federation)" — Federation MDM-кластеров (related but not same)