ivan-igorevich/mdm-ordinis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
692 Commits 1 Branch 0 Tags
597e05db4cca46cfda36fcf6982bb72e43777a89
Commit Graph

7 Commits

Author SHA1 Message Date
Александр Зимин 0269542ebe fix(webhook): User-Agent 'ordinis-alerts' (matches Webhook Bot subscription) 2026-05-15 15:24:12 +00:00
Zimin A.N. 75967edc9a feat(webhook): Phase 5 — dispatcher filter + delivery lifecycle tests 
WebhookDispatcherMatchTest (11 tests):
- empty filters match anything
- scope filter inclusion/exclusion
- dictionary filter inclusion/exclusion + null dictionary handling
- event_type filter inclusion/exclusion
- AND-semantics между всеми тремя фильтрами
- empty list treated as null (match all)

WebhookDeliveryLifecycleTest (7 tests):
- initial state (pending, attempt 0)
- markDelivered → delivered status + statusCode
- markFailed → retrying + attempt++ + scheduled next
- exponential backoff растёт между attempts
- 1000 attempts → DLQ
- resetDlq → retrying (admin recovery path)
- delivered после failures clears lastError

Total ordinis-outbox tests: 6 → 47.
Project total: 132 → 149 unit tests.

Phase 5 e2e tests с embedded HTTP receiver — deferred (нужен JDK
HttpServer or wiremock setup, нетривиально). Текущее покрытие
закрывает critical filter logic + state machine + HMAC + SSRF.
Реальный e2e на staging будет через manual run + UI verification.
 2026-05-06 15:26:51 +03:00
Zimin A.N. e9a7278709 feat(webhook): Phase 3 — REST API CRUD + HMAC/SSRF tests 
REST API endpoints под /api/v1/admin/webhooks/:
- GET    /subscriptions               — list (INTERNAL+ scope)
- POST   /subscriptions                — create (RESTRICTED scope)
- GET    /subscriptions/{id}           — get one (INTERNAL+)
- PUT    /subscriptions/{id}           — update (RESTRICTED)
- DELETE /subscriptions/{id}           — delete (RESTRICTED)
- POST   /subscriptions/{id}/rotate-secret  — regenerate HMAC (RESTRICTED)
- GET    /subscriptions/{id}/deliveries     — delivery history per sub
- GET    /deliveries/dlq                    — DLQ listing

RBAC через ScopeContext: RESTRICTED для mutating, INTERNAL+ для read.
PUBLIC scope получает 403.

DTOs:
- CreateWebhookSubscriptionRequest (Bean Validation: URL regex, length)
- WebhookSubscriptionResponse (с factory `from()` маскированный secret
  vs `fromWithSecret()` plaintext только для create response)
- WebhookDeliveryResponse

Service:
- HMAC secret 32 random bytes → 64 hex chars через SecureRandom
- Plaintext secret виден только в create/rotate response (single-time)
- list/get показывают `sk_****<last4>` masked

Tests (15 SsrfGuard + 8 HmacSigner = 23 new):
- HmacSigner: known vector verification, prefix, hex length, distinct
  inputs produce distinct sigs, empty secret rejected
- SsrfGuard: rejects 127/8, 169.254/16, 10/8, 192.168/16, 0.0.0.0,
  non-http schemes, missing host, unresolvable DNS. Allowlist bypass.
  isPrivate() unit tests для loopback/link-local/site-local/public IP.

Total: 109 → 132 unit tests.

Phase 4 далее: admin-ui /webhooks page (list + create + delivery history).
 2026-05-06 15:12:28 +03:00
Zimin A.N. 9ac35e9ff2 feat(webhook): Phase 2 — dispatcher service + HMAC + SSRF guard 
WebhookDispatcher (@Scheduled, conditional ordinis.webhook.enabled=true):
- matchTick (every 2s): scan unpublished outbox_events × active subs,
  INSERT pending deliveries idempotently (ON CONFLICT DO NOTHING)
- sendTick (every 1s): scan due deliveries (status pending/retrying AND
  next_attempt_at <= now), POST URL with HMAC-SHA256 + SSRF check,
  mark delivered (2xx) / retrying (backoff) / dlq (>= 1000 attempts)

HmacSigner:
- HMAC-SHA256 hex signing для X-Ordinis-Signature header
- Format: `sha256=<hex>` — receiver verifies через shared secret
- JDK Mac, no external deps

SsrfGuard:
- Reject URLs резолвящиеся в private CIDR (10/8, 172.16/12,
  192.168/16, 127/8, 169.254/16, link-local, multicast)
- Через InetAddress.getAllByName() + isXxxAddress() checks
- Allowlist через ordinis.webhook.allowed-hosts CSV для testing

HttpClient: JDK 21 native, connectTimeout 3s, send timeout configurable
ordinis.webhook.send-timeout-ms (default 5000ms)

Метрики Micrometer:
- nsi_webhook_delivered_total (success counter)
- nsi_webhook_failed_total (retry counter)
- nsi_webhook_dlq_total (DLQ counter)
- nsi_webhook_ssrf_rejected_total (SSRF guard counter)
- nsi_webhook_delivery_duration_seconds (timer)

Headers отправляемые receiver'у:
- Content-Type: application/json
- User-Agent: Ordinis-Webhook/1.0
- X-Ordinis-Signature: sha256=<hex>
- X-Ordinis-Event-Id: <id>
- X-Ordinis-Event-Type: <event_type>
- X-Ordinis-Scope: <PUBLIC|INTERNAL|RESTRICTED>

Ещё впереди: REST API CRUD subscriptions (admin-only), unit tests
для HmacSigner/SsrfGuard, admin-ui /webhooks page.
 2026-05-06 15:05:37 +03:00
Zimin A.N. e182b30c58 feat(outbox): attempt cap → DLQ + admin endpoint + 6 unit-тестов 
После N (default 1000) неудачных попыток публикации событие маркируется
dlq_at и исключается из polling. Без cap poller бесконечно ретраил мёртвые
сообщения (orphan topic, ACL deny), забивая логи и lag-метрику.

- 0011-outbox-dlq.xml: dlq_at column + перестроенный idx_outbox_unpublished
  (исключает DLQ) + idx_outbox_dlq для admin-листинга.
- OutboxPoller: ordinis.outbox.attempt-max (default 1000), markDlq() при
  достижении cap, новый counter ordinis_outbox_dlq_total.
- OutboxLagGauge: ordinis_outbox_dlq_size gauge — алерт на > 0.
- OutboxEventRepository: countPending() (без DLQ), findByDlqAtIsNotNull(Pageable).
- OutboxAdminController: GET /admin/outbox/{stats,dlq} для UI.
- 6 unit-тестов через mock-maker-subclass (JDK 25 ломает Mockito inline).
 2026-05-04 15:43:56 +03:00
Zimin A.N. be77c30687 feat(validation,rest-api,outbox): write-side end-to-end working 
ordinis-validation:
- ValidationError, ValidationResult, ValidationException
- RecordValidator: networknt JSON Schema (Draft-7) + custom rule для
  x-localized полей. Walk schema, для каждого x-localized:true property
  проверяет: значение это object с locale keys (BCP 47 pattern xx-XX),
  все ключи в supported_locales, default_locale обязательно присутствует,
  значения non-empty strings.

ordinis-outbox:
- KafkaTopicResolver: ordinis.<bundle>.events.<scope>
- OutboxRecorder: запись события в outbox в той же транзакции (MANDATORY
  propagation), с tracer для trace_id/span_id из MDC если OTel доступен
- OutboxPoller: @Scheduled, batch_size + poll_interval из config, метрики
  ordinis_outbox_published_total / publish_errors_total / kafka_publish_duration_seconds
- OutboxLagGauge: ordinis_outbox_pending_events (gauge для алертов)
- ConditionalOnProperty ordinis.outbox.enabled — отключаем в read-api/projection-writer

ordinis-rest-api:
- DTOs: CreateDictionaryRequest, DictionaryResponse, CreateRecordRequest,
  RecordResponse (с WKT serialization geometry)
- DictionaryDefinitionService: CRUD definitions, outbox events DefinitionCreated/Updated
- DictionaryRecordService: bitemporal write-side, SERIALIZABLE isolation,
  bound check FAR_FUTURE valid_to. Update = close current + create new
  (никогда in-place). Outbox event RecordCreated/Updated/Deleted.
- DictionaryDefinitionController: GET list/by-name, POST create, PUT update
- DictionaryRecordController: GET active/history, POST create, PUT update,
  DELETE (close)
- OrdinisException + ApiErrorResponse + GlobalExceptionHandler
  (422 validation, 404 not found, 409 conflict, 500 internal)

ordinis-app:
- @EnableJpaRepositories + @EntityScan для multi-package сканирования
- application.yml profile-based: dev (PG localhost через port-forward,
  ddl-auto=update, OTel disabled), k8s (Cloud Config + Vault Kubernetes auth)
- spring.kafka producer config с SASL SCRAM-SHA-512

ordinis-domain:
- JpaAuditingConfig: добавлен DateTimeProvider возвращающий OffsetDateTime
  (Spring Data Auditing по дефолту его не поддерживает, без него падает
  IllegalArgumentException 'Cannot convert LocalDateTime to OffsetDateTime')
- @ConditionalOnBean(DataSource) убран — race с lifecycle ломал auditing

E2E test (port-forward к cluster PG+Kafka):
- POST dictionary 'ground_station' → 201, JSONB schema сохранена,
  supported_locales = {ru-RU, en-US}, default_locale = ru-RU
- POST record MOSCOW-1 multi-locale name + geometry POINT(37.618 55.751) → 201
- POST record с en-US без ru-RU → 422 с двумя ошибками:
  * networknt 'required property ru-RU not found'
  * custom 'x-localized.missing_default Default locale ru-RU is required'
- outbox_events: 2 row (DefinitionCreated + RecordCreated), topic
  ordinis.cuod.events.public, ключи правильные
- revinfo + dictionary_records_aud: 1 row (Envers tx-time history активен)
 2026-05-03 13:04:36 +03:00
Zimin A.N. a28fd0b352 feat: Maven multimodule scaffolding for Ordinis MDM service 
- 10 модулей (domain, validation, events-api, outbox, rest-api, app, read-api, projection-writer, cuod-bundle, migrations)
- Parent pom + BOM с Spring Boot 3.4.2, Spring Cloud 2024.0.0, Vault Config 4.2.0
- ordinis-app: Spring Boot main с Actuator/Prometheus/Logstash JSON encoder/OpenTelemetry/Cloud Config/Vault skeleton
- ordinis-migrations: Liquibase changelogs (dictionary_definitions/records, audit_log, outbox_events, idempotency_keys, EXCLUSION CONSTRAINT через btree_gist)
- Multi-stage Dockerfiles для app и migrations
 2026-05-03 00:41:53 +03:00