Commit Graph

21 Commits

Author SHA1 Message Date
Александр Зимин 0cabd81abf fix(migrations): add version column for JPA optimistic locking
DictionaryDefinition and DictionaryRecord entities have @Version (BIGINT)
fields for optimistic locking, but the initial migrations did not create
the column. App fails with 'column dd1_0.version does not exist' on first
query.

Add 0007-add-version-columns.xml with idempotent addColumn changesets
(preConditions guard against re-add).
2026-05-03 20:38:43 +03:00
Александр Зимин 1c4e8e5be2 fix(app): switch hibernate ddl-auto to none in k8s profile
Hibernate validate mode threw 'wrong column type [supported_locales]:
found [_text (Types#ARRAY)] but expecting [text[] (Types#OTHER)]' against
the Liquibase-managed schema. This is a well-known hypersistence-utils +
PostgreSQL TEXT[] friction with Hibernate validate.

Schema is owned by Liquibase migrations; validate adds no safety here. Set
to 'none' (matches read-api and projection-writer).
2026-05-03 20:33:21 +03:00
Александр Зимин d49dd6be06 fix(k8s): add datasource/kafka/redis config to k8s profile
K8s profile relied on shared/dev defaults for spring.datasource.url, but those
live inside the dev profile block — invisible when k8s profile is active.
Result: 'APPLICATION FAILED TO START — Failed to configure a DataSource'.

Move full datasource (and kafka/redis for projection-writer) wiring into the
k8s profile section. ENV vars come from Vault Agent injector
(ORDINIS_PG_USER/PASSWORD, ORDINIS_KAFKA_USER/PASSWORD, ORDINIS_REDIS_PASSWORD)
and chart-supplied ENV (ORDINIS_PG_HOST, ORDINIS_KAFKA_BOOTSTRAP, etc).
2026-05-03 20:24:53 +03:00
Александр Зимин 413bd3449d chore: update default config-server fallback to ordinis-staging
application.yml fallback URL was http://config-server.ordinis-dev:8888 — namespace
removed. Default now points to staging; SPRING_CLOUD_CONFIG_URI env still overrides
via Helm chart for any deployment.
2026-05-03 20:11:51 +03:00
Zimin A.N. e537888554 ci: trigger-deploy-staging вместо -dev (dev перешёл в docker-compose) 2026-05-03 19:05:26 +03:00
Zimin A.N. 5b686abaa8 ci: --insecure-registry для DinD push в registry.k8s.265.nstart.local
Internal registry имеет cert валидный для ingress.local, не для своего DNS name.
Push HTTP-only через insecure-registry flag.
2026-05-03 18:44:55 +03:00
Zimin A.N. 8eb0ec086e ci: pull base images через repo.nstart.cloud (DH official → /library/, vendor → /<vendor>/) 2026-05-03 18:40:29 +03:00
Zimin A.N. fe403f2dbe ci: default retry до 2 раз на mirror failures 2026-05-03 18:19:58 +03:00
Zimin A.N. 00c905401f ci: pull_policy if-not-present (mirror flaky на always) 2026-05-03 18:18:02 +03:00
Zimin A.N. 9a79b53eb0 ci: align с altum pattern — runner mirror'ит Docker Hub
- Убраны префиксы repo.nstart.cloud/ — runner config (DOCKER_AUTH_CONFIG)
  обрабатывает pull из Docker Hub mirror автоматически
- DinD сервис: alias=docker, --tls=false, DOCKER_HOST=tcp://docker:2375
  (proven pattern из altum/.gitlab/ci/docker.yml)
- IMAGE_TAG = ref-slug + sha (как у altum)
- BUILDKIT_INLINE_CACHE=1 для cache-from optimization
- Dockerfiles: FROM maven/eclipse-temurin/liquibase без префиксов
2026-05-03 18:16:21 +03:00
Zimin A.N. 7c06cf7f71 ci: pull images через repo.nstart.cloud (corp Docker Hub mirror)
Runners в кластере не имеют network access к Docker Hub.
Заменили все base images:
- maven:3.9.9-eclipse-temurin-21 → repo.nstart.cloud/maven:...
- eclipse-temurin:21-jre → repo.nstart.cloud/eclipse-temurin:21-jre
- liquibase/liquibase:4.27.0 → repo.nstart.cloud/liquibase/...
- docker:27 / docker:27-dind → repo.nstart.cloud/docker:...
2026-05-03 18:02:46 +03:00
Zimin A.N. aa26724661 fix(migrations): preConditions ДО comment в changelog (XSD), relative include paths
- 0001-extensions.xml: переставил <preConditions> перед <comment> (XSD требует
  preConditions первым в changeSet)
- master.xml: include paths сделаны relative к самому master.xml (changes/...
  вместо db/changelog/changes/...) — CWD inside Docker image это /liquibase,
  Liquibase searchPath = /liquibase/changelog где master.xml лежит, includes
  должны идти от него

CI: insecure-registry: registry.k8s.265.nstart.local (HTTP без auth для
internal cluster), REGISTRY_USER/PASSWORD optional (skip docker login если
не заданы)
2026-05-03 17:54:30 +03:00
Zimin A.N. bc65728d1b feat(ci): GitLab CI pipeline для build + push images
stages:
- test: mvn verify (unit + integration tests когда появятся)
- build: mvn package для всех 4 jar'ов с кэшированием Maven repo
- publish: docker build + push 4 images в registry.k8s.265.nstart.local
  с тегами {short-sha, branch-slug}. Параллельно через 4 jobs.
- trigger-deploy: запускает downstream pipeline в terravault/ordinis-infra
  с UPSTREAM_IMAGE_TAG, branch=main → dev (auto), tag=v*-rc → staging (auto),
  tag=v* (final) → prod (manual approve)

Required GitLab CI variables:
  REGISTRY_USER, REGISTRY_PASSWORD — для docker login в private registry
2026-05-03 16:12:30 +03:00
Zimin A.N. 4723af5bca feat(docker): Dockerfile для read-api и projection-writer
Multi-stage build (Maven 3.9.9 + Eclipse Temurin 21 → eclipse-temurin:21-jre):
- ordinis-read-api/Dockerfile (port 8081)
- ordinis-projection-writer/Dockerfile (port 8082)

JVM args: -XX:+UseZGC -XX:MaxRAMPercentage=75 (low-pause GC + container-aware
memory). MaxRAMPercentage важен потому что Helm задаёт container memory
limit через resources.limits.memory; JVM должна respect'ить это.

Helm chart (terravault/ordinis-infra/charts/ordinis/) теперь имеет полный
набор images для deploy:
  registry.k8s.265.nstart.local/terravault/ordinis-app:<tag>
  registry.k8s.265.nstart.local/terravault/ordinis-read-api:<tag>
  registry.k8s.265.nstart.local/terravault/ordinis-projection-writer:<tag>
  registry.k8s.265.nstart.local/terravault/ordinis-migrations:<tag>
2026-05-03 16:10:21 +03:00
Zimin A.N. 6a6067e70b feat(projection-writer): Kafka consumer + Redis materialization per-locale
ordinis-projection-writer полностью реализован, last module:
- OrdinisProjectionWriterApplication: Spring Boot main + JpaRepositories
- RedisKeys: convention 'ord:<bundle>:<dictionary>:<bk>:<locale>' для flat
  плюс ':_raw' для multi-locale, плюс 'ord:idx:<bundle>:<dictionary>' set index
- ProjectionWriter: для каждой supported locale справочника flatten data и
  записать в Redis. Также пишет _raw multi-locale ключ. На delete стирает
  все варианты + удаляет из index. Метрики ordinis_projection_records_*
- RecordEventListener: @KafkaListener на 3 scope-топика, group
  ordinis-projection-cuod. Парсит EventEnvelope, разбирает payload.type
  (RecordCreated/Updated/Deleted), вызывает upsert или delete.
- application.yml profile-based: dev (port-forward), k8s (Vault role
  ordinis-projection-writer)
- logback-spring.xml как у других сервисов

ordinis-events-api: добавлен явный 'type' accessor в каждом record
(@JsonProperty type). Без него Jackson не сериализовал discriminator
из @JsonTypeInfo (при generic T payload Jackson не знает declared type).

ordinis-infra: Redis StatefulSet (redis:7.4-alpine, AOF, maxmemory 256mb
LRU, secret password) в namespace ordinis-dev на :6379.

E2E (3 сервиса параллельно):
  POST /spacecraft/records {AQUA-1, name:{ru-RU:'Аква',en-US:'Aqua'}, ...}
  → outbox row id=7 published_at=true
  → ordinis_outbox_published_total +1
  → projection log: 'Projection upserted: spacecraft:AQUA-1 (2 locales)'
  → Redis ord:cuod:spacecraft:AQUA-1:ru-RU = {name:'Аква',...}
  → Redis ord:cuod:spacecraft:AQUA-1:en-US = {name:'Aqua',...}
  → Redis ord:cuod:spacecraft:AQUA-1:_raw  = {name:{ru-RU,en-US},...}
  → Redis ord:idx:cuod:spacecraft set member 'AQUA-1'

10 of 10 модулей реализованы. Last module done.
2026-05-03 15:59:40 +03:00
Zimin A.N. ae69a7642b feat(cuod-bundle): JSON Schema драфты для spacecraft/satellite_type/ground_station + auto-import
3 schema драфта в classpath:/bundles/cuod/:

- spacecraft.schema.json: КА с COSPAR designator, NORAD ID, multi-locale
  name/operator/mission, orbit params (type/semimajor/eccentricity/inclination/
  raan/aop/epoch), spectrum_bands (12 bands), launch/decay dates, status enum
- satellite_type.schema.json: классификация КА — code, multi-locale name/desc,
  mission_category enum (10 categories), mass_class, typical_orbit_classes,
  typical_resolution_m, typical_revisit_hours
- ground_station.schema.json: code, multi-locale name/location/operator,
  country (ISO 3166), lat/lon/elevation, antennas array (id, diameter,
  bands, polarizations, tx/rx_capable, G/T, EIRP), status

manifest.json: bundle 'cuod' v1.0.0, 3 dictionaries с supported_locales
[ru-RU, en-US] default ru-RU, scope PUBLIC

ordinis-cuod-bundle модуль:
- BundleManifest record + DictionaryEntry record (Jackson deserializer)
- CuodBundleProperties @ConfigurationProperties (auto-import flag,
  update-existing-schema flag, resource-root path)
- CuodBundleImporter @Service (proxified, @Transactional работает) — читает
  manifest + schemas, для каждого: findByName → если нет, создать definition
  + outbox event DefinitionCreated; если есть, проверить schema_version, при
  отличии и update_existing_schema=true — обновить + DefinitionUpdated outbox
- CuodBundleStartupRunner @Configuration с ApplicationRunner — триггерит
  importBundle при старте Spring Boot. Отделено от Importer чтобы Importer
  оставался proxified (@Transactional на importBundle vs lambda не работает)

ordinis-validation FIX:
- RecordValidator теперь учитывает required array родительской схемы.
  Отсутствие x-localized optional поля (например mission, operator) больше
  не считается ошибкой. Required x-localized поля по-прежнему обязательны.

E2E проверено:
- ORDINIS_BUNDLE_CUOD_AUTO_IMPORT=true → 3 справочника создались с правильными
  scope, locales, schema_version
- POST spacecraft с реальными Kanopus-V данными (orbit + spectrum_bands) → 201
- POST satellite_type с code='lower-case-bad' → 422 pattern error
- Idempotent: повторный старт пропускает existing без ошибок
2026-05-03 14:29:43 +03:00
Zimin A.N. 51fcc95818 feat(read-api): locale negotiation + scope filtering + multi-locale flatten
ordinis-read-api модуль полностью реализован:

- OrdinisReadApiApplication main + EnableJpaRepositories/EntityScan
- LocaleNegotiator: RFC 7231 / Locale.LanguageRange.parse + Locale.lookup
  по supported_locales справочника, fallback на default_locale,
  поддержка Accept-Language: * (raw mode)
- LocaleAwareJsonFlattener: walk schema_json.properties, для x-localized
  полей в data заменяет {locale:text} на string выбранной локали
  с fallback на default_locale; recursive walk для nested objects
- RecordReadService: read с scope filtering + locale negotiation +
  flatten, метрика ordinis_read_api_query_duration_seconds, scope
  access enforcement (ScopeAccessDeniedException)
- DictionaryReadController: GET /api/v1/{name}/records/{businessKey}
  и list, with @RequestHeader(Accept-Language), Content-Language
  response header, Vary: Accept-Language; query param as_scope для
  v1 (заменим на JWT scope claim позже)
- ReadApiExceptionHandler: 404 NoSuchElement, 403 ScopeAccessDenied,
  400 IllegalArgument, 500 fallback
- application.yml profile-based: dev (port-forward, ddl-auto=none —
  read-api не валидирует schema, это work writer/migrations), k8s
  (Vault role ordinis-read-api, hikari read-only=true)
- logback-spring.xml: MDC включает 'locale' для Loki query

E2E (port 8081 параллельно с writer на 8080):
- GET MOSCOW-1 без Accept-Language → name=Москва-1, Content-Language: ru-RU
- GET MOSCOW-1 с Accept-Language: en-US → name=Moscow-1, Content-Language: en-US
- GET MOSCOW-1 с Accept-Language: fr,en-US;q=0.8,ru;q=0.5 → выбрал en-US (fr нет в supported)
- GET MOSCOW-1 с Accept-Language: * → raw {ru-RU,en-US}, _meta.locale=null
- GET list → массив с flatten для каждой записи
- GET NONEXISTENT → 404 not_found
- GET с as_scope=INTERNAL для PUBLIC записи → 403 scope_access_denied
2026-05-03 13:10:07 +03:00
Zimin A.N. be77c30687 feat(validation,rest-api,outbox): write-side end-to-end working
ordinis-validation:
- ValidationError, ValidationResult, ValidationException
- RecordValidator: networknt JSON Schema (Draft-7) + custom rule для
  x-localized полей. Walk schema, для каждого x-localized:true property
  проверяет: значение это object с locale keys (BCP 47 pattern xx-XX),
  все ключи в supported_locales, default_locale обязательно присутствует,
  значения non-empty strings.

ordinis-outbox:
- KafkaTopicResolver: ordinis.<bundle>.events.<scope>
- OutboxRecorder: запись события в outbox в той же транзакции (MANDATORY
  propagation), с tracer для trace_id/span_id из MDC если OTel доступен
- OutboxPoller: @Scheduled, batch_size + poll_interval из config, метрики
  ordinis_outbox_published_total / publish_errors_total / kafka_publish_duration_seconds
- OutboxLagGauge: ordinis_outbox_pending_events (gauge для алертов)
- ConditionalOnProperty ordinis.outbox.enabled — отключаем в read-api/projection-writer

ordinis-rest-api:
- DTOs: CreateDictionaryRequest, DictionaryResponse, CreateRecordRequest,
  RecordResponse (с WKT serialization geometry)
- DictionaryDefinitionService: CRUD definitions, outbox events DefinitionCreated/Updated
- DictionaryRecordService: bitemporal write-side, SERIALIZABLE isolation,
  bound check FAR_FUTURE valid_to. Update = close current + create new
  (никогда in-place). Outbox event RecordCreated/Updated/Deleted.
- DictionaryDefinitionController: GET list/by-name, POST create, PUT update
- DictionaryRecordController: GET active/history, POST create, PUT update,
  DELETE (close)
- OrdinisException + ApiErrorResponse + GlobalExceptionHandler
  (422 validation, 404 not found, 409 conflict, 500 internal)

ordinis-app:
- @EnableJpaRepositories + @EntityScan для multi-package сканирования
- application.yml profile-based: dev (PG localhost через port-forward,
  ddl-auto=update, OTel disabled), k8s (Cloud Config + Vault Kubernetes auth)
- spring.kafka producer config с SASL SCRAM-SHA-512

ordinis-domain:
- JpaAuditingConfig: добавлен DateTimeProvider возвращающий OffsetDateTime
  (Spring Data Auditing по дефолту его не поддерживает, без него падает
  IllegalArgumentException 'Cannot convert LocalDateTime to OffsetDateTime')
- @ConditionalOnBean(DataSource) убран — race с lifecycle ломал auditing

E2E test (port-forward к cluster PG+Kafka):
- POST dictionary 'ground_station' → 201, JSONB schema сохранена,
  supported_locales = {ru-RU, en-US}, default_locale = ru-RU
- POST record MOSCOW-1 multi-locale name + geometry POINT(37.618 55.751) → 201
- POST record с en-US без ru-RU → 422 с двумя ошибками:
  * networknt 'required property ru-RU not found'
  * custom 'x-localized.missing_default Default locale ru-RU is required'
- outbox_events: 2 row (DefinitionCreated + RecordCreated), topic
  ordinis.cuod.events.public, ключи правильные
- revinfo + dictionary_records_aud: 1 row (Envers tx-time history активен)
2026-05-03 13:04:36 +03:00
Zimin A.N. b778832f49 feat(domain,events): bitemporal entities + i18n-aware definitions + sealed event API
ordinis-domain:
- DataScope enum (PUBLIC/INTERNAL/RESTRICTED) с Kafka topic suffix + visibility logic
- DictionaryDefinition entity: scope, schema_json (JsonNode JSONB), schema_version,
  bundle, supported_locales (text[]), default_locale + Spring Data Auditing
- DictionaryRecord entity: bitemporal (Envers tx-time + valid_from/valid_to),
  data JSONB, geometry (JTS via hibernate-spatial), data_scope. EXCLUSION
  CONSTRAINT enforced в БД (Liquibase 0003).
- AuditLog entity для compliance (отдельно от Loki)
- OutboxEvent entity: aggregate_*, kafka_topic/kafka_key, attempt_count,
  markPublished/markFailed, partial index idx_outbox_unpublished
- IdempotencyKey entity для exactly-once семантики write API
- Repositories: findActiveAt, findActiveByDictionaryAndScopes, findUnpublished
- JpaAuditingConfig: ConditionalOnBean(DataSource) — dev profile без БД
  не активирует auditing, metamodel не строится

ordinis-events-api:
- Mirror DataScope enum (events не depend on domain)
- EventEnvelope<T> record: eventId, occurredAt, schemaVersion (semver),
  bundle, dictionaryName, scope, traceId, spanId, payload
- Sealed DictionaryRecordEvent + records Created/Updated/Deleted
- Sealed DictionaryDefinitionEvent + records Created/Updated
- Jackson polymorphism через @JsonTypeInfo discriminator
- Multi-locale payload: события возят ВСЕ локали (consumer сам сплющивает)

i18n design:
- В JSONB поле кодируется как { 'ru-RU': '...', 'en-US': '...' }
- JSON Schema справочника помечает локализуемые поля extension x-localized:true
- supported_locales + default_locale в dictionary_definitions
- Read API будет negotiate через Accept-Language (RFC 7231) и сплющивать

Validation:
- mvn install all 11 modules: BUILD SUCCESS
- ordinis-app boot dev profile: /actuator/health UP, / отвечает
2026-05-03 12:10:36 +03:00
Zimin A.N. d1c8d5755e fix(scaffolding): make ordinis-app boot in dev profile
- Remove spring-cloud-starter-bootstrap (deprecated в SC 2024.x; используем
  spring.config.import вместо bootstrap context)
- Profile-based config: dev отключает Cloud Config + Vault + OTel + JPA
  auto-config для standalone smoke testing; k8s profile подключает реальную
  инфру (vault + config server)
- Dockerfile: COPY всех модулей parent pom их видит при reactor
- Уберём spring-boot-maven-plugin из read-api/projection-writer пока нет main

Validated:
- mvn install all 11 modules: BUILD SUCCESS
- Spring Boot стартует за 2.7s в dev profile
- /actuator/health UP, /actuator/prometheus отдаёт http_server_requests + jvm
- HelloController отвечает на GET /
2026-05-03 01:03:03 +03:00
Zimin A.N. a28fd0b352 feat: Maven multimodule scaffolding for Ordinis MDM service
- 10 модулей (domain, validation, events-api, outbox, rest-api, app, read-api, projection-writer, cuod-bundle, migrations)
- Parent pom + BOM с Spring Boot 3.4.2, Spring Cloud 2024.0.0, Vault Config 4.2.0
- ordinis-app: Spring Boot main с Actuator/Prometheus/Logstash JSON encoder/OpenTelemetry/Cloud Config/Vault skeleton
- ordinis-migrations: Liquibase changelogs (dictionary_definitions/records, audit_log, outbox_events, idempotency_keys, EXCLUSION CONSTRAINT через btree_gist)
- Multi-stage Dockerfiles для app и migrations
2026-05-03 00:41:53 +03:00