CustomsHive Hardening Plan

Security fixes, XML golden tests, valuation dedupe, crash recovery, and scoped architecture/domain cleanups — 18 work packages in three tiers, produced from the 2026-07 architecture review.

Status: COMPLETE (2026-07-04). All 18 work packages implemented, each as its own commit (grep the log for WP<n>:). The golden-file test suite (tests/CustomsHive.Tests, 73 tests) now guards the generators; scripts/ai-probe is the reusable Azure OpenAI diagnostic that fell out of the work. Only the manual steps at the bottom remain the owner's.

Context

An architecture review of CustomsHive (single-company Belgian customs declaration tool; ASP.NET Core 10 Razor Pages, EF Core + SQL Server, AI extraction, Descartes SMF/XML output) identified a short list of scope-appropriate fixes. The app runs single-instance with human review before every submission, so scalability/multi-tenancy findings were deferred. What remains is: real security exposure (tracked secrets, spoofable proxy trust, unauthenticated file mount, global 1 GB body limit), a correctness time bomb (transport-cost valuation math duplicated between the review UI and the XML generator — the human review process is structurally blind to drift between them), zero automated tests over the legally binding XML output, and dossiers stuck forever in Queued/Processing when the in-memory job channel is lost on restart.

Repo: CustomsHive (its own git repo). Solution: CustomsHive.slnx (slnx format — new projects must be added there). Known quirk: all projects use CustomsHive.Web.* namespaces regardless of assembly. Keep this convention in every WP except WP16, which fixes it wholesale — do not rename namespaces piecemeal in other WPs. Commit rule (user preference): stage explicit paths only, never git add -A; no Co-Authored-By lines in commit messages.

Execution order matters: WP5 (tests) must land before WP6 (valuation dedupe) so the refactor is protected by characterization tests.

Tier 1 — security & correctness essentials

WP1 — Secrets hygiene (code side)

Problem: src/CustomsHive.Web/appsettings.Development.json is git-tracked and contains live SQL credentials (User Id=sa;Password=…, both DefaultConnection and TarbelConnection). (appsettings.dv.json / appsettings.pr.json are already gitignored — the AI key in dv.json is local-only, leave those files alone.)

Changes: 1. Replace the two connection strings in appsettings.Development.json with placeholder values (e.g. Server=localhost;Database=customstuf_dv;User Id=CHANGE_ME;Password=CHANGE_ME;TrustServerCertificate=true). 2. Program.cs already loads user secrets for Development/dv (lines ~31–35). Document in docs/configuration.md: real dev connection strings go in user secrets (dotnet user-secrets set "ConnectionStrings:DefaultConnection" ... for project src/CustomsHive.Web). 3. Do not run any git history rewrite — see Manual Steps.

Acceptance: git show HEAD:src/CustomsHive.Web/appsettings.Development.json after commit contains no real password; app still boots locally when user-secrets are populated.

WP2 — Forwarded-headers trust (spoofing fix)

Problem: Program.cs (~lines 433–441) clears KnownIPNetworks/KnownProxies before UseForwardedHeaders. Any internet client can send X-Forwarded-For: 10.0.0.1 and (a) satisfy the RFC-1918 allowlist on /health, (b) choose its own rate-limit partition (the pages limiter partitions on RemoteIpAddress).

Changes in Program.cs: 1. Set forwardedHeadersOptions.ForwardLimit = 1 (only the last XFF entry — the one appended by the trusted proxy — is consumed). 2. Instead of clearing everything, trust configurable networks: read Proxy:KnownNetworks (string array of CIDRs, default ["10.0.0.0/8","172.16.0.0/12","192.168.0.0/16","127.0.0.1/32"]) and add each via IPNetwork.Parse to KnownIPNetworks. Add the key with its default to appsettings.json and document in docs/configuration.md (Proxy__KnownNetworks__0 env-var form for Docker). 3. Keep the existing comment block but update it to explain the new model (headers only honored when the direct peer is a known proxy network; ForwardLimit=1 prevents client-supplied XFF chains from being trusted).

Acceptance: with the app behind its proxy, real client IPs still appear in [PERF] logs / rate limiting; a request sent directly to Kestrel with a forged X-Forwarded-For does not change RemoteIpAddress.

WP3 — Authenticated /downloads

Problem: Program.cs (~lines 541–551) mounts {DataFolder}/downloads as static files before routing/auth with ServeUnknownFileTypes = true — anything in that folder is public to the internet.

Changes: 1. Delete the pre-routing UseStaticFiles block for /downloads. 2. Add an authorized minimal API endpoint in the endpoint section (near the locode endpoint): app.MapGet("/downloads/{**path}", ...) that: - resolves Path.GetFullPath(Path.Combine(downloadsFolder, path)) and returns 404 unless the result StartsWith(downloadsFolder) (traversal guard) and the file exists; - serves via Results.File(fullPath, contentType, fileDownloadName: Path.GetFileName(fullPath)) using FileExtensionContentTypeProvider with fallback application/octet-stream; - .RequireAuthorization("RequiresUser"). 3. Keep Directory.CreateDirectory(downloadsFolder) at startup.

Acceptance: unauthenticated request to /downloads/anything → redirect to login / 401; authenticated user can still download a file dropped in data/downloads; /downloads/..%2fappsettings.json style traversal returns 404.

WP4 — Scope the 1 GB request body limit

Problem: Program.cs (~lines 87–96) sets Kestrel MaxRequestBodySize and MultipartBodyLengthLimit to 1 GB globally (needed only for the Tarbel ZIP upload). Every endpoint — including anonymous POST /api/declarations, which feeds XmlSerializer — accepts 1 GB bodies.

Changes: 1. Remove the global Kestrel 1 GB override (revert to Kestrel default ~30 MB). Keep the global FormOptions.ValueCountLimit = 8192 (transit forms need it) but drop the global 1 GB MultipartBodyLengthLimit. 2. On src/CustomsHive.Web/Pages/Admin/Tarbel.cshtml.cs (OnPostUploadAsync page model class): add [RequestSizeLimit(1_073_741_824)] + [RequestFormLimits(MultipartBodyLengthLimit = 1_073_741_824)] — same pattern already used by NewDossier.cshtml.cs:8 and Tools/DescartesReports/Import.cshtml.cs:9. 3. POST /api/declarations (Program.cs minimal API): cap at 5 MB by setting ctx.Features.Get<IHttpMaxRequestBodySizeFeature>()!.MaxRequestBodySize = 5_242_880; as the first statement of the handler (before body read), guarded for null/read-started. 4. Check other upload pages still within new default: NewDossier (150 MB attr — fine), DescartesReports Import (50 MB attr — fine). Grep .cshtml.cs for IFormFile to catch any page relying on the old global limit and give it an explicit attribute (e.g. Admin EBTI/Locode upload handlers if present).

Acceptance: Tarbel ZIP upload of several hundred MB still works; a >5 MB POST to /api/declarations gets 413; a >30 MB POST to a page without an attribute gets 413.

WP5 — Test project + golden XML tests (do BEFORE WP6)

Problem: zero test projects in the solution; the XML generators and valuation math are the product's legal output and are refactored blind.

New project: tests/CustomsHive.Tests/CustomsHive.Tests.csproj - xUnit (xunit, xunit.runner.visualstudio, Microsoft.NET.Test.Sdk), Microsoft.EntityFrameworkCore.Sqlite. - ProjectReference: src/CustomsHive.Web (transitively brings Core + all modules; matches how types actually resolve given the shared namespaces). - Add to CustomsHive.slnx under a /tests/ folder. - Note Directory.Build.props disables source-control queries — no interference expected.

5a. TransportCostCalculator unit tests

Target: src/CustomsHive.Web/Services/TransportCostCalculator.cs (pure static; TransportCostGroup is in src/CustomsHive.Web/Pages/Dossiers/TransportCosts.cshtml.cs). Cases: road EF / DAP / C incoterm groups; CaEqualsFa (GB provenance 100/0); air allocation by net weight with Annex 23-01 fraction; air client-specific Transport BE (rate/kg vs. minimum); zero-total guards (totalValueEur=0, totalNet=0); rounding at 2 decimals; single-group allocation sums equal totals.

5b. Characterization test of IE415B valuation (pre-refactor lock)

Before touching the generator, add a test that seeds a dossier fixture (see 5d) with known lines, runs IE415BXmlGenerator.GenerateAsync, extracts the AK/CA/FA/statistical values from the generated XML, and asserts exact expected numbers computed by hand for: road+provenance EF case, road DAP, air case, and CaEqualsFa case. These tests must pass unchanged after WP6.

5c. Golden-file generator tests

For IE415BXmlGenerator (H1B and H2B variants), CC015CXmlGenerator: - Golden files checked in under tests/CustomsHive.Tests/Golden/*.xml. First run writes them (env-var switch e.g. UPDATE_GOLDEN=1); afterwards byte/normalized comparison. - Normalization required: the SMF envelope contains a per-run message number (DescartesSmfWrapperService.NextMessageNumberAsync — DB-backed counter) and timestamps. Normalize by replacing those element values with NORMALIZED tokens before comparison (XDocument walk over known element names), or test the generator body separately from the envelope where the API allows. - XSD validation: also run IDescartesSmfWrapperService.ValidateSmfXsd over each golden output with Descartes:Smf:ValidateXsd=true and XsdFolder pointing at the repo schemas/smf folder; assert IsValid.

5d. Test fixture infrastructure

• SqliteFixture: open SqliteConnection("DataSource=:memory:"), keep open, DbContextOptionsBuilder<CustomsTufDbContext>().UseSqlite(conn), ctx.Database.EnsureCreated(). Warning: EnsureCreated will also apply the HasData prompt seeds — fine. Watch for SQL Server-specific model bits failing under SQLite (decimal precision is fine; if IsDescending index config trips SQLite, configure conditionally or ignore — resolve at implementation time).
• Seed: one Client (with AuthC503, Fr1, Et14000, Ref4007, air rates), one Dossier + CustomsDeclaration per scenario (H1B road, H1B air, H2B, transit), CorrectedData JSON fixtures checked in under tests/.../Fixtures/*.json (shape per CorrectedDataReader.ReadImport / transit reader in src/CustomsHive.Core/Domain/Models/CorrectedDataReader.cs), AppSettings rows for all declarations:* and descartes:* keys the generators read, ProvenanceTypes (GB + TR at minimum).
• Fakes (hand-rolled, no mocking lib needed): IHsNomenclatureService (fixed dictionary + real-ish NormalizeHs10), ICountryCodeService (identity resolve for ISO2, map a name→code pair), ITarbelAdditionalRefService, IAirFreightZoneService (fixed outside-fraction), IAppSettingsService (backed by the seeded context — or use the real implementation from src/CustomsHive.Core/Services/AppSettingsService.cs against the SQLite context). Use the real DescartesSmfWrapperService.

5e. CI

Add dotnet test step to .github/workflows/build.yml after build (and to docker.yml if it builds independently).

Acceptance: dotnet test green locally and in CI; golden files reviewed by the user (they are the domain expert on correct XML) before merge.

WP6 — Valuation dedupe: generator calls TransportCostCalculator

Problem: the AK/CA/FA/Statistical division algorithm exists twice: TransportCostCalculator.Calculate (used by the TransportCosts review pages) and re-implemented inline in IE415BXmlGenerator.GenerateAsync — the ~100-line block after the // ----- Transport Cost Division (H1B) comment. Both human review gates (in-app + Descartes) see the same generated XML, so drift between the two implementations is invisible to the review process.

Blocker to solve: the calculator lives in CustomsHive.Web and its TransportCostGroup type lives in a page model file; CustomsHive.Module.Descartes cannot reference Web (would be circular).

Changes: 1. Move TransportCostGroup (from src/CustomsHive.Web/Pages/Dossiers/TransportCosts.cshtml.cs) and TransportCostCalculator.cs into src/CustomsHive.Core/Services/ (Core is referenced by both Web and Descartes). Keep their existing namespaces (CustomsHive.Web.Services for the calculator; move the group class into that same namespace — update the one using CustomsHive.Web.Pages.Dossiers in the calculator and any page usings that break; Pages.Dossiers namespace types referenced from cshtml may need a @using fix in TransportCosts.cshtml / StandaloneTransportCosts.cshtml). 2. In IE415BXmlGenerator.GenerateAsync, replace the inline valuationByIndex computation: - build List<TransportCostGroup> from the already-grouped lines (TotalValue = EUR-converted line value from the existing lineValues list; TotalNet from NetMass); - road path: pass the loaded provenance, incotermGroup, decl.TransportCostTotal, decl.DeclarationCost; - air path: pass transportType: "AIR", airOutsideFraction: airZones.GetOutsideFraction(decl.CountryOfDispatch) ?? 0m, transportTotal: awbEur (keep the existing EUR conversion of the AWB before the call), chargeableWeight, useClientSpecific, client rate/minimum; - map results back into valuationByIndex[i] = (g.AK, g.CA, g.FA, g.Statistical). 3. Preserve current behavior exactly — the WP5b characterization tests are the gate. Any intentional behavior difference discovered during the merge (there is a subtle one: the inline road path checks totalValueAll > 0 before looping while air allocates by net weight — replicate guards precisely) must be flagged to the user, not silently chosen.

Acceptance: WP5b characterization tests and WP5c golden files pass unchanged; TransportCosts + StandaloneTransportCosts pages compile and behave identically.

WP7 — Stuck-dossier recovery on startup

Problem: ProcessingChannel is an in-memory Channel<ProcessingJob>; any restart loses queued/in-flight jobs, leaving dossiers stranded in Queued/Processing forever with no operator signal.

Changes: 1. In the existing startup scope in Program.cs (right after ctx.Database.Migrate() / seeding — before the host starts serving), run a sweep: all dossiers with Status IN (Queued, Processing) → Status = Failed, LastErrorMessage = "Processing was interrupted by an application restart. Use Retry to reprocess.", UpdatedAt = utcnow, UpdatedBy = "system". At boot the channel is empty by definition, so every such dossier is an orphan — no time threshold needed. 2. Use ExecuteUpdateAsync-style set-based update (pattern exists in IngestionApiKeyService.RevokeAsync). 3. Log count at Information level. 4. The Retry flow already exists (Details.cshtml.cs OnPostRetryAsync re-enqueues with IsRetry: true) — verify it accepts dossiers in Failed (it does; that's its purpose).

Acceptance: create a dossier, kill the app while it's Processing, restart → dossier shows Failed with the interrupt message and a working Retry button.

Tier 2 — reliability & AI hygiene

WP8 — Ingestion idempotency

Problem: CanonicalServiceBusReceiverService abandons on any error → redelivery. A redelivered message either creates duplicate dossiers (no UCR in payload → fresh generated UCR each attempt) or hits the unique UCR index → exception → abandon-loop until DLQ.

Changes in src/CustomsHive.Core/Services/CanonicalIngestionService.cs: 1. At the start of IngestAsync, if declaration.Ucr is non-empty and a dossier with that UCR exists → log + return the existing dossier (treat as success). 2. In CanonicalServiceBusReceiverService.OnMessageAsync (src/CustomsHive.Module.Ingestion/Services/ServiceBus/CanonicalServiceBusReceiverService.cs): rely on UCR dedupe plus catching DbUpdateException unique-violation and calling CompleteMessageAsync instead of abandon. (Do not reuse LastQueueMessageId for inbound dedupe — that field tracks outbound Descartes sends. UCR-first dedupe + unique-violation → Complete is enough; avoid schema changes.) 3. Same UCR-exists check benefits POST /api/declarations (return 200 with existing dossier info instead of 500).

Acceptance: replaying the same canonical message (ServiceBusTester project in servicebus-testing/ can send) produces one dossier and completes the message both times.

WP9 — Prompt seeding fix: stop HasData clobbering production edits

Problem: Prompts are seeded via HasData in CustomsTufDbContext.SeedDefaultPrompts (src/CustomsHive.Core/Data/CustomsTufDbContext.cs, ~line 125) and editable at /Admin/Prompts. Any change to seed text generates a migration that UPDATEs the row, silently overwriting admin edits in production.

Changes: 1. Remove SeedDefaultPrompts / the HasData call from the model. 2. Critical migration caution: removing HasData makes EF generate a migration that DELETEs the seeded prompt rows. Generate the migration, then hand-edit Up()/Down() to be empty (keep the snapshot changes). Verify the generated snapshot no longer contains prompt seed data. 3. Add runtime insert-if-missing seeding for the 9 prompts in Program.cs SeedDefaultData (exact pattern already there for AppSettings — key-absent check, never update existing). Move the prompt contents to a static class (e.g. CustomsHive.Core/Data/DefaultPrompts.cs) so the DbContext stays clean. 4. Future default-prompt changes then require a deliberate mechanism (admin re-applies via UI, or bump a key) — out of scope; just log a warning at startup when a DB prompt differs from the shipped default (cheap drift visibility).

Acceptance: dotnet ef migrations list clean; migration applies with zero data change on an existing DB; fresh SQLite test DB (WP5 fixture) still gets all prompts; editing a prompt in /Admin/Prompts survives an app upgrade that changes a default.

WP10 — Details page over-fetch

Problem: Details.OnGetAsync (src/CustomsHive.Web/Pages/Dossiers/Details.cshtml.cs, ~line 62) does .Include(d => d.Extractions) — pulling RawResponse, AdiRawResponse (megabytes of ADI JSON), and PdfPigRawText for every historical extraction on every page view, only to compute a cost sum and render metadata.

Changes: 1. Replace the Include(d => d.Extractions) with a separate projected query: db.Extractions.Where(e => e.DossierId == Id).Select(e => new ExtractionSummary(e.Id, e.ExtractorType, e.ModelUsed, e.CreatedAt, e.ExtractionTimeMs, e.EstimatedTotalCostUsd, ...)) — include exactly the fields Details.cshtml renders (audit the view first; it may show a raw-JSON panel for the latest extraction — if so, fetch that one blob by id only). 2. Any handler that needs a full extraction record loads it individually. 3. Check ReviewTransit.cshtml.cs and other pages for the same pattern and apply where it's equally mechanical; skip if the page genuinely renders the blobs.

Acceptance: Details page renders identically (compare before/after on a dossier with multiple extractions); [PERF] Details(...).OnGet load time drops for blob-heavy dossiers.

Tier 3 — architecture & domain

WP11 — Dossier status state machine

Problem: DossierStatus transitions are raw property assignments scattered across NewDossierModel, ProcessingWorker, Details.cshtml.cs handlers, and review pages. Nothing prevents an illegal transition (e.g. Submitted → Draft, or dispatching a Failed dossier); the workflow — the core of the product — has no guardian.

Changes: 1. Add to Dossier (src/CustomsHive.Core/Domain/Models/Dossier.cs) a TransitionTo(DossierStatus next, string? by) method backed by a static allowed-transitions map — derive the exact map from current call sites, don't invent it: grep \.Status\s*= across src/ and enumerate every existing transition; the map legalizes what the app actually does today, then throws InvalidOperationException on anything else. Method also sets UpdatedAt/UpdatedBy. 2. Replace every direct dossier.Status = … assignment with TransitionTo (mechanical; ~10 call sites). 3. Keep the setter public for EF but mark direct use discouraged via doc-comment (full encapsulation with private setter is more churn than value here). 4. Unit tests in the WP5 project: allowed matrix + a few forbidden transitions.

Acceptance: all existing flows (upload, process, fail, retry, reprocess, review, declare, submit, reopen) still work; tests document the legal workflow in one place.

WP12 — Money as decimal in corrected-data models

Problem: line values in the corrected-data DTOs are double? (ReviewImportLine.TotalPrice, ExtraCost, UnitPrice, plus totals on the corrected-data roots), summed in floating point and then cast to decimal in IE415BXmlGenerator before rounding/currency conversion. Binary floating point on customs values feeding a legal declaration is a correctness defect (cent-level drift under summation).

Changes: 1. In src/CustomsHive.Core/Domain/Models/ (CorrectedImportData.cs, CorrectedExportData.cs, CorrectedTransitData.cs, and the ReviewLine/ReviewImportLine types): change monetary fields (TotalPrice, UnitPrice, ExtraCost, TotalValue, transport/declaration cost fields) from double? to decimal?. Leave masses/weights as double — they're measurements, not money. 2. Existing stored JSON deserializes fine, but AI-derived JSON may contain values like 1.0E+2 or long fractions — add a tolerant JsonConverter<decimal?> (parse via double fallback when direct decimal parse fails) registered in JsonSharedOptions (src/CustomsHive.Core/Infrastructure/JsonSharedOptions.cs), same pattern as the existing KilosOrObjectJsonConverter/StringOrArrayJsonConverter. 3. Fix the compile fallout: generator casts ((decimal)(l.TotalPrice ?? 0d) become direct), review page bindings (the InvariantDecimalModelBinder already handles decimal form fields), XLSX builders. 4. Run WP5b/5c tests — golden values must not change (they were already rounded to 2 decimals; if a golden file shifts by a cent, stop and report rather than regenerate).

Acceptance: solution compiles; all WP5 tests pass with unchanged golden files; an existing production-shaped CorrectedData JSON fixture (with plain numbers and with scientific notation) round-trips.

WP13 — Program.cs decomposition

Problem: Program.cs is 761 lines: seed data tuples, Tarbel migration baselining, raw-SQL buffer-pool warmup, temp-file cleanup, two timing middlewares, security headers, and the full /api/declarations endpoint incl. authentication — all inline. Un-navigable and untestable.

Changes (pure extraction, zero behavior change): 1. src/CustomsHive.Web/Infrastructure/StartupTasks.cs (or similar): move SeedDefaultData, BaselineTarbelSchema, the temp-upload cleanup block, and the WP7 stuck-dossier sweep into named static methods called from the same startup scope. 2. src/CustomsHive.Web/Infrastructure/TarbelWarmupService.cs: move the two raw-SQL warmup queries into an IHostedService (fire-and-forget on StartAsync, keeping the existing try/catch-log semantics), registered alongside CacheWarmupService. 3. src/CustomsHive.Web/Infrastructure/RequestTimingMiddleware.cs + SecurityHeadersMiddleware.cs: convert the inline app.Use lambdas to middleware classes (keep log categories and thresholds identical). 4. src/CustomsHive.Web/Endpoints/IngestionEndpoints.cs + LocodeEndpoints.cs: MapIngestionEndpoints(this IEndpointRouteBuilder) extension moving the two minimal APIs out of Program.cs (auth logic goes with it; consider extracting the API-key-or-Bearer check into a small helper). 5. Target: Program.cs under ~250 lines of pipeline wiring with section comments preserved.

Acceptance: app boots identically (compare startup log sequence); [PERF] logs, security headers, seeding, and both endpoints behave unchanged; WP5 tests green.

WP14 — Dead code & repo hygiene

Small, zero-risk deletions — one commit: 1. Delete the ghost src/CustomsHive.Module.Qargo/ folder (only bin/obj, not in the solution). 2. Remove the vestigial named HttpClients "DocumentIntelligence" and "OpenAI" in Program.cs (10s timeouts, unused by the SDK-based clients) — verify first with a grep for CreateClient("DocumentIntelligence") / CreateClient("OpenAI"); OpenAiClient receives IHttpClientFactory via AiClientFactory, check which named client it actually resolves before deleting that one. 3. Delete commented-out XML blocks in IE415BXmlGenerator.cs (the // w.WriteStartElement("OperatorIdentity") exporter block, the disabled ArrivalTransportMeans guard comments — keep the active code). 4. Move repo-root scratch files (__dump.ps1, __query.ps1, __q.ps1, __corrected_1042.json, __restore-dossiers-from-qa.ps1, update_extract_invoice_prompt.sql) into scripts/archive/ or delete after user confirms they're dead; add __* to .gitignore. 5. Fix GenerateUcr's no-op [..Math.Min(100, 100)] in CanonicalIngestionService.cs (either a real length clamp against the 100-char UCR column or drop the slice).

Acceptance: solution builds; grep confirms no references to deleted items; user confirms scratch scripts are disposable before deletion.

WP15 — NewDossier transaction + optimistic concurrency

Problem: NewDossier.OnPostAsync performs three separate SaveChangesAsync calls (client, dossier, declaration) — a failure between them leaves a dossier without its CustomsDeclaration. Separately, no entity has a concurrency token, so two users editing the same review page last-write-wins silently.

Changes: 1. Wrap dossier + declaration creation (and inline client creation) in a single transaction, or restructure to a single SaveChangesAsync (add the CustomsDeclaration via the Dossier.Declaration navigation before the first save — EF orders the inserts; this is the simpler fix). 2. Add [Timestamp] byte[] RowVersion to Dossier (+ migration). On DbUpdateConcurrencyException in the review-page save handlers and ProcessingWorker, catch and surface a friendly "this dossier was modified by someone else — reload" ModelState error (pages) / retry-once-with-reload (worker). 3. Keep scope tight: Dossier only; don't add tokens to every entity.

Acceptance: killing the app mid-NewDossier can no longer produce a declaration-less dossier; two concurrent edits to the same review page produce a visible conflict message instead of silent overwrite; migration applies cleanly.

WP16 — Namespace honesty

Problem: every project declares CustomsHive.Web.* namespaces regardless of assembly (Core is CustomsHive.Web.Domain/Data/Services, Processing is CustomsHive.Web.Services.Extraction, etc.). A developer or agent cannot tell from a using which assembly they're coupling to.

Changes: 1. Precondition: clean working tree. This touches nearly every file — it must be its own commit with nothing else in flight. Verify git status is clean before starting; abort if not. 2. Rename namespaces to match assemblies: CustomsHive.Web.Domain.* → CustomsHive.Core.Domain.*, CustomsHive.Web.Data → CustomsHive.Core.Data, Core services → CustomsHive.Core.Services.*, Core infrastructure → CustomsHive.Core.Infrastructure; each module's types → CustomsHive.Module.{Name}.* (e.g. CustomsHive.Web.Services.Extraction → CustomsHive.Module.Processing.Extraction, the Dispatch sender's CustomsHive.Web.Services.ServiceBus → CustomsHive.Module.Dispatch.ServiceBus); Web keeps CustomsHive.Web.*. 3. Mechanics: per-project find/replace on namespace declarations, then fix usings solution-wide (including GlobalUsings.cs in each project and @using directives in .cshtml/_ViewImports.cshtml). Do NOT touch Migrations/ — verify no phantom model diff afterwards (build + Migrate() against a scratch SQLite DB via the WP5 fixture, or a pending-model-changes check). EF table/column names are unaffected (attribute/fluent-based, not namespace-based). 4. Sequencing: run after WP5 exists (tests catch fallout) and ideally before WP13/WP17 so those new files are born with honest namespaces.

Acceptance: solution builds; dotnet test green; app boots and ctx.Database.Migrate() reports no pending model changes; grep for namespace CustomsHive.Web outside src/CustomsHive.Web/ returns nothing.

WP17 — Generator consolidation: shared SMF writers + Web generators into modules

Problem: party/operator XML blocks, AppSettings loading, contact fallbacks, and diacritics handling are copy-pasted across IE415BXmlGenerator, CC015CXmlGenerator, and CC515CXmlGenerator; completing AES export (roadmap) means a fourth copy. Separately, DossierXlsxGenerator and the Descartes report builders live in CustomsHive.Web/Services/Generators instead of a module.

Gated by WP5 golden tests — do not start without them.

Changes: 1. Extract into src/CustomsHive.Module.Descartes/Services/Generators/SmfWriters.cs (static helpers over XmlWriter, matching the existing WriteOptional/WritePostalCode extension style): - WriteOperatorParty(w, elementName, eori?, crin?, name?, address?, contact?) covering the Declarant/Importer/Exporter/Representative/Consignee block permutations (nullable sections control what's emitted — derive the exact permutation matrix from the three generators' current output, golden tests enforce byte-identical results); - WriteSmfAuthorisation, WriteProducedDocument, WriteAdditionalReference, WriteFiscalReference (currently private/duplicated per generator); - shared RemoveDiacritics (currently duplicated) → one internal helper. 2. Refactor the three generators to call the shared writers. Golden files must not change — any diff is a bug in the extraction, not a new golden. 3. Move DossierXlsxGenerator + DescartesImportReportXlsxBuilder + TransportCostXlsxBuilder from src/CustomsHive.Web/Services/Generators/ into the Descartes module. Update DI registration in DescartesModuleExtensions.cs, remove from Program.cs. 4. Leave the deeper phase-split (typed document model → serializer) deferred — this WP is deduplication only.

Acceptance: WP5c golden files byte-identical (post-normalization); solution builds; Descartes report pages and XLSX download still work.

WP18 — CorrectedData schema version + write-time validation

Problem: the CorrectedData JSON blob is the de-facto domain model — schema-less, versionless, validated nowhere. Any shape change silently breaks old dossiers; a bad save from a review page is discovered at XML-generation time (or worse, in Descartes).

Affordable slice (full typed model stays deferred): 1. Add a "schema_version": 1 field to the three corrected-data root models. Readers (CorrectedDataReader) treat absent as version 1 — all existing dossiers remain readable with zero migration. 2. Add a CorrectedDataValidator (in Core next to the models) invoked by the review-page save handlers and CanonicalIngestionService before persisting: per-regime structural checks (lines array present; per-line: HS code empty-or-digits, non-negative masses/values, currency code shape, transit: commodity_code 8 digits, gross ≥ net). Violations → ModelState errors on pages / 422 detail on the API — warn-and-block on save, never mutate data silently. 3. Unit tests: validator accepts the WP5 fixtures, rejects targeted corruptions.

Acceptance: saving a review page with an invalid line shows a field-level error instead of persisting; existing dossiers (no version field) load and re-save normally; API returns 422 with per-field details for structurally invalid canonical payloads.

Deliberately deferred (out of scope while single-tenant)

Recorded so nobody mistakes absence for oversight: - Full typed Declaration domain model replacing JSON persistence (WP18 captures the version+validation slice; the rest is a multi-week data-spine rewrite). - Generator phase-split into typed document → serializer (WP17 captures the deduplication slice). - Tenant/company model, per-tenant settings, DB-backed job queue, horizontal scaling, OpenTelemetry, Key Vault, API versioning/ProblemDetails.

Manual steps (owner, not the coding agent)

1. Rotate the sa password on the database server and create a dedicated least-privilege login for the app (the ALTER DATABASE call at startup currently needs elevated rights — either grant just that or accept db_owner for now; full permission reduction is future work).
2. Rotate the oi.xact.cloud API key (in local appsettings.dv.json — not in git, but treat as exposed).
3. Decide on git-history purge for appsettings.Development.json (password is in history). Given the repo is private and the password is being rotated, rotation alone is acceptable; purge is optional.
4. Review and bless the golden XML files produced in WP5c — the owner is the domain authority on correct declaration output.

Suggested execution order & sizing

Each WP is an independent commit (explicit file staging). WP1–WP4+WP7 are safe to ship immediately and independently.

Verification (end-to-end)

• dotnet build the solution and dotnet test after every WP.
• Run the app locally (dv environment) and walk the core flow: NewDossier upload → extraction → Review → TransportCosts → Declare → Download XML; confirm XML unchanged vs. a pre-change reference dossier (WP6 especially).
• Security checks: curl /downloads/x unauthenticated (expect 401/redirect), curl with forged X-Forwarded-For (expect no effect on rate-limit partition/health allowlist), 6 MB POST to /api/declarations (expect 413), Tarbel ZIP upload still accepted.
• Kill-and-restart during processing → stuck dossier surfaces as Failed with Retry.
• Replay a canonical Service Bus message via servicebus-testing/ServiceBusTester → single dossier.