## Demo
Proves the fix flags & demotes the 3 confirmed BU-rename phantom movers — verified against live prod Redshift, not fixtures.
Backend — phantom detection + escalation demote
Ran uv run python /tmp/demo-b2-artifact.py from klair-api/; it imports the changed service and calls _account_bu_class_at_quarters, _base_table_quarter_a_values, and _reconcile_account_artifacts directly (no HTTP), with two read-only prod queries feeding the decision.
_Live read #1 — the rename exists (aws_spend_budget_account_mapping):_
086775481754 KayakoProd bu@2025-Q4='JigTree' → bu@2026-Q1='Canopy' [RENAME]727712672144 Contently-Prod bu@2025-Q4='Contently' → bu@2026-Q1='Canopy' [RENAME]
654654284640 Jigsaw bu@2025-Q4='JigTree' → bu@2026-Q1='Canopy' [RENAME]
_Live read #2 — quarter_a spend IS material (mapping-free base table; floor $100):_
086775481754 KayakoProd base 2025-Q4 = $62,686.37 base 2026-Q1 = $64,880.51727712672144 Contently-Prod base 2025-Q4 = $28,790.75 base 2026-Q1 = $29,707.51
654654284640 Jigsaw base 2025-Q4 = $21,059.59 base 2026-Q1 = $20,093.49
_The fix — _reconcile_account_artifacts on mover rows as get_cost_movement_by_account emits them (q_a=$0 is the bug; q_b is real prod Q1'26 spend):_
086775481754 KayakoProd VP/CFO → Informational artifact=Truereason: BU renamed JigTree→Canopy; prior-quarter spend recorded under prior BU
q_a_base=$62,686.37 (unchanged: q_a=$0 q_b=$64,881 diff=$64,881 is_new=True)
727712672144 Contently-Prod VP/CFO → Informational artifact=True
reason: BU renamed Contently→Canopy; prior-quarter spend recorded under prior BU
654654284640 Jigsaw VP/CFO → Informational artifact=True
reason: BU renamed JigTree→Canopy; prior-quarter spend recorded under prior BU
All 3 phantoms are badged is_likely_artifact and dropped from VP/CFO to Informational, with the exact prior→current BU named. Raw q_a/q_b/diff/is_new are left untouched — only the tier + 3 artifact fields change.
Most at risk from this change — the new step runs only in the account path, and the 3 new fields sit on the shared _CostMovementBase. Verified that a genuinely-new account and a normal non-is_new mover both pass through untouched, and that reconcile short-circuits with zero DB reads when nothing is flagged:
999999999999 REAL-NEW-ACCT tier=VP/CFO (preserved) artifact=False111111111111 NORMAL-MOVER tier=Team (preserved) artifact=False
Scoped regression suite (other-grain serialization defaults, short-circuit, silent-failure guard): uv run pytest tests/test_cost_movement_artifact.py → 18 passed in 0.31s.
---
## QoQ B2 — Backend: mover validation + escalation demote
Part of the Cost Movement (QoQ) feature (features/aws-spend/cost-movement-qoq), slice B2. Fixes the confirmed BU/class-rename phantom "New account" bug so a data artifact never escalates to a VP.
Linear: [KLAIR-2862 — QoQ B2 — Backend: mover validation + escalation demote](https://linear.app/builder-team/issue/KLAIR-2862)
---
## The bug
The account-grain QoQ mover path (get_cost_movement_by_account) manufactures phantom "New account" movers whenever an account's BU label changes between the two compared quarters. The per-quarter BU/Class mapping join in _build_cost_movement_value_query (bam.quarter = dwm.quarter) strands the account's prior-quarter spend under its old BU, so under the new BU the prior quarter reads $0, the account looks brand-new (is_new=True), and it escalates.
Confirmed in prod (Q4'25→Q1'26): 3 Canopy accounts, ~$446K total annualized phantom Δ, all BU renames:
| Account | Name | Rename | base Qa |
|---|---|---|---|
| 086775481754 | KayakoProd | JigTree→Canopy | $62,686 |
| 727712672144 | Contently-Prod | Contently→Canopy | $28,791 |
| 654654284640 | Jigsaw | JigTree→Canopy | $21,060 |
## The fix (mechanism)
A reconciliation step on the account grain only. For each flagged mover (abs(q_a) < 1 and q_b > 0), reconcile against a single batched, mapping-free base-table read (aws_spend_net_amortized_costs_adjusted joined ONLY to aws_spend_date_week_map, NO aws_spend_budget_account_mapping, aws_account_number IN (…)), which recovers the account's true quarter_a actuals regardless of which BU it mapped to that quarter.
When base q_a >= $100 (the _ARTIFACT_MATERIAL_FLOOR material floor) AND the account's bu@quarter_a != bu@quarter_b:
- set is_likely_artifact=True
- set the exact reason "BU renamed {prior}→{current}; prior-quarter spend recorded under prior BU"
- record q_a_base
- demote: force escalation_tier="Informational" so the phantom drops out of the Team/Director/VP/CFO/Exec tiers and never reaches a VP
The raw q_a/q_b/diff/annualized_diff are preserved for transparency — only the tier is overwritten. A generic reason ("Prior-quarter spend recorded under a different mapping; this account is not new") is used when material-but-no-rename or the prior BU is unresolvable. A structurally-parallel class-rename branch handles class renames too. prior_bu is recovered deterministically from aws_spend_budget_account_mapping at quarter_a — (account, quarter) is strictly unique (verified prod, 14,792 groups, one row each), so it yields exactly one prior BU with no aggregation.
## New endpoint
POST /api/aws-spend/cost-movement/confirm-artifact — one-click "confirm artifact" dismissal of a badged mover. Takes ConfirmArtifactRequest (awsAccountNumber, quarterA, quarterB, optional note), gated with validate_single_bu_access on the account's BU, INSERTs into core_finance.aws_spend_cost_movement_artifact_confirmations, returns ConfirmArtifactResponse. Thin caller of confirm_cost_movement_artifact(...) via asyncio.to_thread; exceptions propagate to the FastAPI error handlers.
## Schema fields
Three optional fields added to the shared _CostMovementBase (wire-compatible defaults, so every drill level inherits them; populated only on the account grain):
- isLikelyArtifact: bool (default false)
- artifactReason: str | null (default null)
- qABase: float | null (default null)
## Scope
Reconciliation + demote only — this slice does NOT replace the per-quarter bam join with a consistent latest-quarter mapping (the deeper "consistent-mapping rewrite" is explicitly deferred per locked ticket scope). Reconciliation runs only on flagged account rows, so it is cheap and leaves all roll-up invariants and the other drill levels untouched. Applies to both the adjusted and non-adjusted source views (identical column shape, identical artifact). Frontend badge rendering is a separate slice (B4).
Investigation (prod Redshift, documented in the spec): no usable rename-tracking/audit table exists — account_mapping_audit.previous_bu is never populated, bu_class_registry_audit.old_bu is NULL on every row and not account-keyed. The per-quarter aws_spend_budget_account_mapping is the sole source of truth for an account's BU per quarter.
## Test coverage
klair-api/tests/test_cost_movement_artifact.py — 18 tests, all passing (mocked _execute_query):
- the 3 confirmed phantom accounts (KayakoProd / Jigsaw JigTree→Canopy, Contently-Prod Contently→Canopy) flagged with the exact prior→current reason + populated q_a_base
- the mapping-free reconciliation query shape (adjusted view joined ONLY to aws_spend_date_week_map, NO aws_spend_budget_account_mapping, IN (…))
- the $100 material-floor boundary
- demotion to Informational while q_a/q_b/diff/annualized_diff preserved
- demotion isolation (a genuine-new account with base Qa ≈ 0 is NOT flagged and keeps its real tier)
- the generic-reason branch (material-but-no-rename)
- the class-rename branch
- model defaults (non-artifact items carry false/null/null)
- the silent-failure guard on the confirm-artifact INSERT
ruff + pyright clean.
## Self-review fix
Self-review found and fixed one CRITICAL silent-success bug: a failed confirm_cost_movement_artifact INSERT was returning success=True. It now raises so the error propagates to the FastAPI error handlers (consistent with the no-swallowing convention) — covered by the silent-failure guard test.
---
## ⚠️ Required before merge / deploy
The confirm-artifact endpoint INSERTs into core_finance.aws_spend_cost_movement_artifact_confirmations, which does not exist yet — the DDL is intentionally left for the user to run (no db:push in this slice). The service method and endpoint are written against this schema; if the table is absent at call time, the INSERT error propagates rather than being swallowed.
Create the table before deploying. Required columns:
| Column | Notes |
|---|---|
| aws_account_number | the artifact account |
| quarter_a | literal 'YYYY-QN' (e.g. '2025-Q4') |
| quarter_b | literal 'YYYY-QN' (e.g. '2026-Q1') |
| note | optional free-text note |
| confirmed_by | the confirming user |
| confirmed_at | timestamp of confirmation |
Suggested DDL:
CREATE TABLE IF NOT EXISTS core_finance.aws_spend_cost_movement_artifact_confirmations (aws_account_number VARCHAR(32) NOT NULL,
quarter_a VARCHAR(8) NOT NULL,
quarter_b VARCHAR(8) NOT NULL,
bu VARCHAR(256),
note VARCHAR(2000),
confirmed_by VARCHAR(256),
confirmed_at TIMESTAMP DEFAULT GETDATE()
);
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)