The Tracking Migration Evidence Review Framework

TRACKING MIGRATION REVIEW — original Growthcraft synthesis
Decision, migration owner, reviewer and deadline:
Old collector/version and new collector/version:
Destination/property/test isolation and permission to run:
1. CONTRACT — analyst + privacy owner
Business event definition, unique identity and reference source:
Eligibility/consent rule, exclusions and unavailable evidence:
Event-time interval [start,end), timezone and arrival cutoff:
Source export versions; late data policy:
2. RECONCILE — engineer + analyst
Reference unique count:
Old/new raw rows, duplicate extras, unique eligible IDs, outside-reference IDs:
Both / old-only / new-only / neither:
Old/new coverage, discordance and shared/union overlap:
Payload type/value/currency mismatches, missing required fields:
Checks by browser/device/consent/checkout route; known unsupported paths:
3. REVIEW — growth + engineering
Does identical volume conceal different IDs?
Can shared missing events or wrong payloads explain apparent agreement?
Evidence for consent enforcement, destination processing and downstream joins:
Business-specific tolerances agreed BEFORE seeing results, rationale and owners:
Unresolved issues and explicit hold conditions:
4. CUTOVER PACKET — named approver, not an automated decision
HOLD_FOR_EVIDENCE / READY_FOR_HUMAN_REVIEW:
Isolation/deduplication plan; no uncontrolled dual-firing into production:
Rollback owner, trigger, last known configuration and recovery check:
Next action, evidence required and review timestamp:
No automatic account edits, budget changes or consent bypass.

When to use this framework

Use it when an ecommerce, subscription or startup team replaces a tracking collector, pixel, tag or server-side route. The question is whether the evidence supports a controlled migration, not whether the new dashboard has a similar headline total. It is suitable for bounded verification windows with an independent eligible-event reference. Do not use it to certify privacy compliance or compare two unrelated attribution reports.

The review has four distinct owners: an analyst defines the event contract, an engineer explains collection and destination behavior, a privacy owner verifies permitted processing, and a business owner accepts operational risk. One person may fill multiple roles, but unresolved evidence does not disappear in a small team.

Freeze the reference before comparing pipelines

Define an event identity, event-time interval, eligibility rule and extraction cutoff. A paid order is not necessarily the same event as a browser checkout callback. Restrict the reference to the behavior both collectors are expected and permitted to observe. Preserve excluded and unknown-eligibility counts separately; never reinterpret denied consent as a tracking bug to work around.

Normalize identifiers inside your controlled environment and count duplicate rows before reducing observations to presence. Reconcile events outside the reference separately. The online worksheet needs only anonymous aggregate counts, source labels and redacted findings, not order IDs or email addresses.

Four review gates

1. Contract: agree what should be observed and when the comparison is complete. A missing identity map or unresolved permission boundary holds the review.
2. Reconcile: classify reference events into both, old-only, new-only and neither. Review raw duplicate extras and outside-reference records alongside this table.
3. Inspect: validate values, currency, required fields, event semantics, destination processing and downstream joins. Segment by relevant device and consent paths without collecting new personal data.
4. Prepare: document a bounded cutover and rollback plan for human approval. Do not send uncontrolled duplicate production events merely to create a comparison.

Agree tolerances before looking at results and tie them to business consequences, not a universal 95% benchmark. A consent violation or an unexplained high-value mismatch can be a hold condition even when the average looks strong.

A volume match that fails identity review

Synthetic: 100 eligible purchase events; 80 appear in both collectors, 10 only in the old one, 10 only in the new one and none in neither. Both totals are 90 and both coverage rates are 90%, but 20 events disagree. Shared/union overlap is 80%. The net coverage change is zero percentage points.

The analyst assigns the old-only group to a checkout-route investigation and the new-only group to identity/eligibility verification. If destination payload validation is still missing, the packet remains HOLD_FOR_EVIDENCE. Nothing in the arithmetic proves an implementation improved or harmed real conversion.

Failure modes and stop conditions

Similar totals can hide different events. High agreement can reflect joint omissions. Deduplicated presence can conceal repeated purchases in a destination. An incomplete reference makes neither unknowable. Comparing snapshots with different late-arrival maturity creates artificial losses. A correct payload can still fail to appear in reports, while a malformed request may not produce an HTTP error.

Keep a rollback configuration and monitoring owner. This framework does not execute a migration, modify permissions or determine lawful processing. Confirm vendor-specific identity and deduplication behavior independently.

Shopify script-tag migration timeline motivates this review; Google Measurement Protocol validation guidance separates request validation from reporting evidence. This is an original operating method, not a platform-certified migration or legal consent assessment.