A purchase can land three days after an email decision. The decision may have been keyed only by email. The purchase may arrive with phone and postal address, and no shared session id. If your learning table requires the same identifier on both rows, that purchase never rewards the policy. If you treat every open window as a loss, young choices look worse than they are.

BanditBuzz’s answer is eventual consistency on purpose. Keep young choices pending. Resolve people across identifiers when you can. Apply one explicit attribution rule before each policy update. Fast writes matter less than a correct learning table.

This post is implementation advice for engineers joining decisions to purchases, renewals, or cancellations. It mirrors what the nightly attribution path actually does.

The sequence that must work

Picture one person across four calendar days.

  1. Monday, 10:00. Your product calls /decide with { "email": "sam@x.com" } and gets template_b. BanditBuzz stores the decision with normalized email and a reward window of seven days starting at decision time.
  2. Thursday, 14:00. Checkout records a purchase with { "phone": "+1 802 555 0123", "address": { ... } }. No email is present. BanditBuzz stores the event with normalized phone and address fields.
  3. Friday night. Identity resolution stamps both rows with the same resolved person id. The same nightly run attributes the purchase, matures the decision, and recounts the posterior. The next policy call can use Thursday’s purchase.

No correlation id crossed the wire. The join is identity plus time, not a client-supplied decision id on the purchase.

A decision made with email and a later purchase sent with phone join after identity resolution.

The join can arrive after the event. The event time still has to fall inside the original reward window.

Keep young choices pending

A pending choice is not a loss. The data science brief states the training rule plainly: unrewarded decisions enter training only after their window closes. Until then, the Beta posteriors and the contextual fit ignore them as incomplete trials.

That design matches delayed-feedback bandit theory. Howson, Pike-Burke, and Filippi’s Delay-Adaptive Learning in Generalized Linear Contextual Bandits study learners that must act while rewards are still outstanding, and they support keeping scheduled actions pending until outcomes become available. BanditBuzz does not claim their finite-time regret bound for the production loop. The landing page is explicit: long delays slow learning by the declared window plus batch lag, and we do not claim a finite-time regret bound for this delayed production loop. The practical takeaway still holds. Punishing incomplete trials invents false negatives.

Set within_days to the business result you care about. A day-7 purchase goal needs a seven-day window. A day-30 renewal needs thirty. The tradeoff is speed: the learner must wait through the full window, then attribution and policy updates run nightly.

Decision time versus release time

The clock that starts the reward window is not always the decide timestamp.

Request-time /decide. The window starts when the choice is recorded (created_at). An eligible event must satisfy occurred_at between that start and start plus within_days.

Scheduled push with a future act_at. Planning is not release. The sweep can write a decision days before the destination should act. The reward window stays unset until a direct destination accepts the action. Delivery sets reward_window_started_at at that acceptance time. Attribution then requires occurred_at inside [reward_window_started_at, reward_window_started_at + within_days]. Failed, canceled, and still-pending deliveries do not enter timing training as negative outcomes. Canceled decisions are excluded from attribution entirely.

nothing and holdout on push. There is nothing to release. Cooldown and the reward clock start at decision time, because the treatment is the absence of an outbound action for that cycle.

If you attribute a scheduled send from plan time instead of release time, early purchases can reward an action the person has not received yet. That is a labeling bug, not a modeling subtlety.

How identity joins decisions to events

Decide and record do not need to share an identifier. Each row carries whatever the caller sent. Matching uses three paths, in this product sense:

  1. Exact normalized email. Lowercase, trimmed, must contain @.
  2. Exact normalized phone. Digits only; a 10-digit US number gets a leading 1.
  3. Resolved person id. When Faraday’s identity graph later assigns both rows the same universal person id, attribution can join on that id even when email and phone never overlapped.

Name and address fields feed identity resolution. They are not used as direct equality keys in the attribution SQL. Until both sides share email, phone, or a resolved person id, the purchase stays unmatched.

The nightly job order matters. Passport resolution stamps local ledger rows from each dataset’s latest completed passport, then attribution runs. Late identity merges are therefore first-class: a matured-but-unrewarded decision stays eligible on the next pass as long as the event itself occurred inside the original window. The product does not yet report identity-match lag or the unmatched share. Plan for that gap in your own monitoring.

Unmatched rows are normal

Expect unmatched volume. Common causes:

  • Decision and event never share a join key and never receive the same resolved person id.
  • The event name does not match the decision point’s reward event.
  • The event falls outside the window.
  • The decision was canceled before release.
  • Another decision already claimed the event under last-decision-wins.

Do not treat unmatched purchases as proof the policy failed. Treat them as incomplete joins. Warehouse history with richer identifier sets usually tightens resolution; that is one reason connecting a database helps even the free tier.

There is no match-confidence weight. A join either qualifies under the rule above or it does not. Soft scores do not dilute a reward.

Latest-eligible-decision attribution

When several decisions for the same person could claim one event, BanditBuzz gives the reward to the latest eligible decision. Eligibility means:

  • same account
  • reward event name matches
  • event time inside that decision’s window
  • decision not canceled
  • decision not already attributed
  • person match by normalized email, normalized phone, or resolved person id
  • the event has not already rewarded any other decision

“Latest” is ordered by COALESCE(reward_window_started_at, created_at) descending. For pull decisions, that is decision time. For released scheduled actions, that is release time. Among events that could reward one decision, the earliest occurred_at wins. One event rewards at most one decision, across nightly runs as well as within a single run: once an event id is written to attributed_event_id, later passes will not reuse it.

Tests encode the product rule directly. Two decisions for the same email inside a seven-day window, with one later purchase, credit only the later option. An earlier option stays unrewarded while its window remains open. A decision keyed only by email and an event keyed only by phone attribute when both carry the same resolved person id.

Why this is a product rule, not a causal result

Last-decision-wins is clear and implementable. It is not multi-touch measurement, and it does not identify effects when actions overlap or people affect one another. The data science brief and landing page both state that limit. If two emails land inside the same window and a purchase follows, crediting the later email does not prove the later email caused the purchase. It proves only that the ledger applied a deterministic assignment rule before learning.

The same honesty applies to organic conversion after nothing. Attribution will still join a matching event to that pending no-action decision. That is how the policy learns when waiting beats sending. It is not a claim that waiting caused the purchase. It is a labeled trial under the declared reward definition.

If overlap is common in your traffic, shrink cooldown, shorten within_days, or stop stacking decision points on the same people until you can measure interference separately. BanditBuzz does not yet stop the same person from entering two decision points or receiving two actions that both affect the scoreboard. The live holdout lift still estimates the whole policy against control. Option-level rates remain descriptive summaries under a changing policy.

Maturation and the nightly loop

Attribution is one step in a fixed sequence:

  1. Stamp resolved person ids from the latest passports.
  2. Attribute unattributed decisions to eligible events.
  3. Mature decisions that earned a reward or whose window has closed empty.
  4. Recount option posteriors from non-holdout matured decisions.
  5. Fit contextual learning and refresh projected client tables when applicable.

Every step is idempotent. Rerunning after a crash is safe. Holdout decisions mature for the lift denominator but never enter option posteriors.

A choice becomes a zero only after the full window closes without an attributed event. A later identity match can still correct a matured zero when the event occurred on time: matured-but-unrewarded rows remain joinable until they receive an attributed_event_id. Once attributed, recounting folds the reward into the next posterior rebuild.

Policy updates are nightly or on demand. That batch design is intentional. Competitors that require same-millisecond reward feedback force session stitching. BanditBuzz needs the reward before the next policy update, not in the same request.

Returns, chargebacks, and reversals

BanditBuzz does not reverse a reward for a later return or chargeback. Once attributed_event_id is set, a later refund event does not clear it. If the business result can reverse, send a settled event as the reward rather than an early purchase event. For example, reward on order_settled or payment_captured after the refund window, not on checkout_started. Changing the reward event, window, or value field creates a new decision-point version with a fresh policy. Existing decisions stay under the rule they were made with.

Multi-touch credit is also out of scope. The product is not a new journey attribution system and does not rewrite your GA, Adobe, or MMM reports. Those tools still answer journey and budget questions. BanditBuzz answers a narrower one: did this randomized decision policy change the declared result under this attribution rule?

Buyer-visible identity match rates, lag reports, and reward reversal remain named gaps on the landing page. Until they ship, export or query unmatched decisions yourself if that share is an operating risk.

Concrete implementation checklist

Use this when wiring decide and record:

  1. Send every identifier you have on both paths. Email on decide and phone on purchase is fine. Richer sets improve later resolution.
  2. Reuse idempotency keys on retries so you do not double-log decisions or events.
  3. Align event names with the decision point’s reward event. Attribution is an exact name match.
  4. Set within_days to the real outcome lag, not to your reporting cadence.
  5. For scheduled actions, treat release as treatment time. Do not start learning clocks at plan time.
  6. Prefer settled reward events when refunds and chargebacks are material.
  7. Expect unmatched rows and measure them yourself until the product exposes match lag and unmatched share.
  8. Do not interpret last-decision-wins as causal credit when multiple actions overlap.
  9. Wait for maturity before reading option rates or lift. Pending is not failure.
  10. Keep decision ids for debugging, but do not require them on purchase events. Identity is the join.

What this design is for

Identity-resolved, delayed attribution is the loop that makes two API verbs enough. Decide with the identifiers you have now. Record the business result with the identifiers you have later. Nightly resolution and an explicit latest-eligible-decision rule build the learning table. Young choices stay pending. Late merges can still correct zeros when the event was on time. Returns are handled by choosing a better reward event, not by silent reverse credit.

That is enough to train a bandit on real lifecycle outcomes. It is not enough to claim multi-touch truth or interference-free causal effects. Keep those limits visible, and the learning table stays honest.