Scheduling an action without pretending it happened

A scheduled outbound system has a quiet failure mode: it treats a plan as if the treatment already happened. The policy picks a message, stores a send time, and the learner starts counting days toward a purchase window. If the message never leaves, or leaves late, or leaves after the person unsubscribed, the ledger still looks like an experiment. It is not.

The useful question is narrower than “when should we send?” It is: when does a planned action become an observed treatment? BanditBuzz answers by splitting the clock. Decision time, intended release time, accepted release time, and reward maturity are four different facts. Only acceptance starts the reward window for a scheduled action.

Four times, not one

For a real-time /decide call, the choice is the treatment. The reward window starts when the decision is recorded. For scheduled push, that rule would be wrong. The hourly planner can decide hours or days before anything reaches the person.

The decision row therefore carries distinct times:

• decided_at (the decision’s created_at): when the sweep chose the action and, if timing is on, the local window.
• act_at: the intended UTC release time for that window. It is a plan, not proof of delivery.
• reward_window_started_at: set only after a direct destination accepts the pick. This is when within_days begins for an actionable scheduled decision.
• Reward maturity: when a matching result arrives inside the window, or when the full window closes empty.

A no-action choice and a global holdout choice are different. They start their reward window at decision time, because there is nothing left to release. An actionable pick leaves reward_window_started_at null until delivery succeeds.

That split matches the delayed-feedback view in Howson, Pike-Burke, and Filippi: keep actions pending, and update learning from outcomes that have actually been revealed. Pending is not missing data. It is unfinished treatment.

The state model

Think in states a decision can occupy, not only in timestamps.

Planned and pending. The sweep wrote an actionable decision with an act_at in the future (or due now). Delivery is still pending. The person is blocked from another decision on that decision point: pending counts as cooling down. Cooldown for a released action starts at release, not at planning.

Due. act_at has passed. A separate worker, not the hourly planner, tries to release the pick.

Canceled. Before release, the person became ineligible, unsubscribed, the push declaration changed, the destination was archived, or the account closed. The delivery row is marked canceled with a reason. The decision row keeps canceled_at. The audit history stays. Learning ignores canceled rows. They do not mature as zeros.

Failed. The destination call threw. Delivery stays in error with the last error text and can be retried. The reward window still does not start. A failure is not a negative reward for timing or action choice.

Accepted. The destination accepted the pick. Delivery is delivered, and reward_window_started_at is set to that acceptance time. Only then can events after that time reward the decision, and only then can an empty window later mature as a zero.

Matured. Attribution found a reward event inside the window, or the window closed. Nightly recount updates posteriors from matured, non-holdout, non-canceled decisions.

Illustrative product view using fixed demo data. Timing limits live in the same push declaration as eligibility, cooldown, and destination.

Planner and release are different jobs

The hourly sweep is the planner. For each push-enabled decision point it finds eligible people, applies cooldown, chooses an action (including "nothing"), and when timing is enabled chooses an allowed local window. It inserts decision rows and queues pick deliveries. It does not pretend the message was sent.

Release is a one-minute job. It loads pending or errored deliveries whose act_at is null or already due, rechecks whether the pick may still go out, then calls the destination. Success and failure are recorded on the delivery row. Only success stamps reward_window_started_at on actionable decisions.

That lag is intentional. Planning can be coarse. Release can be fine. Mixing them into one hourly batch would either send early or wait up to an hour past the chosen window.

Release-time checks

A plan ages. Between decided_at and act_at, the person may unsubscribe, buy (and fail a not_within_days rule), or fall out of eligibility. The push config or destination may change.

Before sending, the delivery worker checks:

• the decision point is still current, push-enabled, and pointed at the same destination
• there is no unsubscribe event for that person
• every eligibility predicate still holds against current events

If any check fails, the worker cancels. It does not deliver a stale treatment and then invent a reward story. Canceled history remains available for ops and debugging; it does not enter timing or action training as a loss.

Direct delivery versus a recommendation row

Learned timing is only offered when release itself triggers the action. Today that means a direct action destination (Klaviyo). A warehouse table, object bucket, or downloadable CSV can still receive picks that include decided_at and act_at. Those rows are recommendations for another system. They cannot prove when that system acted, so BanditBuzz does not claim learned timing from them.

If you declare timing against a non-direct destination, the API refuses: the destination records recommendations but does not execute them at a precise time. That boundary keeps the treatment timestamp honest. Klaviyo’s Personalized Send Time product similarly chooses an hour inside a marketer-set local window and keeps a timing control; BanditBuzz’s version of the same idea only starts its own reward clock after its connector accepts the pick.

Local clocks and daylight saving

Marketers set allowed days and local hours. They choose a person clock with a fallback time zone, or one fixed time zone. Time zone is a fact about the person. It does not change identity matching or holdout assignment.

BanditBuzz divides each allowed range into three-hour slots (at most 35 slots). For each person it resolves the clock, builds the next occurrence of every slot after “now,” then assigns a slot: random for the stable timing control, Thompson sampling for the rest.

Slot recurrence uses Temporal with IANA zones and Temporal’s compatible daylight-saving rule:

• a local time that does not exist in spring (the skipped hour) moves forward to the next valid instant
• a local time that occurs twice in fall (the repeated hour) picks the earlier occurrence

Tests pin that behavior. A Sunday 02:00 to 05:00 window in America/New_York on the 2026 spring-forward morning does not invent a nonexistent 02:00; the occurrence lands on the valid later UTC instant. A fall 01:00 window uses the first 01:00, not the second. Learning therefore sees stable local labels without hand-rolled DST branches.

What the planner writes

When the policy chooses to act and timing is enabled, the decision stores more than the option name. It records the person’s resolved time zone (or the fallback), every offered slot, the selected slot signature, the timing propensity, whether the person landed in the timing control, and the UTC act_at. Action propensity and timing propensity are logged separately. That keeps the timing action space small and lets action evidence transfer from real-time decisions.

If the person is inside an open slot, act_at can be “now.” If the slot has already closed for this week, recurrence advances to the next matching weekday. The delivery worker will not release early: it only selects rows where act_at is null or act_at <= now().

Changing the reward definition, archiving the destination, or turning push off cancels pending actions for that point. Again the rows remain. Fresh decisions after a reward-rule change belong to a new decision-point version with its own policy and holdout. Old plans should not quietly finish under a rule they were not made under.

Delayed feedback, not silent zeros

Attribution for a scheduled action requires act_at and a started reward window. Events before acceptance cannot reward the action. Pending, failed, and canceled actions do not mature as negative results. That is the product rule on the landing page: planning does not start the reward clock; release to a direct destination starts it only after acceptance.

Nightly attribution joins reward events to decisions by resolved person identifiers (and email or phone when needed), with last-decision-wins inside the window. For scheduled actions the window is measured from reward_window_started_at, not from decided_at. For unscheduled decisions the window still starts at decision time. A late identity match can still attribute an event that occurred on time; it cannot resurrect an event that happened before release.

Timing posteriors count only released, non-canceled decisions whose reward window started inside the last 90 days. Old timing evidence ages out; action-choice evidence still uses full history. The 90-day timing window is a launch setting, not a result proved by the papers below. It aims to keep roughly twelve weekly cycles while letting seasonal habits replace stale ones. It still needs a production benchmark against discounted or uncertainty-based history rules.

Nested experiments stay nested

Two causal questions share the same ledger.

Whole-policy lift uses the stable person-level holdout: the full push policy against "nothing". That answers whether acting under the declared limits helps on the declared reward.

Timing lift is nested. Among non-holdout decisions where the action policy chose to act and a direct destination accepted the action, a stable timing control gets a uniformly random allowed window. Everyone else gets a window from the timing policy. That estimates learned timing against ordinary allowed timing, conditional on acting. It does not estimate whether acting was useful.

Both propensities are logged. Previews evaluate action and timing separately today: action replay leaves the logged timing process in place, while timing replay conditions on an accepted action. There is no joint action-by-time replay using the product of both propensities yet.

What the research supports, and what it does not

No single paper is this implementation. Together they justify a bounded design.

Howson et al. support learning under delay when pending actions stay pending and models update from revealed outcomes. That is why BanditBuzz refuses to treat “not yet released” as “did not convert.”

Okoshi, Tsubouchi, and Tokuda report large click-rate gains from context-aware push scheduling in a Yahoo! JAPAN deployment (382,518 users). Their outcome was interruptibility and clicks. BanditBuzz still optimizes the declared business event (purchase, renewal, and so on), not opens or clicks, so their field result motivates timing as a real lever without licensing channel metrics as the training target.

Sinha et al. model email send times with survival methods that treat unopened mail as censored. That supports time as useful context. Predictive open timing is still not the same as incremental business impact under a holdout and a nested timing control.

Klaviyo Personalized Send Time shows the product shape buyers already understand: choose an hour inside marketer limits, send in local time, keep a timing control. BanditBuzz’s nested timing experiment answers a related but narrower question: among actions the main policy already chose to take, does learned timing beat a random allowed window?

Limits that stay visible

Timing assignment is myopic. It chooses one future window for one action. It does not plan a week of sends or reconsider the person at every clock tick. Frequency and pacing are a later problem; other production systems often separate a slow pacing planner from a fast message selector for that reason.

The live timing comparison is conditional on acting. Whole-policy lift (learned policy versus no action) remains the holdout’s job.

Three-hour windows trade precision for enough evidence. Very large senders may later support hourly models, but sparse accounts should not start with 168 weekly arms. Delivery is a sound treatment timestamp only for direct action destinations. Warehouse and file destinations remain useful for audit and downstream jobs; they are not timing ground truth.

Repeated decisions for one person create dependence that the current ranges do not yet cluster by person. Long reward windows plus nightly attribution mean learning waits for the declared window plus batch lag. Pending choices stay out of training on purpose; the cost is slower updates, not cleaner fiction.

If you build your own scheduler, keep the same discipline: store the plan, release on a separate clock, recheck consent and eligibility at release, start the reward window only on acceptance, and leave canceled and failed rows in the audit trail outside the learning table. The schedule is not the treatment. Acceptance is.