The timing estimand comes before the timing model

“Send-time optimization” sounds like one product feature. In practice it hides at least three statistical questions.

1. Does the full contact policy beat doing nothing?
2. Given that we decided to act, does a learned send window beat a random allowed window?
3. At each clock tick, should we send now or wait, considering fatigue and later chances?

Those questions have different populations, treatments, controls, outcomes, and units. A model name (Thompson sampling, survival model, deep Q-network) does not settle which question you are answering. The estimand comes first. BanditBuzz’s current timing design is built around that order: choose whether and how to act, then assign one future allowed window among actions, and report timing lift only conditional on acting.

Two questions BanditBuzz answers today

The product reports two live causal comparisons. They look similar on a scoreboard. They are not interchangeable.

	Whole-policy lift	Timing lift (conditional on acting)
Population	Matured decisions under one decision-point version	Non-holdout decisions where the action policy chose a non-nothing action, timing was assigned, and a direct destination accepted the release
Treatment	The live action policy (and, when enabled, its timing policy)	Learned timing windows from that policy
Control	Stable person-level holdout (about 10% of stable person keys), which receives nothing on push	Nested stable timing control (about 10% of stable person keys among actionable decisions), which receives a uniformly random allowed window
Outcome	Declared business reward inside the declared window	Same declared business reward, with the reward clock starting at successful release
Unit	One matured decision	One matured, released decision with a timing slot

The two current estimands. Frequency and long-term pacing need a separate design.

Both comparisons use the same reward definition you declared (for example, a purchase within seven days). Neither silently switches the goal to opens or clicks. The difference is what is held fixed and what is randomized.

Whole-policy lift: whether contact helps

Whole-policy lift answers question 1. A stable hash of the decision-point version and a person key puts about one in ten people in holdout. On push, those people always get nothing. Everyone else gets the live policy’s choice among the approved options, including nothing when the learner prefers waiting for that cycle.

The absolute effect is the matured reward rate under the learned policy minus the matured reward rate in holdout. Relative lift divides that difference by the holdout rate when the holdout rate is positive. The live range uses an asymptotic 95% confidence sequence in the style used for enterprise A/B platforms in Maharaj et al.’s Anytime-Valid Confidence Sequences in an Enterprise A/B Testing Platform (2023). You may inspect that range repeatedly. It still does not remove dependence when one person contributes many decisions.

This estimand is whole-policy by design. It mixes action choice, eligibility, cooldown, and timing whenever timing is on. It does not isolate “Tuesday at 3pm” as a free-standing treatment effect. Option-level reward rates on the scoreboard remain descriptive summaries. Because the policy changes who receives each option, comparing those rates does not identify a fixed causal effect between messages.

Greenewald, Tewari, Murphy, and Klasnja’s Action Centered Contextual Bandits (2017) give the conceptual reason to keep a clear no-action baseline. In their mobile-health setting, “there is almost always a ‘do nothing’ action usually called action 0,” and the expected reward for that action is a baseline that can move in complex ways, while the incremental effect of intervening is often the quantity worth modeling simply. BanditBuzz’s push path requires nothing in the option list for the same identification reason: without it, whole-policy lift cannot mean “policy versus no contact.”

Timing lift: when to act, given that we act

Timing lift answers question 2. It is nested inside the treated side of the whole-policy experiment.

After the action policy chooses a non-nothing option, BanditBuzz resolves the person’s clock, builds the next occurrence of each allowed three-hour window inside the marketer’s days and hours, and assigns one of those windows. About 10% of stable person keys are timing control. They get a uniformly random allowed window. The rest get a window from the timing policy: Beta-Bernoulli Thompson sampling over windows on the standard tier, or a separate hierarchical linear Thompson model on the contextual tier. Action and timing propensities are logged separately.

The comparison is learned allowed window versus randomized allowed window, among actions the main policy already chose to take and a direct destination already accepted. That sentence has three hard filters:

1. Global holdout people are excluded. Their job is whole-policy control, not timing control.
2. nothing choices are excluded. There is no send time for silence.
3. Pending, failed, and canceled deliveries do not enter timing training as negative outcomes. The reward window starts only after successful release.

So timing lift does not estimate whether acting was useful. A large positive timing lift with a flat or negative whole-policy lift is possible: you may be sending better timed messages that still do not beat leaving people alone. That is a feature of honest estimands, not a bug in the chart.

The third question: repeated act-or-wait

Question 3 is a different experimental object. Liao et al.’s sample size work for micro-randomized trials (2016) describes treatments that are “sequentially randomized throughout the conduct of the study,” so each participant may be randomized at hundreds or thousands of occasions. The scientific target is often a proximal, near-future effect of a just-in-time prompt. That design needs a carefully chosen proximal outcome, availability rules, and methods that respect within-person dependence across many decision times.

Notification pacing systems push the same idea further. Yuan et al.’s Offline Reinforcement Learning for Mobile Notifications (2022) argue that a user’s experience depends on a sequence of notifications, that attributing long-term engagement to a single send is often inaccurate, and that reinforcement learning is a better framework when short-term and long-term impacts diverge. Their field deployment optimized sequential send-or-wait decisions for site engagement while reducing volume pressure. That is evidence about full contact policies over time, not about one future-window assignment after an action was already chosen.

BanditBuzz does not claim that third estimand today. Timing assignment is myopic: one future window for one action inside a cooldown cycle. The hourly push sweep is an eligibility loop, not a micro-randomized trial at every clock tick. There is no repeated reconsideration of the same pending person minute by minute, and no learned weekly volume planner inside the timing bandit. Frequency and quiet hours remain marketer-declared limits (eligibility, cooldown, allowed days and hours). They are hard constraints, not arms the learner explores.

If your real question is “how often should we contact this person this month?”, you need a pacing or frequency estimand, a reward that reflects fatigue and long-horizon value, and usually a sequential method. Hiding that problem inside a send-time bandit will produce a number that looks like timing lift while answering neither question cleanly.

Proximal clicks versus business windows

Many published timing gains use proximal engagement. Okoshi, Tsubouchi, and Tokuda’s Yahoo! JAPAN deployment of adaptive push notification scheduling (2019) reported large click-rate gains from context-aware timing. Predictive open-time models, such as Sinha et al.’s RNN-survival approach to email send times (2020), treat unopened messages as censored observations and forecast convenient hours. Those results support the idea that time matters. They do not automatically transfer to a seven-day purchase window.

BanditBuzz keeps the declared business event as the reward for both action and timing learning. That choice is slower and noisier than optimizing opens. It is also closer to what lifecycle teams usually mean by “worked.” Delayed feedback is handled as delayed, not missing: Howson, Pike-Burke, and Filippi’s Delay-Adaptive Learning in Generalized Linear Contextual Bandits (2022) support keeping actions pending and updating only after outcomes become available. In product terms, act_at is the intended release time; reward_window_started_at is set only after the destination accepts the pick; events before release cannot reward the action.

Direct execution is part of the treatment

A warehouse row or download file can carry an act_at recommendation. It cannot prove when the client’s system acted. BanditBuzz therefore claims learned timing only for destinations where releasing the pick triggers the action. Advisory destinations can still receive planned times for humans or downstream jobs, but they do not support the timing estimand as an observed treatment.

That rule is semantic, not cosmetic. Causal timing needs a treatment timestamp you control. If your ESP or CRM fires hours later, or a batch job ignores the suggested time, the logged window is no longer the intervention. The nested timing control then compares policies over intentions, not over contact times.

Action-by-time interactions and factorization

BanditBuzz factorizes action choice and timing choice. Each has its own offered set and propensity. Contextual timing can include the chosen action as a feature, so the timing model can learn that a reminder and an offer prefer different windows. The current previews still evaluate the two stages separately: action replay leaves the logged timing process in place, and timing replay conditions on an accepted action. There is no joint action-by-time replay that multiplies both propensities and rewrites both stages together.

That is an honest limit. Factorization keeps the timing arm space small (three-hour windows, capped so a declaration cannot explode into dozens of arms) and lets action evidence transfer from pull decisions. It does not identify every action-by-time interaction a fully crossed design would target. If your scientific claim is “this exact message only works on Saturday morning,” you need that crossed estimand and enough traffic to support it. The product does not claim it yet.

What BanditBuzz runs today

Concrete product facts that fix the estimand:

• Nested controls. Person-level holdout for whole-policy lift; nested timing control for timing lift among actionable, accepted sends.
• Three-hour windows. Allowed local days and hours are partitioned into three-hour learning slots (product maximum 35 slots per declaration). Finer hourly arms need denser data than most accounts start with.
• Ninety-day timing evidence. Timing posteriors and timing analysis use released actions from about the last 90 days so seasonal habits can replace old evidence. That length is a launch setting. It is not a result proved by the cited papers, and it still needs a production benchmark against discounted or uncertainty-based history rules. Action-choice evidence currently keeps full history.
• No repeated clock-tick reconsideration. One future window per actionable decision; pending plans block another decision for that person; cooldown begins at release, not at planning.
• Decision as the scoreboard unit. The current unit is a matured decision, not a unique person. A person can contribute more than one matured decision. The live ranges do not yet cluster by person, so within-person dependence remains a known gap.
• Standard versus contextual timing. Standard accounts learn which windows work for the decision point as a whole. The paid contextual tier can use person context plus the chosen action. Both still answer timing conditional on acting.

Thompson sampling for the free action and timing policies follows the probability-matching tradition of Thompson (1933). The model family is secondary. The primary product claim is that the logged randomization matches the question on the scoreboard.

How to choose the estimand before the vendor pitch

Ask these questions in order.

Are you trying to prove contact helps? You need a stable no-action control and a whole-policy effect. Timing alone cannot answer that.

Are you trying to prove better clocks help among sends you already approve? You need a timing control inside allowed windows, a treatment timestamp you actually execute, and an outcome that matches the business claim. BanditBuzz’s nested timing lift is built for that question.

Are you trying to prove a better long-run contact policy? You need repeated act-or-wait randomization or an offline RL design, proximal and long-horizon rewards that match fatigue, and methods that treat sequences as sequences. Micro-randomized trials and notification RL papers are the right literature. A single future-window bandit is not a substitute.

Will the reward be a click or a purchase? State it. Proximal engagement can move while business results do not. BanditBuzz will not silently redefine your reward to make timing look better.

Is timing an executed treatment or a suggestion? If your stack cannot fire at act_at, do not report timing lift as causal contact-time optimization.

Vendors will keep selling “send-time optimization” as one phrase. Your job is to name the population, treatment, control, outcome, and unit before you trust the chart. BanditBuzz’s current answer is explicit: whole-policy lift against holdout, and timing lift against a random allowed window conditional on acting, with three-hour windows, ninety-day timing evidence, direct delivery as the treatment clock, and no claim yet to repeated clock-tick pacing. Choose that question on purpose, or choose a different one. Do not let a model name choose it for you.