Imagine two decision trees for the same win-back person.

In the first tree, the only question is “which message wins?” The arms are three templates. Every eligible person gets one of them. The learner can find a favorite template. It cannot ask whether any message was better than silence, because silence was never an option.

In the second tree, the first question is “act or wait?” If the answer is act, a second question picks among the approved sends. Waiting is an ordinary arm with the same reward definition as the sends. Organic purchases still count. That is the design this article defends.

The failure mode of “always treat”

Lifecycle tools are good at producing activity: emails sent, journeys entered, offers shown. A bandit that only chooses among treatments inherits that bias. It can report which treatment looks best among treated people while the right answer for many people is not to contact them this cycle.

That gap is not a taste preference. It is an identification problem. If every decision forces a send, the data cannot estimate the value of waiting. Conversion after a message is compatible with conversion that would have happened anyway. The policy then optimizes relative ranks among treatments and never learns whether treatment helps.

William R. Thompson’s classic probability-matching rule in “On the Likelihood that One Unknown Probability Exceeds Another” (1933) is useful here only if the arms match the real choice. BanditBuzz’s free policy uses a Beta-Bernoulli form of that rule. The posteriors are only as meaningful as the option list. If nothing is missing, the sampler compares treatments. It does not compare treatment with the baseline the business actually cares about: what happens when you leave the person alone for this cycle.

What “no action” estimates

In BanditBuzz, nothing is an ordinary option string. It is not a special UI state, a skipped sweep, or a missing log row. The policy can pick it for a treated person. Holdout can require it. Delivery workers skip it because there is nothing to execute, but the decision still enters the ledger and the scoreboard.

The estimand for that arm is simple: under the declared reward event and window, what is the matured result rate when the logged choice is no action? Purchases, renewals, or other business events that arrive without a send still attribute to that decision when identity joins them. That is intentional. Organic conversion is the baseline the action arms must beat.

Two distinctions matter immediately.

First, a learned nothing choice for a non-holdout person is not the same object as holdout. Holdout is a stable person-level assignment: about 10% of stable person keys for a decision-point version. Those people always receive the control choice. When nothing is in the option list, that control is nothing with selection probability 1. When it is absent (allowed only on request-time /decide calls), holdout receives a uniform random offered option. Push refuses a declaration that omits nothing, because outbound work without a clear no-action control cannot answer the question push exists to ask.

Second, the scoreboard’s live action lift is a whole-policy effect: matured reward under the learned policy minus matured reward in holdout. It is not an effect for any single offer. Option-level rates remain useful summaries. They are not clean causal contrasts for each message, because the policy itself decides who sees each option.

Why research separates baseline from treatment

Mobile-health and notification systems have long treated “do nothing” as a first-class action, not a missing label. Greenewald, Tewari, Murphy, and Klasnja’s Action Centered Contextual Bandits (2017) put the point directly: in those settings “there is almost always a ‘do nothing’ action usually called action 0,” and the expected reward for that action is a baseline that can move in complex ways, while the incremental effect of a non-zero action is often the quantity worth modeling simply. Their algorithms center estimation on the treatment effect relative to that baseline rather than forcing a single stationary model over the full observed reward.

That paper also marks a boundary BanditBuzz respects. Repeated clock-tick act-or-wait choices (reconsider the same person many times a day under fresh randomization) are a different problem from assigning one future action time after the policy has already chosen to act. BanditBuzz’s current push loop uses per-person cooldown and a later timing choice among allowed windows. It does not claim the full action-centered regret design, and it does not pretend a single no-action arm solves every pacing question those authors discuss.

Micro-randomized trials make the same conceptual cut for intervention science. Klasnja and colleagues describe micro-randomized trials (2016) as designs that can include whether an intervention component is on or off at a decision point, so researchers can estimate proximal effects of delivering versus not delivering a component. BanditBuzz’s business rewards often use multi-day windows rather than proximal JITAI outcomes, so we do not claim an MRT analysis. We do borrow the structural lesson: “send nothing now” must be in the randomized choice set if you want evidence about whether sending helps.

Five words that are easy to confuse

Marketers and data scientists often use contact-policy language loosely. BanditBuzz keeps five ideas separate.

No action (nothing). An ordinary arm in the action set. The learner may select it for people who are not in holdout. It earns organic conversions under the same reward rule as the other arms. Choosing it means: for this decision cycle, do not deliver an actionable pick.

Holdout. A fixed control group for measuring whole-policy lift. Holdout people are not exploring among offers. On push, they always get nothing. Their matured results define the control rate on the scoreboard. A treated person who draws nothing from Thompson sampling is still in the treated policy, not in holdout.

Cooldown. A marketer-set limit on how soon the same person may be reconsidered. It is the client-facing pacing contract for push. For a real outbound action, cooldown starts when the destination accepts the release. For nothing and holdout, cooldown starts at decision time. A pending scheduled action also blocks another decision while it waits. Cooldown is not learned. It is a guardrail.

Frequency. How often contact happens over a longer horizon (a week, a month, a subscription lifetime). Cooldown bounds reconsideration for one decision point. It is not a learned weekly budget, a cross-journey frequency cap, or a full sequential send/wait planner. Products that optimize long-term notification volume need a different method and usually a different reward. BanditBuzz does not yet ship that planner.

Timing. After the action policy chooses to act, timing picks a local three-hour window inside the days and hours you allow. A nested timing control receives a random allowed window so the scoreboard can report timing lift conditional on acting. Timing does not decide whether to act. A no-action decision has no act_at to learn. Timing lift answers “given we are sending, did the learned window beat a random allowed window?” It does not answer “was sending useful?”

If you collapse these five into one “optimization,” you will misread the scoreboard. Whole-policy lift can be positive while timing lift is flat. Cooldown can look “conservative” while frequency across other tools remains high. A rising nothing share can be either good policy or a sign that every offered action is weak.

How BanditBuzz implements the distinction

On push, options must include nothing. The API returns nothing_required if they do not. The public reason matches the product job: holdout members always draw it, and whether to act at all is the decision push exists to make.

Each sweep resolves population, eligibility, and cooldown, then runs the same policy path as /decide. Holdout keys get nothing. Other people get a Thompson (or contextual) draw over the full list, including nothing. Only non-nothing, non-holdout picks are actionable. Those may receive a timing plan and later release. Delivery destinations that execute actions receive only actionable picks. Warehouse and file destinations can carry the full decision log, including nothing and holdout, for audit.

Reward timing follows the same split. An actionable scheduled decision is not an observed treatment until the destination accepts it. Its reward window starts at successful release. Events before release cannot reward that action. Pending, failed, and canceled actions do not mature as negative results. A no-action or global-holdout decision starts its reward window at decision time. That rule keeps organic baseline on the same clock as the decision, while delayed sends wait for a real treatment timestamp.

The free policy still uses Beta-Bernoulli Thompson sampling on matured action results. New options enter with a Beta(1, 1) prior. There are no traffic floors, ceilings, or pinned options today. Action learning currently keeps full history; only timing uses a 90-day evidence window. Those product settings matter when you interpret a growing nothing share: the arm can accumulate evidence for a long time, and the system will not automatically forget an old quiet period when creative or seasonality changes.

What marketers should put in the box

You still own strategy, creative, consent, and execution. BanditBuzz chooses among actions you approve. For scheduled work, that list should almost always include nothing, plus the sends you are willing to run. Fixed launches and legally required messages should stay outside the learner. Quiet hours and allowed local days belong in the timing block, not in creative naming tricks.

A useful option list looks like execution keys your tools already understand, for example email_offer, sms_reminder, postcard, and nothing. Crossing channels in one decision is fine when your destination path can execute each choice. Packing offer, channel, and creative into one flat name is also fine; the current learner treats that string as one arm and does not factor the parts.

Do not expect nothing to replace every contact rule. Unsubscribe still excludes people from push sweeps. Eligibility still defines who is in the population. Cooldown still spaces reconsideration. Those controls protect attention even when the bandit would prefer to act.

Limits you should keep visible

No-action in the arm set is necessary. It is not sufficient for every contact-policy question.

It does not by itself solve frequency across multiple decision points. BanditBuzz does not yet stop the same person from entering two decision points, receiving two actions, or affecting both scoreboards. A person can be quiet in one journey and noisy in another.

It does not solve long-term pacing. LinkedIn-style send/wait reinforcement learning and related production systems treat spacing over many decisions as its own sequential problem. BanditBuzz’s current myopic reward window and cooldown are a bounded first design, not that planner.

It does not make timing lift a substitute for whole-policy lift. The nested timing experiment conditions on acting. Positive timing lift with negative whole-policy lift would mean better clocks on a policy that still should have waited more often.

It does not remove dependence across repeated decisions for one person. The scoreboard unit is a matured decision. Ranges do not yet cluster by person.

It does not auto-pin a minimum send rate. If every offered action is weak relative to organic baseline, the policy can move mass toward nothing. That can be the correct business answer. If your operating plan requires a floor of outreach, that constraint is not yet a BanditBuzz feature; keep required volume outside the learner or change the approved action set and eligibility.

Delayed business rewards and nightly updates remain part of the loop. Delayed-feedback bandit theory, including work such as Howson, Pike-Burke, and Filippi (2022), supports updating after outcomes become available. It does not make a seven-day purchase window behave like an immediate click.

A practical checklist

Before you call a decision point “optimized,” ask:

  1. Is nothing in the option list for every push declaration?
  2. Is holdout present and stable for this decision-point version, and is its control truly no action?
  3. Are you reading whole-policy lift against that holdout, not only option rates among sends?
  4. Is cooldown set to the reconsideration pace you intend, knowing nothing still starts cooldown at decision time?
  5. Are timing limits answering “when,” while nothing answers “whether”?
  6. Are fixed or required sends kept outside this learner so the no-action arm is not fighting a parallel always-on journey?

If the answer to the first question is no, the rest of the machinery can still look busy. It cannot learn whether treatment helps. Putting “do nothing” in the action set is how scheduled decisioning keeps a clear control, how organic conversion enters the same ledger as paid attention, and how a bandit earns the right to wait.