Most bandit tutorials assume the world stays put. The reward rate for each action is a fixed unknown number. Collect enough matured wins and losses, and the posterior settles. That story is useful for teaching Thompson sampling. It is incomplete for marketing.

Seasonal calendars move. Creative wears out. A competitor changes an offer. A new template starts strong and then fades. The same person who responded in March may ignore the same message in July. If your learner treats every past matured decision as equally informative forever, yesterday can keep voting after it stops describing today.

This post is a frank research agenda, not a claim that BanditBuzz has solved drift. The product is explicit about that gap on the research landing page: action and contextual models keep full history; timing alone uses a 90-day window; there is no automatic drift switch yet.

Stationarity is a modeling choice

A stationary reward process means the probability that an action produces the declared result does not change with calendar time. Under that assumption, older evidence is still about the same quantity as new evidence. Pooling all matured decisions is then the efficient thing to do.

Marketing traffic rarely meets that assumption for long. Two common failure modes matter for policy design:

Abrupt changes. A breakpoint arrives. A holiday ends, a product ships, a major creative rotates out. Expected rewards jump and then sit still until the next break. Between breaks the world looks stationary again. Across breaks it does not.

Smooth drift. Expected rewards crawl. Wear-out, habit change, and slow competitive pressure move rates by a little each week. There may be no single day you can point to as the change.

Both appear in the non-stationary bandit literature. Fiandri, Metelli, and Trovò analyze sliding-window Thompson sampling for environments whose means can change without forcing one special form of non-stationarity, then specialize the same analysis to abrupt and smooth regimes. Their paper is a next test for BanditBuzz on seasonality and creative wear-out, not a shipped product feature.

If you keep a full-history posterior under either regime, the learner does not “know” that the world moved. It only sees a growing pile of counts. After a long quiet stretch, a sudden change has to fight a large stock of old wins and losses before the posterior moves. That lag is the practical cost of assuming stationarity when it is false.

What full history does to a Thompson posterior

BanditBuzz’s standard action policy is Beta-Bernoulli Thompson sampling. Each option has a Beta posterior built from matured decisions for that decision point. New options start at Beta(1, 1). Every night, attribution recounts matured wins and losses across the option’s full history. There is no action-side discount and no action-side sliding window.

That design has clear strengths when the reward process is roughly stable: it uses every honest observation, it needs no exploration-rate knob, and it keeps logging randomized propensities for later replay. It also has a blunt failure mode under drift. Suppose an email template earned a strong rate for six months and then wore out. The posterior still carries those early months at full weight. The policy keeps sampling as if the old rate is still plausible until enough new zeros accumulate to pull it down. During that lag, people keep receiving a worn option while a newer one waits for evidence.

The contextual action model has the same history rule in a different shape. It fits a hierarchical linear Thompson model on matured decisions with decision-time context. Nightly learning still uses the available action history without an automatic forget rule. Old rows keep influencing coefficients. If the mapping from context to reward shifts, the model can be slow for the same reason a Beta count is slow: yesterday’s evidence is still in the fit.

None of this is a hidden bug. The policy contract on the landing page states it directly: action learning keeps full history; only timing uses a 90-day window; there is no automatic response to seasonality or creative wear-out.

Timing already forgets. Action choice does not.

Push timing is a separate policy from action choice. After the main policy picks a non-nothing action, timing chooses a three-hour local window inside days and hours you allow. Standard timing uses Beta-Bernoulli Thompson sampling over those windows. Contextual timing uses a separate hierarchical linear model that can include person context and the chosen action.

Both timing models train on released actions from the most recent 90 days. That constant lives in product code as a launch setting, not as a theorem. The research note behind it is practical: ninety days covers roughly twelve weekly cycles while letting older seasonal patterns fall out of the timing posterior. It is not a claim that ninety is optimal, and it is not a result proved by the sliding-window papers. It still needs a production benchmark against shorter windows, longer windows, and discounted alternatives as real accounts accumulate data.

So BanditBuzz already runs two history rules side by side:

  • Action choice and the contextual action model: full matured history.
  • Timing: latest 90 days of released, trainable actions.

That split is intentional for launch, not a proof that action choice should keep infinite memory. Timing habits often change faster than the relative ranking of offers. Forgetting on the timing side is a smaller first bet. Leaving action evidence permanent is the stronger stationarity assumption, and it is the one most likely to lag when creative wears out or a season flips.

What sliding windows change

A sliding-window policy keeps only the last τ observations (or the observations inside a fixed calendar lookback) when it updates each arm’s posterior. Classic Thompson sampling still draws once per option and picks the largest draw. The change is which counts enter the Beta parameters.

In the Beta sliding-window form studied by Fiandri, Metelli, and Trovò, each arm’s posterior at time t is built from pulls and rewards inside the window, still starting from a Beta(1, 1)-style prior. Older rows leave the window and stop counting. After a breakpoint, the learner can recover once the window fills with post-change data. Before that fill completes, estimates are noisier because the effective sample size is capped by τ.

That tradeoff is the whole point:

  • Smaller τ: reacts faster after a change; variance stays higher in quiet periods; rare options may look under-sampled forever.
  • Larger τ: more stable estimates when the world is quiet; slower recovery after a break; more risk that pre-change data still dominates.

A window does not detect change. It passively forgets. If nothing changed, you paid for that forgetfulness with weaker estimates. If something did change, you paid less than a full-history posterior would.

BanditBuzz’s timing 90-day rule is a calendar sliding window on released actions. It is not yet applied to action posteriors or to the contextual action model. Calling the timing window a “solved drift system” would oversell it. It is one history rule on one subproblem.

What discounting changes

Discounting is the other common passive forget rule. Instead of a hard cutoff, older observations keep a shrinking weight. A discount factor γ close to 1 forgets slowly. A smaller γ forgets quickly. Discounted UCB and discounted Thompson sampling appear throughout the non-stationary survey in Fiandri, Metelli, and Trovò; related product research also points at discounted and sliding-window Thompson analyses as the next tests before changing the live method.

Compared with a hard window:

  • Discounting never fully deletes an observation; it fades it.
  • There is no single day when an old win disappears, so monitoring is smoother and less “cliff-like.”
  • Tuning γ is as consequential as tuning τ. The wrong discount either keeps ancient seasons alive or throws away useful stable signal.

Neither rule is free. Both trade bias under stationarity for responsiveness under change. Active methods go further: they try to detect a breakpoint and reset. Those detectors need false-alarm control and enough post-change samples. BanditBuzz does not ship an automatic detector today.

New options and worn options are different problems

Drift is easy to confuse with cold start.

A new option starts with almost no evidence. Under Beta(1, 1) Thompson sampling it explores heavily until matured decisions arrive. That is uncertainty about a young arm, not seasonality.

A worn option often has a large, flattering history and a recent decline. Full-history pooling is worst here: the old wins keep the posterior optimistic. A sliding window or discount can help the worn option fall, but only after enough recent losses enter the recent evidence set.

A marketer who rotates creative every few weeks creates both problems at once. New templates explore. Old templates may still look strong on lifetime counts. If you only watch “which option has the highest all-time reward rate,” you will often keep a worn winner too long. The live holdout still estimates the whole policy against control, but it will not by itself tell you that one template’s calendar season ended.

Changing the reward definition is not the same as adapting to drift

BanditBuzz treats a changed reward event, window, or value field as a new decision-point version. The new version gets a fresh policy state and a fresh holdout assignment. Existing decisions stay tied to the rule they were made under. Pending scheduled actions that have not been released are canceled.

That version boundary is the right response when the definition of success changes. It is not an automatic response when the rate of the same success event drifts. Black Friday ending is not a new reward definition. It is the same purchase event with a different seasonal rate. Creative wear-out is the same. You still need a history rule, a manual version cut, or both.

If you change the reward to paper over a bad season, you also break continuity of the scoreboard. Prefer changing history rules or opening a new version when the business question truly changed, not when you merely dislike last month’s estimate.

What a practical monitor should watch

Until there is a production benchmark for action-side windows or discounts, operators still need a visible drift agenda. Useful checks, without pretending they are a shipped detector:

  1. Compare recent matured reward rates with lifetime rates for the same option. A large gap is a wear-out or season signal, not proof by itself.
  2. Watch the live holdout range and the estimate together. A policy that adapted too slowly can look fine on lifetime descriptive rates while the holdout gap stalls or shrinks for the wrong reason. The scoreboard’s anytime-valid confidence sequence is built for repeated viewing (Maharaj et al.); it does not yet call out seasonality as a separate alarm.
  3. Separate timing lift from whole-policy lift. Timing already forgets on a 90-day clock. If timing lift moves while action choice lags, that is consistent with the current split history rules.
  4. Treat delayed rewards as part of the lag. Unrewarded choices stay pending until their window closes. After a true breakpoint, you wait for both new decisions and maturity. Batch nightly updates add another delay on top of that. Delayed-feedback bandit work such as Howson, Pike-Burke, and Filippi supports keeping outcomes pending and updating when they arrive; it does not remove calendar drift.
  5. Do not read per-option rates as fixed effects. The policy changes who receives each option. Descriptive rates can move because assignment moved, not only because the world moved.

None of these checks replace a controlled comparison of full history against sliding-window and discounted policies on logged BanditBuzz traffic.

Nearby research that is not a drift patch

Two other papers on the landing research list matter for the next policy tests, and neither one should be mistitled as “the drift fix.”

Gouverneur et al. give an information-theoretic analysis of Thompson sampling for logistic bandits. That work is aimed at a better treatment of binary rewards than a Gaussian linear approximation. A cleaner reward model can still be wrong about calendar time if it keeps full history. Logistic TS is a candidate upgrade for the contextual reward link, not a substitute for a forget rule.

Anand and Liaw benchmark Feel-Good Thompson sampling and related variants across linear, logistic, and approximate-posterior settings. That paper is a comparison set for modern contextual exploration, including cases where exact posteriors are replaced by approximate samples. It is useful when we evaluate whether a compact contextual policy should change how it explores. It does not, by itself, choose a history window for seasonality.

Keep those distinctions sharp. Better exploration under a stationary model and better forgetting under a changing model are different knobs.

What BanditBuzz will not claim yet

Be wary of the vague sentence “the AI adapts.” Adaptation needs a defined history rule, a defined update schedule, and a defined evaluation. Today the honest product statements are:

  • Action posteriors and the contextual action model use full history.
  • Timing uses the latest 90 days of released actions.
  • There is no automatic drift switch, action-side discount, or action-side window.
  • The scoreboard does not yet flag seasonality or creative wear-out as named events.
  • Sliding-window and discounted policies remain tests to run before changing the live action method.
  • The 90-day timing window itself still needs a production benchmark.

That is not a marketing apology. It is the difference between a research agenda and a pretend result. Full history is a strong default when rates are stable. When yesterday stops predicting tomorrow, the next sound step is to measure which forget rule recovers faster on real decision logs, not to hide the lag behind a slogan.