Your Mannequin’s MSE Is Mendacity to You: Half II

In a earlier article, we noticed why coaching with MSE forces a mannequin to report solely the conditional imply, and the way Gaussian NLL lets it be taught an sincere per-step uncertainty $sigma$ as nicely. That was the one-step story. This put up picks up the place that left off.

All the pieces we constructed in Half I — the Gaussian NLL, the two-term loss, the proof that the mannequin learns each the conditional imply and the conditional variance — was about one-step-ahead prediction: given a context window of $T$ noticed values, predict the distribution of the very subsequent worth $x_{T+1}$

At inference time, in any actual utility, that is hardly ever sufficient. You wish to know what occurs over the following 10, 100, or 1000 steps. You need a forecast, not a single prediction. And in case your mannequin is probabilistic, you need that forecast to hold sincere uncertainty all the way in which to the top of the horizon.

This put up is about how to do that accurately. One might suppose instantly of the naive strategy, that’s making use of the mannequin repeatedly, feeding its predictions again as inputs is secretly a deterministic operation that discards the uncertainty the mannequin spent a lot effort studying. Fixing this requires pondering fastidiously about what a joint distribution over future values really is, and find out how to approximate it with Monte Carlo sampling. The repair is barely extra code than the naive model, but it surely offers a basically totally different and basically higher forecast.

All the pieces right here is constructed and checked within the companion pocket book: the 2 rollout approaches, utilized to the identical educated mannequin, on the identical sign, with protection and PIT diagnostics (all defined on this put up).

···

Recapping the one-step setting

In Half I, we educated a mannequin that takes a context window of $T$ noticed values and outputs two numbers:

$(mu_{T+1},; sigma_{T+1}) = f_theta(x_1, x_2, ldots, x_T)$

These parameterise a Gaussian distribution over the following time step $T+1$

$x_{T+1} mid mathbf{x}_{1:T} ;sim; mathcal{N}(mu_{T+1},; sigma_{T+1}^2)$

It is a conditional distribution, it tells us every part about $x_{T+1}$

Clear, well-defined, and precisely what the loss was designed to be taught. The issue begins the second somebody asks: okay, however what occurs over the following 50 steps?

···

The multi-step inference drawback

In any actual utility, you do not simply want the following worth. You want a forecast, a prediction over an prolonged horizon. “Will this sign cross the alarm threshold someday within the subsequent hour?” “What does the load profile seem like for the remainder of the day?” These questions require predicting not one step however many.

Suppose we wish to forecast the following $H$ steps: $x_{T+1}, x_{T+2}, ldots, x_{T+H}$

$p(x_{T+1},, x_{T+2},, ldots,, x_{T+H} mid mathbf{x}_{1:T})$

Discover what that is. It isn’t $H$ separate distributions. It is one distribution over $H$-dimensional area, a distribution over complete sequences. The worth at step $T+3$

There’s an ordinary software from chance principle, which is the chain rule that components any joint distribution right into a product of conditionals:

$p(x_{T+1:T+H} mid mathbf{x}_{1:T}) = prod_{h=1}^{H} p(x_{T+h} mid mathbf{x}_{1:T+h-1})$

Every issue is a one-step conditional distribution: “what is the distribution of the following worth given every part earlier than it?”, which is strictly what the mannequin was educated to provide.

So in precept, you’ll be able to construct the joint distribution one step at a time: predict step $T+1$

However here is the catch. For step $T+2$

The way you deal with this unknown is your entire story of this put up. There are two methods that look comparable in code however are conceptually and statistically very totally different.

Two approaches for probabilistic autoregressive forecasting.

Technique A: The deterministic rollout: feed $mu$ again

The only factor you could possibly do: at every step, predict $(mu, sigma)$, take solely the$mu$, plug it into the context as if it have been an noticed worth, and transfer on. The recursion appears to be like like this:

$hat{mathbf{x}}_{1:T+1} = [mathbf{x}_{1:T},; mu_{T+1}] qquad textual content{(append } mutext{, discard } sigmatext{)}$

$(mu_{T+2},, sigma_{T+2}) = f_theta(hat{mathbf{x}}_{1:T+1})$

$hat{mathbf{x}}_{1:T+2} = [hat{mathbf{x}}_{1:T+1},; mu_{T+2}]$

$vdots$

You get a sequence of means $mu_{T+1}, mu_{T+2}, ldots, mu_{T+H}$

def deterministic_rollout(mannequin, context, horizon):    mus, sigmas = [], []    for h in vary(horizon):        mu, log_sigma = mannequin(context)        sigma = torch.exp(log_sigma)        mus.append(mu.merchandise())        sigmas.append(sigma.merchandise())        # slide the window: drop the oldest worth, append mu        ctx = torch.cat([ctx[1:], mu.reshape(1)])    return np.array(mus), np.array(sigmas)

Easy and clear, however, sadly unsuitable in a approach that really issues.

Take into consideration what the mannequin is being requested at step $T+2$

However, that is the unsuitable query. The earlier worth was not precisely $mu_{T+1}$

The second query at all times has a strictly bigger reply. All the time. As a result of it has to account for two sources of randomness: the noise on the present step plus the uncertainty inherited from the earlier step.

By feeding $mu$ again, you are pretending the earlier prediction was good. The $sigma$ values you get seize solely the native noise at every step in isolation, the irreducible randomness if the enter have been identified precisely. They fully miss the propagated uncertainty, the truth that each enter from step $T+1$

Think about you are navigating by compass, and every studying has a small error. One step off track is ok, that is $sigma_{T+1}$

Now here is the important thing: should you recalculate your place at every step assuming you are precisely the place you deliberate to be, not the place you really are, your navigation uncertainty stays small on paper despite the fact that you are really drifting. After 50 steps, your map says “uncertainty: 2 meters.” Actuality says you is likely to be 200 meters off. The map is not mendacity about any single step’s compass error. It is mendacity by pretending that each earlier step landed completely, which none of them did.

Probabilistic multi-step inference drawback.

That is precisely what deterministic rollout does. Every $sigma_{T+h}$

Technique B: The stochastic rollout: pattern and feed again

The repair is straightforward: as a substitute of feeding $mu$ again, draw a random pattern from the anticipated distribution and feed that again. Run this $M$ occasions to get $M$ believable futures.

Step-by-step, for one trajectory (labeled $m$):

$x_{T+1}^{(m)} sim mathcal{N}(mu_{T+1},; sigma_{T+1}^2)$

$(mu_{T+2}^{(m)},, sigma_{T+2}^{(m)}) = f_theta(mathbf{x}_{1:T},, x_{T+1}^{(m)})$

$x_{T+2}^{(m)} sim mathcal{N}(mu_{T+2}^{(m)},; sigma_{T+2}^{(m),2})$

$vdots$

The superscript $(m)$ marks one trajectory, one doable future. As a result of step 1 was a random draw, the worth $x_{T+1}^{(m)}$

However one trajectory is only one doable future, a single random path by the area of sequences. To approximate the full joint distribution, you run$M$ unbiased trajectories, every from the identical noticed historical past $mathbf{x}_{1:T}$

In code, the important thing trick is to course of all $M$ trajectories as a single batch, one ahead move per horizon step, not $M$ separate passes:

def stochastic_rollout(mannequin, context, horizon, M=500):    # copy the context M occasions: (M, T)    ctx = context.unsqueeze(0).increase(M, -1).clone()    trajectories = torch.zeros(M, horizon)    for h in vary(horizon):        mu, log_sigma = mannequin(ctx)               # (M,) every        sigma = torch.exp(log_sigma)        # every trajectory attracts its personal random pattern        pattern = mu + sigma * torch.randn(M)        trajectories[:, h] = pattern        # slide every window: drop oldest, append this trajectory's pattern        ctx = torch.cat([ctx[:, 1:], pattern.unsqueeze(1)], dim=1)    return trajectories 

Take a minute to check the 2 features. The construction is similar, a loop over horizon steps, with a sliding context window. The solely distinction is one line: deterministic rollout appends mu, stochastic rollout appends pattern. That one-line change is the distinction between throwing away uncertainty and propagating it accurately.

Why uncertainty should develop: the legislation of complete variance

The instinct above, propagated uncertainty makes issues wider could be made exact with a single theorem: the legislation of complete variance:

$textual content{Var}(X) = underbrace{mathbb{E}[text{Var}(X mid Y)]}_{textual content{common native noise}} + underbrace{textual content{Var}(mathbb{E}[X mid Y])}_{textual content{uncertainty concerning the enter}}$

In plain language: the complete uncertainty about $X$ is the sum of two elements. First, the common noise you’d see even should you knew $Y$ precisely. Second, the further unfold brought on by the truth that totally different values of $Y$ result in totally different predictions for $X$. Since variances cannot be unfavorable, the overall is at all times at the very least as massive as both half alone.

Apply this to forecasting. Let $X = x_{T+2}$

$textual content{Var}(x_{T+2} mid mathbf{x}_{1:T}) = underbrace{mathbb{E}[text{Var}(x_{T+2} mid mathbf{x}_{1:T}, x_{T+1})]}_{textual content{Time period I: aleatoric}} + underbrace{textual content{Var}(mathbb{E}[x_{T+2} mid mathbf{x}_{1:T}, x_{T+1}])}_{textual content{Time period II: propagated}}$

Time period I is the anticipated one-step noise at step $T+2$

Time period II is the variance of the conditional imply $mu_2(x_{T+1})$

• $mu_1 = mathbb{E}[x_{T+1} mid mathbf{x}_{1:T}]$

• $mu_2(x_{T+1}) = mathbb{E}[x_{T+2} mid mathbf{x}_{1:T+1}]$

• $sigma_2^2(x_{T+1}) = textual content{Var}(x_{T+2} mid mathbf{x}_{1:T+1})$

Observe the important thing factor: $mu_2$

Deterministic rollout evaluates $sigma_2^2$

$textual content{Var}_{textual content{det}}(x_{T+2}) = sigma_2^2(mu_1)$

Whereas stochastic rollout, by its many trajectories, captures each phrases of the legislation of complete variance:

$textual content{Var}_{textual content{sto}}(x_{T+2}) = mathbb{E}[sigma_2^2(x_{T+1})] + textual content{Var}(mu_2(x_{T+1}))$

Subsequently:

$boxed{textual content{Var}_{textual content{stochastic}} = underbrace{mathbb{E}[sigma_2^2]}_{textual content{common noise}} + underbrace{textual content{Var}(mu_2)}_{textual content{propagated}} ;geq; underbrace{sigma_2^2(mu_1)}_{textual content{deterministic}}}$

The inequality holds as a result of $textual content{Var}(mu_2) geq 0$

Legislation of complete variance in probabilistic autoregressive forecasting

At step $T+3$

Every step provides its personal layer on high of every part gathered earlier than. Deterministic rollout misses each one and the hole would not simply exist, it grows at each step.

···

Monte Carlo Approximation: $M$ parallel trajectories

The stochastic rollout captures the fitting uncertainty, however one trajectory is only one pattern from the joint distribution. To approximate the total distribution, you run $M$ unbiased trajectories.

Think about 500 folks in a room, every holding a duplicate of the identical noticed historical past on a chunk of paper. At this second, all 500 papers are similar.

Spherical 1: everybody feeds the identical historical past into the mannequin. For the reason that inputs are similar, everybody will get the identical $(mu_1, sigma_1)$

Spherical 2: every particular person feeds their very own paper into the mannequin. Particular person 1’s paper ends in 0.41, particular person 2’s ends in 0.62, simply totally different inputs produce totally different $(mu_2, sigma_2)$

Spherical 3, 4, 5, … $H$: similar factor. At each spherical, the papers diverge additional as a result of every particular person’s context is formed by all their earlier random attracts.

After $H$ rounds, you’ve got 500 totally different doable futures. They began similar (similar actual historical past) and diverged as a result of every particular person rolled their very own cube at each step. This divergence is the uncertainty propagation and the quantity they’ve unfold aside by spherical $H$ is the sincere forecast uncertainty at that horizon.

Monte Carlo approximation of stochastic rollout.

This process is the Monte Carlo approximation of the joint distribution. You’ll be able to’t write that distribution down in closed kind, for the reason that mannequin is a nonlinear neural community, however you’ll be able to pattern from it precisely by the stochastic rollout. Every trajectory follows the chain rule factorisation we wrote earlier, sampling from the proper one-step conditional at each step. With sufficient samples, you’ll be able to estimate any property of the joint distribution you need: means, quantiles, exceedance possibilities, no matter.

What number of is sufficient?

The estimation error shrinks as $O(1/sqrt{M})$

• For the median (central forecast): $M = 20$

• For the fifth or ninety fifth percentile (tail behaviour): you want extra. At $M = 50$

• For protection checks throughout many take a look at examples: average $M$ per instance $50$–$100$) is ok, as a result of averaging throughout take a look at circumstances smooths the noise.

The fee is linear: $M$ trajectories means$M$ ahead passes per horizon step. However with batching, processing all $M$ contexts as a single batch, as within the code above, that is one GPU kernel launch per step, not $M$ separate calls. On the artificial sign from Half I, 500 trajectories over 50 steps runs in seconds on a CPU.

···

From trajectories to a forecast

You might have a matrix of form $M occasions H$

The central forecast: the median. At every horizon step, take the median throughout trajectories:

$hat{x}_{T+h} = textual content{median}(tilde{x}_h^{(1)},, tilde{x}_h^{(2)},, ldots,, tilde{x}_h^{(M)})$

Why median and never imply? If a couple of trajectories get lost to excessive values (it occurs, particularly at lengthy horizons), the imply will get pulled towards them. The median would not care, it simply appears to be like on the center. For symmetric distributions each are equal, so there is not any price in switching.

Uncertainty bands: quantiles. Type the $M$ values at every step. The band between the $(alpha/2)$-th and $(1 – alpha/2)$

median_forecast = np.median(trajectories, axis=0)     # (H,)lower_90 = np.percentile(trajectories, 5, axis=0)     # (H,)upper_90 = np.percentile(trajectories, 95, axis=0)    # (H,)

No Gaussian assumption right here. You are not computing $mu pm 1.645sigma$

Exceedance possibilities. Keep in mind the alarm threshold $tau$ from Half I? With $M$ trajectories, estimating the chance of crossing it’s trivial: depend what number of cross it, divide by $M$. If 35 out of 500 cross $tau$ sooner or later in the course of the horizon, your estimated exceedance chance is 7%. No closed-form integral wanted.

The important thing property of all these summaries: at early horizon steps, the trajectories cluster tightly, so bands are slender. At late horizon steps, the trajectories have fanned out, so bands are broad. The widening occurred naturally from the compounding randomness, you did not tune something, you simply sampled truthfully.

···

QA the forecast I: the coverage-horizon plot

One instance is anecdotal. The actual take a look at: run each rollout methods from many beginning factors within the take a look at set, and verify whether or not the 90% band really comprises the true worth about 90% of the time, at each horizon step.

For every take a look at window $i$ with true future $x_{T+1}^{(i)}, ldots, x_{T+H}^{(i)}$

$textual content{inside}_{h}^{(i)} = mathbf{1}!left[hat{q}_{0.05}^{(h,i)} ;leq; x_{T+h}^{(i)} ;leq; hat{q}_{0.95}^{(h,i)}right]$

Common over all take a look at home windows:

$widehat{textual content{cov}}^{(h)} = frac{1}{N_text{take a look at}}sum_{i=1}^{N_text{take a look at}} textual content{inside}_{h}^{(i)}$

For a calibrated forecast, $widehat{textual content{cov}}^{(h)} approx 90$

Now plot $widehat{textual content{cov}}^{(h)}$

• Flat line close to 90%: the rollout is nicely calibrated. The bands are sincere at each horizon. That is what you need.

• Line that drops as $h$ grows: the bands are too slender at lengthy horizons. The mannequin is overconfident concerning the distant future. That is the basic signature of deterministic rollout, it misses the propagated uncertainty, so even by step 50 the band is way too tight.

• Line that rises as $h$ grows: the bands are too broad. The mannequin is overly cautious.

Within the companion pocket book, the distinction is stark. The deterministic rollout’s protection crashes under 50% inside a couple of steps and retains falling, by step 50, it is barely above probability. The stochastic rollout holds roughly secure throughout the total horizon. Similar mannequin, similar context, similar educated weights, one line of code modified. That one line, changing mu with pattern within the context append is the distinction between a forecast you’ll be able to belief and one which’s silently mendacity about its confidence.

QA the forecast II: the PIT histogram

The protection plot tells you “are the bands the fitting width?” The Likelihood Integral Rework (PIT) tells you one thing richer: is the complete predicted distribution appropriate, not simply the width?

For every take a look at window $i$ and horizon step $h$, compute the fraction of trajectories that fell under the true worth:

$u_h^{(i)} = frac{1}{M}sum_{m=1}^{M} mathbf{1}!left[tilde{x}_h^{(m,i)} leq x_{T+h}^{(i)}right]$

That is the empirical quantile of the reality throughout the ensemble. If the reality landed proper within the center, $u approx 0.5$

Key property: if the ensemble is completely calibrated, $u$ must be uniformly distributed on $[0, 1]$. The reality ought to land within the backside 10% of trajectories about 10% of the time, within the center 10% about 10% of the time, within the high 10% about 10% of the time. Acquire $u$ values throughout all take a look at home windows and all horizons, and plot a histogram. The form is the prognosis:

PIT histogram shapes

One histogram, 4 doable diagnoses. Within the accompanying pocket book, the deterministic rollout produces a dramatic U-shape, the reality is within the excessive tails way more usually than it must be, as a result of the bands are too slender. The stochastic rollout is far flatter.

···

Sensible Issues

Maintain the clamp throughout rollout

Throughout rollout, the mannequin feeds by itself outputs, information it by no means noticed throughout coaching. That is distribution shift: the inputs look totally different from coaching information as a result of they’re the mannequin’s personal predictions, not actual observations.

On this unfamiliar setting, $sigma$ can drift to excessive values. With out the identical $logsigma$ clamp used throughout coaching, a single unfortunate trajectory would possibly produce a wild pattern that drives $sigma$ to zero (infinite confidence, gradient explosion) or infinity (complete ignorance) on the subsequent step. That corrupts the trajectory, and if it occurs early, each subsequent step is rubbish. The repair is definitely trivial, simply apply the identical clamp you used throughout coaching:

log_sigma = torch.clamp(log_sigma, min=-5.0, max=2.0)

If this clamp is already in your mannequin’s ahead() methodology (accurately from Half I), you needn’t change something. Simply do not take away it for inference.

Single-step head vs multi-step heads

Some architectures embrace multi-step prediction heads (resembling DeepSeek or SeismoGPT), auxiliary outputs that immediately predict the distribution at horizons 2, 3, …, $H$ from the present context, with out rolling out. These are helpful throughout coaching as regularisers and provides quick marginal distributions at every horizon. However marginal is the important thing phrase. A marginal distribution at step $h$ tells you the unfold of $x_{T+h}$

Solely the autoregressive stochastic rollout produces a joint distribution, precise sequences the place every step is dependent upon the earlier one. That is what you want for:

• Coherent trajectory sampling (a single believable future)

• Exceedance possibilities over home windows of time (“will the brink be crossed someday within the subsequent hour?”)

• Appropriate quantile bands that account for compounding uncertainty

The appropriate strategy: use the single-step head plus Monte Carlo rollout for inference. Deal with multi-step heads as coaching auxiliaries, not inference instruments.

Batching is essential for pace

The stochastic rollout processes all $M$ trajectories as a single batch of form $(M, T)$ one ahead move per horizon step. That is vital. $M$ separate ahead passes of form $(1, T)$ would give the identical outcome however be dramatically slower, particularly on a GPU the place the kernel launch overhead dominates for small fashions.

On the artificial sign from Half I (transformer with 32-dimensional embeddings, 2 layers, 4 heads), 500 trajectories over 50 steps runs in about 2 seconds on a CPU. Scaling to actual fashions and longer horizons, the batched strategy stays environment friendly.

···

What this does not assure

The stochastic rollout propagates uncertainty accurately, however solely the uncertainty the mannequin has realized to specific. If the one-step mannequin is miscalibrated (its $sigma$ is systematically too small or too massive), the rollout faithfully propagates that miscalibration. Rubbish in, rubbish out.

For this reason the diagnostics from Half I nonetheless matter. The one-step mannequin should be nicely calibrated first, sincere $sigma$ at each particular person step, earlier than rolling it ahead. The coverage-horizon plot and PIT histogram then inform you whether or not the rollout launched extra issues on high of that.

There’s additionally the query of distribution shift throughout rollout: the mannequin was educated on home windows of actual information, however throughout multi-step inference it is feeding by itself predictions. If the mannequin’s predictions drift for instance, if the imply trajectory slowly diverges from the true sign dynamics, the later context home windows look more and more not like something the mannequin noticed throughout coaching. The mannequin would possibly nonetheless output reasonable-looking $(mu, sigma)$, however the high quality degrades. It is a actual drawback, particularly at very lengthy horizons, and it is why multi-step coaching losses (coaching the mannequin by itself rollout outputs, not simply on actual information) may help.

There’s additionally a deeper assumption we’ve not questioned: the predictive distribution at every step is Gaussian. The rollout propagates regardless of the one-step mannequin outputs, but when the true noise is skewed, heavy-tailed, or multimodal, even a wonderfully calibrated Gaussian is becoming the closest bell curve to a non-bell-curve actuality. The imply and variance is likely to be proper, however the form is unsuitable and form issues for tail possibilities and threshold crossings.

···

Conclusion

One-step prediction offers you a distribution over one worth. Multi-step prediction wants a distribution over complete sequences. The chain rule says you’ll be able to construct it one step at a time, however provided that you propagate uncertainty accurately by the chain.

Deterministic rollout (feed $mu$ again) treats every intermediate prediction as sure. This captures solely the native noise at every step and ignores the inherited uncertainty from all earlier steps. The legislation of complete variance ensures that this underestimates the true forecast variance, and the underestimation compounds at each step. Protection degrades with horizon.

Stochastic rollout (pattern and feed again, $M$ occasions) lets uncertainty compound naturally by the autoregressive loop. Totally different trajectories diverge due to the randomness at every step, and the unfold of the ensemble at any horizon step displays each the native noise and all of the gathered propagation. Protection stays sincere.

The distinction in code is one line. The distinction in forecast high quality is prime.

The coverage-horizon plot and the PIT histogram are the instruments that inform you whether or not your multi-step forecast is calibrated. They apply whatever the mannequin structure, the distribution household, or the sign area. When you’re producing probabilistic forecasts at any horizon longer than one step, you want each.

And there is a deeper level right here. We began this sequence asking “ought to I fear about this prediction?” Half I gave the mannequin a voice, a approach to categorical uncertainty at a single step, by $sigma$. Half II gave that voice a reminiscence, a approach to carry uncertainty ahead over an prolonged future, by stochastic sampling. However take heed to what that voice is definitely saying. At each step it says: “I feel the following worth is drawn from a bell curve with this heart and this width.” A bell curve. Symmetric, unimodal, light-tailed.

Some alerts do not match that form. As an example, a seismic sign can have sudden regime modifications that create multimodal futures. For all of those, a single Gaussian per step is just too inflexible, regardless of how nicely you propagate it. That is the place richer predictive distributions, quantized tokens, combination fashions, movement matching grow to be important. The rollout equipment stays precisely the identical (pattern and feed again, $M$ occasions), however the one-step distribution turns into expressive sufficient to seize the total complexity of what would possibly occur subsequent. That is the place the following a part of this sequence begins.

···

References

[1] M. Seitzer, A. Tesch, N. Rasiwasia and G. Martius, On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks (2022), ICLR 2022

[2] W. Esmail, S. Russell, J. Klinge, A. Kappes and C. Thomas, Information-Pushed Forecasting of three-Part Seismograms Utilizing Transformer Architectures (2026), arXiv:2606.02912