Welford-style online (streaming) statistics for RevIN normalization.
Mirrors the Python update_running_stats() in torch/util.py and
flax/util.py.
These utilities are called at every patch boundary during autoregressive
decoding. Numerical errors here compound over long horizons, so we
implement the numerically-stable two-pass algorithm and skip NaN/Inf
values to prevent data corruption.
Welford-style online (streaming) statistics for RevIN normalization.
Mirrors the Python
update_running_stats()in torch/util.py and flax/util.py.These utilities are called at every patch boundary during autoregressive decoding. Numerical errors here compound over long horizons, so we implement the numerically-stable two-pass algorithm and skip NaN/Inf values to prevent data corruption.