 Remaining-life validation error

A remaining-life validation error compares a forecast of remaining useful life made at a specified time with the subsequently observed time to the same end-of-life criterion. If the criterion is reached at $t_f$ and a prediction is issued at $t<t_f$, the reference life is $R(t)=t_f-t$. Define the signed error asWith this convention, positive error means predicting more useful life than was observed. A forecast must retain its issue time and endpoint definition for this difference to have a physical meaning.

For an original cycle-count example, suppose the specified endpoint is observed at cycle100. At cycle60, a forecast of50 remaining cycles has reference life40 and error $+10$. At cycle90, a forecast of20 has reference life10 and the same error $+10$. Their absolute relative errors are25% and100%, respectively. Equal cycle errors therefore need not have equal significance near the endpoint. A relative error divided by the reference life is undefined at the endpoint itself.

Errors should also be distinguished from aggregate scores. Two forecasts with errors $+10$ and $-10$ have zero mean signed error, but mean absolute error10 and mean squared error100 in squared cycle units. Cancellation can hide large individual discrepancies. Reporting errors by prediction time preserves information that a single average discards.

NASA's controlled battery tests supply a concrete physical setting: measured current, terminal potential and temperature accompany charge/discharge records, with a defined capacity-loss aging endpoint. The records can support testing forecasts against later observations while retaining the endpoint definition.^[ref] Their different operating histories also make indiscriminate pooling at equal cycle count inappropriate.

The remaining-life validation cutoff separates information available when issuing the forecast from information revealed later for scoring it. Using the eventual crossing time to choose the forecast, or tuning a model on the same future record used to claim validation, changes the experiment. For a prediction interval, one can additionally count how often the subsequently observed life lies inside the interval, while retaining its stated probability level and assumptions about future operation.