= Remaining-life validation error
A <remaining-life validation error> compares a forecast of <remaining useful life> made at a specified time with the subsequently observed time to the same <end-of-life criterion>. If the criterion is reached at $t_f$ and a prediction is issued at $t<t_f$, the reference life is $R(t)=t_f-t$. Define the signed error as
$$
e(t)=\widehat R(t)-[t_f-t].
$$
With this convention, positive error means predicting more useful life than was observed. A forecast must retain its issue time and endpoint definition for this difference to have a physical meaning.
For an original cycle-count example, suppose the specified endpoint is observed at cycle100. At cycle60, a forecast of50 remaining cycles has reference life40 and error $+10$. At cycle90, a forecast of20 has reference life10 and the same error $+10$. Their absolute relative errors are25% and100%, respectively. Equal cycle errors therefore need not have equal significance near the endpoint. A relative error divided by the reference life is undefined at the endpoint itself.
Errors should also be distinguished from aggregate scores. Two forecasts with errors $+10$ and $-10$ have zero mean signed error, but mean absolute error10 and <mean squared error>100 in squared cycle units. Cancellation can hide large individual discrepancies. Reporting errors by prediction time preserves information that a single average discards.
NASA's controlled battery tests supply a concrete physical setting: measured current, terminal potential and temperature accompany charge/discharge records, with a defined capacity-loss aging endpoint. The records can support testing forecasts against later observations while retaining the endpoint definition.<Li-ion Battery Aging Datasets by NASA>[“Data Acquisition”, measured discharge fields and “Intended Use”, run-to-failure records]{ref} Their different operating histories also make indiscriminate pooling at equal cycle count inappropriate.
The <remaining-life validation cutoff> separates information available when issuing the forecast from information revealed later for scoring it. Using the eventual crossing time to choose the forecast, or tuning a model on the same future record used to claim validation, changes the experiment. For a <prediction interval>, one can additionally count how often the subsequently observed life lies inside the interval, while retaining its stated probability level and assumptions about future operation.
Back to article page
