Floating-point underflow Created 2026-10-20 Updated 2026-10-25
Floating-point underflow occurs when a nonzero numerical result is smaller in magnitude than the representable range or loses precision in the subnormal range. A long product of probabilities can become zero despite a mathematically positive result. Adding logarithms avoids forming the product; adding alternative path probabilities then requires log-sum-exp or controlled rescaling.
Forward algorithm 2026-10-20
The forward algorithm computes sequence likelihoods by summing over all hidden paths through a Hidden Markov model. For emitting states, obeys . It reuses partial sums rather than enumerating exponentially many state sequences. Silent states require additional transitions without consuming an observation. Scaling or log-sum-exp prevents floating-point underflow.
Use a uniform independent DNA background, so each nucleotide has probability . Conditional on length five, its sequence probability is . The requested score, often called log-odds in sequence analysis, is a log-likelihood ratio against this background:
Using natural logarithms instead gives . Identically distributed independent positions do not, by themselves, require equal base frequencies. If the intended background instead has specified frequencies , the general answer is
The numerical value above uses the conventional equal-frequency random DNA model.
Taking a logarithm only after multiplying probabilities does not prevent floating-point underflow: the product may already have rounded to zero, and its logarithm is then . Accumulate logarithms before forming tiny products; where alternative paths must be added, use log-sum-exp in the forward algorithm or rescale its intermediate probabilities.
For the product of finite probability distributions, enumerate the Cartesian product of the two supports and multiply the two probability masses.
def product[T,U](d: FiniteDist[T], d2: FiniteDist[U]): FiniteDist[(T,U)] =
  new FiniteDist[(T,U)] {
    private val dom = for (x <- d.getDom; y <- d2.getDom) yield (x,y)
    def getDom: List[(T,U)] = dom
    def getProb(p: (T,U)): Double = d.getProb(p._1) * d2.getProb(p._2)
  }
Each pair appears once because both original supports have no duplicates. Exact probabilities are positive on that support, and normalization follows by finite distributivity:
For the pushforward of a finite probability distribution, sum all masses in each fiber of a function; simply overwriting repeated image values would lose probability.
def image[T,U](d: FiniteDist[T], f: T => U): FiniteDist[U] =
  new FiniteDist[U] {
    private val weighted = d.getDom.map(x => (f(x), d.getProb(x)))
    private val dom = weighted.map(_._1).distinct
    def getDom: List[U] = dom
    def getProb(y: U): Double =
      weighted.foldLeft(0.0) { case (total, (z,p)) =>
        if (z == y) total+p else total
      }
  }
Assume the given finite probability distributions are fixed and f is a pure total function on the source support. Distinct image values form exactly the positive-mass support. The fibers partition the original support, proving
For example, sending a discrete uniform distribution on to parity produces masses and , not four distinct outcomes. Exact algebra establishes the finite probability distribution laws; Double computes approximations, with the usual possibility of rounding and floating-point underflow for very small products.
Equivalence partitioning divides the input domain into classes expected to exercise the same relevant behavior, then chooses representatives and boundary cases from each class. It is a way to select tests, not a proof that every member of a class must behave identically in a faulty implementation.
Useful classes for this parser include integer-only inputs, a leading point, a trailing point, and digits on both sides; all-zero inputs and inputs containing ones; leading or trailing zeroes; one-digit and longer blocks; and large integer or tiny fractional magnitudes near floating-point rounding, floating-point overflow and floating-point underflow boundaries. For instance, compare '1', '1.', '.1', '1.1', '0001.1000', and digit strings just beyond the exact integer precision of Double. Empty digit blocks have the zero convention used above. If a surrounding validator rejects an entirely empty representation or a bare point, test those as separate invalid-format classes. Unexpected digits and multiple points are likewise validator tests, since the conversion fragments assume well-formed input. Include representatives with independently calculated positional values, rather than relying only on inverse conversion.