Nauseam, Chad

Android Calculator

Making a precise calculator is not trivial. While \(10^{100} + 1 - 10^{100} = 1\), iOS’s calculator gives back \(0\). explores how Android’s calculator gets this right courtesy of .

Real number representations are fundamentally imprecise because we can’t squeeze in infinitely many real numbers into a finite number of bits. Floating-point representations have a base \(\beta\) (assumed to be even) and a precision \(p\), e.g., if \(\beta = 10\) and \(p = 3\), then \(0.1\) is represented as \(1.00 \times 10^{-1}\). IEEE 754, a dominant floating point representation, requires \(\beta = 2, p = 24\) for single precision and \(p = 53\) for double precision. In IEEE 754, \(0.1\) cannot be represented exactly as \(1/10\) is not a factor of \(2\); it is approximately \(1.10011001100110011001101 \times 2^{-4}\).