Skip to main content

Posts

Showing posts from May, 2021

Floating point reproducibility in parallel processing

The floating point reproducibility problem On a typical computer, the results of floating point operations can differ from the mathematical result. In general, floating point operations like addition may not satisfy all mathematical properties for that operation. For example, the associativity property of addition may not hold for all numerical values: $(x + y) + z \stackrel{?}{=} x + (y + z)$ A decimal representation of a number cannot always be represented by a finite binary representation in computer hardware. For example, the computed result of $0.1 + 0.01 - 0.1 - 0.01$ is not equal to $0$, but mathematically these values should be equivalent. For example, you can demonstrate this using the Python interpreter (Python 3.8.5): >>> 0.1 + 0.01 - 0.1 - 0.01 -5.204170427930421e-18 But mathematical properties do hold in other cases. For example, the result of $0.1 - 0.1 + 0.01 - 0.01$ is equal to $0$. >>> 0.1 - 0.1 + 0.01 - 0.01 0.0 Another problem results from rounding