Floating-Point Numbers
Why 0.1 + 0.2 is not 0.3: how IEEE 754 stores real numbers, where rounding error comes from, and how to compare, sum, and store money safely.
intermediate35 min#floating-point#ieee-754#precision#rounding#numerics
Prerequisites: Bitwise Operations
Why it matters
Every language gives you decimals that look like exact numbers and are not. A shopping cart that totals to 19.990000000000002, a test that fails because 0.1 + 0.2 === 0.3 is false, a loop with a float counter that never terminates, an embedding similarity of 1.0000001: all come from the same cause. Knowing how floating-point works lets you predict these issues, write correct comparisons, and pick the right type for money.
Intuition
You cannot write one third exactly as a decimal: 0.3333... goes on forever, so you cut it off and accept a small error. Computers have the same problem, but in binary. A fraction can be written exactly in binary only if its denominator is a power of two. Halves, quarters, and eighths are fine. Tenths are not: 0.1 in binary is 0.000110011001100... repeating forever, so the machine stores the closest value it can fit, which is slightly off.
Think of a ruler with markings that are not evenly spaced: very dense near zero and gradually further apart for larger numbers. Every real number is rounded to the nearest marking. Around 1 the markings are about 2.2e-16 apart. Around one quadrillion they are about 0.125 apart. Floating-point has constant relative precision, not constant absolute precision.
How it works
Almost all languages use the IEEE 754 double (64 bits): the same format as Python's float and every JavaScript number. It is scientific notation in binary:
value = (-1)^sign * 1.fraction * 2^(exponent - 1023)
1 bit 11 bits 52 bits
+------+------------+------------------+
| sign | exponent | fraction |
+------+------------+------------------+
As a concrete example, 0.15625 is exactly 1.25 * 2^-3, so it is stored with sign 0, exponent 1023 - 3 = 1020, and fraction 0.25.
- The fraction (mantissa) holds about 15 to 17 significant decimal digits.
- The exponent sets the scale, from about
1e-308to1e308. - Special values:
Infinity(overflow, or dividing a positive number by zero in JavaScript),-0, and NaN ("not a number"), which is never equal to anything, including itself. In Python,1.0 / 0.0raises an error, while JavaScript returnsInfinity.
Consequences
- Rounding error: most decimal fractions are approximations. Each operation rounds again, so errors accumulate. Adding 0.1 ten times gives
0.9999999999999999, not 1. - Equality is unreliable: never compare computed floats with
==or===. Compare with a tolerance:abs(a - b) < eps, or usemath.isclosein Python. A relative tolerance is better when magnitudes vary. - Integers are exact up to 2^53: doubles represent every integer up to 9,007,199,254,740,992 (
2^53). Beyond that, adding 1 can do nothing:2^53 + 1equals2^53. JavaScript exposes this asNumber.MAX_SAFE_INTEGER; useBigIntbeyond it. Python's ownintis arbitrary precision, so onlyfloathas the limit. - Order of operations matters:
(a + b) + ccan differ froma + (b + c). Adding a tiny number to a huge one loses the tiny number completely (absorption). - Catastrophic cancellation: subtracting two nearly equal numbers wipes out most significant digits and leaves mostly noise.
Taming error
- Use integers for money: store cents (or the smallest unit) as integers and format at the edge. Or use a decimal type:
decimal.Decimalin Python, or a decimal library in JavaScript. - Compensated summation: Kahan summation keeps a running correction of the low-order bits lost at each addition, giving much more accurate totals for long lists. Python's
math.fsumdoes something similar. - Sort or group before summing when you need reproducible results, since the order changes the rounding.
- Compare with tolerances and choose the tolerance for the problem (about
1e-9for typical values), not blindly.
| Operation | Time | Notes |
|---|---|---|
| Add / multiply two floats | O(1) | One hardware instruction |
| Naive sum of n floats | O(n) | Error can grow with n |
| Kahan sum of n floats | O(n) | Same cost class, far smaller error |
| Decimal arithmetic | O(1) to O(digits) | Exact for decimals but slower |
Code: seeing the error and fixing it
import math
def yes(flag):
return "true" if flag else "false"
print(0.1 + 0.2, yes(0.1 + 0.2 == 0.3))
print(yes(abs((0.1 + 0.2) - 0.3) < 1e-9))
total = 0.0
for _ in range(10):
total += 0.1
print(total, yes(total == 1.0))
def kahan_sum(values):
s, c = 0.0, 0.0 # c carries the lost low-order bits
for v in values:
y = v - c
t = s + y
c = (t - s) - y
s = t
return s
print(yes(kahan_sum([0.1] * 10) == 1.0))
cents = 10 + 20 # money as integer cents
print(cents, "->", "$" + str(cents // 100) + "." + str(cents % 100).zfill(2))
big = 2.0 ** 53
print(yes(big + 1 == big), yes(math.isnan(float("nan") - float("nan"))))const yes = (flag) => (flag ? "true" : "false");
console.log(0.1 + 0.2, yes(0.1 + 0.2 === 0.3));
console.log(yes(Math.abs((0.1 + 0.2) - 0.3) < 1e-9));
let total = 0.0;
for (let i = 0; i < 10; i++) total += 0.1;
console.log(total, yes(total === 1.0));
function kahanSum(values) {
let s = 0.0, c = 0.0; // c carries the lost low-order bits
for (const v of values) {
const y = v - c;
const t = s + y;
c = (t - s) - y;
s = t;
}
return s;
}
console.log(yes(kahanSum(new Array(10).fill(0.1)) === 1.0));
const cents = 10 + 20; // money as integer cents
console.log(cents, "->", "$" + Math.floor(cents / 100) + "." + String(cents % 100).padStart(2, "0"));
const big = 2 ** 53;
console.log(yes(big + 1 === big), yes(Number.isNaN(NaN - NaN)));Both versions print:
0.30000000000000004 false
true
0.9999999999999999 false
true
30 -> $0.30
true true
Line 1 shows the classic surprise. Line 2 shows the fix: compare within a tolerance. Line 3 shows error accumulating over ten additions. Line 4 shows that Kahan summation recovers exactly 1.0. Line 5 shows money kept in integer cents. Line 6: 2^53 + 1 is no different from 2^53, and NaN - NaN is NaN.
Why line 1 comes out that way: neither 0.1 nor 0.2 nor 0.3 is stored exactly, and the tiny errors do not cancel.
Common pitfalls
- Using
==on computed floats: compare with a tolerance, or usemath.isclose(a, b, rel_tol=1e-9). - Float loop counters:
for x = 0; x != 1; x += 0.1never hits exactly 1. Loop on an integer and computex = i / 10. - Assuming addition is associative: parallel and vectorized sums add in a different order and give slightly different answers. That is normal, so test with tolerances.
- Trusting printed digits: printing shows the shortest text that round-trips, not the exact stored value.
Decimal(0.1)in Python reveals the true value,0.1000000000000000055511151231257827.... - Ignoring NaN:
NaN == NaNis false, and NaN poisons every calculation it touches. Check withmath.isnanorNumber.isNaN. In sorting, NaN breaks comparisons. - Large integers in JavaScript: database ids above
2^53silently lose precision when parsed as numbers. Treat them as strings or BigInt.
Practice
Why is 0.5 exact but 0.1 not? easy
Hint
Which fractions have a denominator that is a power of two?
Show solution
0.5 = 1/2 and 0.25 = 1/4 have power-of-two denominators, so they are finite in binary. 0.1 = 1/10 has a factor of 5 in the denominator, so its binary expansion repeats forever and gets rounded. The same reason makes 1/3 impossible to write exactly in decimal.
Write an approximate-equality function medium
Hint
Absolute tolerance fails for huge numbers; relative tolerance fails near zero. Combine them.
Show solution
isclose(a, b) is true when abs(a - b) <= max(rel * max(abs(a), abs(b)), abs_tol). With rel = 1e-9 and abs_tol = 1e-12, values near 1e12 get a scaled tolerance, while values around zero still compare sensibly. This is the idea behind math.isclose.
Sum a list accurately medium
Hint
Track what each addition loses and feed it back in.
Show solution
Use Kahan summation: keep a compensation c. For each value compute y = v - c, t = s + y, c = (t - s) - y, then s = t. The correction c is the portion of y that was rounded away, subtracted from the next term. O(n) time, O(1) space. In production, prefer math.fsum in Python.
Detect a hidden 2^53 bug hard
Hint
Which integers can a double represent exactly? What happens at 2^53 + 1?
Show solution
A service parses 64-bit ids as JavaScript numbers. Ids 9007199254740993 and 9007199254740992 both become 9007199254740992, so two different records collide. Fixes: send ids as strings in JSON, parse with BigInt, or use a typed id format. Detect it with tests at the boundary (Number.MAX_SAFE_INTEGER + 2) and by checking Number.isSafeInteger on parsed values.
Interview questions
- Why does 0.1 + 0.2 not equal 0.3? Neither 0.1 nor 0.2 is exactly representable in binary; the nearest doubles add up to a value just above 0.3, so equality fails.
- How should you compare floats? With an absolute and relative tolerance, not equality.
- How do you store currency? Integer cents or a decimal type, never binary floats.
- What is the largest integer a double stores exactly?
2^53, about 9 quadrillion; beyond that, gaps appear.
Interviewers look for: a clear explanation of binary representation, tolerance-based comparison, and knowing the money and large-integer pitfalls.
Quick check
Which is the safest way to total prices like 19.99 and 5.01 in code?
Finished this lesson?
Further resources
- Floating-point arithmetic (Wikipedia) (article)
- Python tutorial: floating-point arithmetic issues and limitations (docs)
- The Floating-Point Guide (article)
- IEEE 754 (Wikipedia) (article)