DevPath
Tracks

Floating-Point Numbers

Why 0.1 + 0.2 is not 0.3: how IEEE 754 stores real numbers, where rounding error comes from, and how to compare, sum, and store money safely.

intermediate35 min#floating-point#ieee-754#precision#rounding#numerics

Prerequisites: Bitwise Operations

Why it matters

Every language gives you decimals that look like exact numbers and are not. A shopping cart that totals to 19.990000000000002, a test that fails because 0.1 + 0.2 === 0.3 is false, a loop with a float counter that never terminates, an embedding similarity of 1.0000001: all come from the same cause. Knowing how floating-point works lets you predict these issues, write correct comparisons, and pick the right type for money.

Intuition

You cannot write one third exactly as a decimal: 0.3333... goes on forever, so you cut it off and accept a small error. Computers have the same problem, but in binary. A fraction can be written exactly in binary only if its denominator is a power of two. Halves, quarters, and eighths are fine. Tenths are not: 0.1 in binary is 0.000110011001100... repeating forever, so the machine stores the closest value it can fit, which is slightly off.

Think of a ruler with markings that are not evenly spaced: very dense near zero and gradually further apart for larger numbers. Every real number is rounded to the nearest marking. Around 1 the markings are about 2.2e-16 apart. Around one quadrillion they are about 0.125 apart. Floating-point has constant relative precision, not constant absolute precision.

log10 of the numberlog10 of the gap to the next double11e31e61e91e121e15
Both axes are powers of ten. The points form a straight line: every time the number grows 1000 times, the gap between neighbouring doubles also grows about 1000 times (from 2.2e-16 near 1 up to 0.125 near 1e15).

How it works

Almost all languages use the IEEE 754 double (64 bits): the same format as Python's float and every JavaScript number. It is scientific notation in binary:

value = (-1)^sign  *  1.fraction  *  2^(exponent - 1023)

 1 bit      11 bits        52 bits
+------+------------+------------------+
| sign |  exponent  |     fraction     |
+------+------------+------------------+

As a concrete example, 0.15625 is exactly 1.25 * 2^-3, so it is stored with sign 0, exponent 1023 - 3 = 1020, and fraction 0.25.

0
sign (1 bit)
01111111100
exponent (11 bits) = 1020
0100000000000000000000000000000000000000000000000000
fraction (52 bits) = .01 in binary = 0.25
The three fields of 0.15625. The exponent field stores 1020, and the real exponent is 1020 - 1023 = -3. The leading 1 of 1.25 is implicit, so only the .25 is stored.
  • The fraction (mantissa) holds about 15 to 17 significant decimal digits.
  • The exponent sets the scale, from about 1e-308 to 1e308.
  • Special values: Infinity (overflow, or dividing a positive number by zero in JavaScript), -0, and NaN ("not a number"), which is never equal to anything, including itself. In Python, 1.0 / 0.0 raises an error, while JavaScript returns Infinity.

Consequences

  • Rounding error: most decimal fractions are approximations. Each operation rounds again, so errors accumulate. Adding 0.1 ten times gives 0.9999999999999999, not 1.
  • Equality is unreliable: never compare computed floats with == or ===. Compare with a tolerance: abs(a - b) < eps, or use math.isclose in Python. A relative tolerance is better when magnitudes vary.
  • Integers are exact up to 2^53: doubles represent every integer up to 9,007,199,254,740,992 (2^53). Beyond that, adding 1 can do nothing: 2^53 + 1 equals 2^53. JavaScript exposes this as Number.MAX_SAFE_INTEGER; use BigInt beyond it. Python's own int is arbitrary precision, so only float has the limit.
  • Order of operations matters: (a + b) + c can differ from a + (b + c). Adding a tiny number to a huge one loses the tiny number completely (absorption).
  • Catastrophic cancellation: subtracting two nearly equal numbers wipes out most significant digits and leaves mostly noise.

Taming error

  • Use integers for money: store cents (or the smallest unit) as integers and format at the edge. Or use a decimal type: decimal.Decimal in Python, or a decimal library in JavaScript.
  • Compensated summation: Kahan summation keeps a running correction of the low-order bits lost at each addition, giving much more accurate totals for long lists. Python's math.fsum does something similar.
  • Sort or group before summing when you need reproducible results, since the order changes the rounding.
  • Compare with tolerances and choose the tolerance for the problem (about 1e-9 for typical values), not blindly.
OperationTimeNotes
Add / multiply two floatsO(1)One hardware instruction
Naive sum of n floatsO(n)Error can grow with n
Kahan sum of n floatsO(n)Same cost class, far smaller error
Decimal arithmeticO(1) to O(digits)Exact for decimals but slower

Code: seeing the error and fixing it

import math
 
def yes(flag):
    return "true" if flag else "false"
 
print(0.1 + 0.2, yes(0.1 + 0.2 == 0.3))
print(yes(abs((0.1 + 0.2) - 0.3) < 1e-9))
 
total = 0.0
for _ in range(10):
    total += 0.1
print(total, yes(total == 1.0))
 
def kahan_sum(values):
    s, c = 0.0, 0.0              # c carries the lost low-order bits
    for v in values:
        y = v - c
        t = s + y
        c = (t - s) - y
        s = t
    return s
 
print(yes(kahan_sum([0.1] * 10) == 1.0))
 
cents = 10 + 20                  # money as integer cents
print(cents, "->", "$" + str(cents // 100) + "." + str(cents % 100).zfill(2))
 
big = 2.0 ** 53
print(yes(big + 1 == big), yes(math.isnan(float("nan") - float("nan"))))

Both versions print:

0.30000000000000004 false
true
0.9999999999999999 false
true
30 -> $0.30
true true

Line 1 shows the classic surprise. Line 2 shows the fix: compare within a tolerance. Line 3 shows error accumulating over ten additions. Line 4 shows that Kahan summation recovers exactly 1.0. Line 5 shows money kept in integer cents. Line 6: 2^53 + 1 is no different from 2^53, and NaN - NaN is NaN.

Why line 1 comes out that way: neither 0.1 nor 0.2 nor 0.3 is stored exactly, and the tiny errors do not cancel.

you writedouble actually stored
0.1
0.1000000000000000055511
0.2
0.2000000000000000111022
0.1 + 0.2 (result)
0.3000000000000000444089
0.3
0.2999999999999999888978
The first two doubles are slightly too big, and the sum rounds to a double that is one step above the double nearest 0.3 (the stored 0.3 is slightly too small). The two doubles differ, so == says false.
0.1
after add 1
0.2
after add 2
0.30000000000000004
after add 3
0.7999999999999999
after add 8
0.9999999999999999
after add 10
The running total while adding 0.1 ten times (line 3 of the output). Some sums land exactly on the neat value, others drift one step off, and the error shows up in the final total.

Common pitfalls

  • Using == on computed floats: compare with a tolerance, or use math.isclose(a, b, rel_tol=1e-9).
  • Float loop counters: for x = 0; x != 1; x += 0.1 never hits exactly 1. Loop on an integer and compute x = i / 10.
  • Assuming addition is associative: parallel and vectorized sums add in a different order and give slightly different answers. That is normal, so test with tolerances.
  • Trusting printed digits: printing shows the shortest text that round-trips, not the exact stored value. Decimal(0.1) in Python reveals the true value, 0.1000000000000000055511151231257827....
  • Ignoring NaN: NaN == NaN is false, and NaN poisons every calculation it touches. Check with math.isnan or Number.isNaN. In sorting, NaN breaks comparisons.
  • Large integers in JavaScript: database ids above 2^53 silently lose precision when parsed as numbers. Treat them as strings or BigInt.

Practice

Why is 0.5 exact but 0.1 not? easy

Hint

Which fractions have a denominator that is a power of two?

Show solution

0.5 = 1/2 and 0.25 = 1/4 have power-of-two denominators, so they are finite in binary. 0.1 = 1/10 has a factor of 5 in the denominator, so its binary expansion repeats forever and gets rounded. The same reason makes 1/3 impossible to write exactly in decimal.

Write an approximate-equality function medium

Hint

Absolute tolerance fails for huge numbers; relative tolerance fails near zero. Combine them.

Show solution

isclose(a, b) is true when abs(a - b) <= max(rel * max(abs(a), abs(b)), abs_tol). With rel = 1e-9 and abs_tol = 1e-12, values near 1e12 get a scaled tolerance, while values around zero still compare sensibly. This is the idea behind math.isclose.

Sum a list accurately medium

Hint

Track what each addition loses and feed it back in.

Show solution

Use Kahan summation: keep a compensation c. For each value compute y = v - c, t = s + y, c = (t - s) - y, then s = t. The correction c is the portion of y that was rounded away, subtracted from the next term. O(n) time, O(1) space. In production, prefer math.fsum in Python.

Detect a hidden 2^53 bug hard

Hint

Which integers can a double represent exactly? What happens at 2^53 + 1?

Show solution

A service parses 64-bit ids as JavaScript numbers. Ids 9007199254740993 and 9007199254740992 both become 9007199254740992, so two different records collide. Fixes: send ids as strings in JSON, parse with BigInt, or use a typed id format. Detect it with tests at the boundary (Number.MAX_SAFE_INTEGER + 2) and by checking Number.isSafeInteger on parsed values.

Interview questions

  1. Why does 0.1 + 0.2 not equal 0.3? Neither 0.1 nor 0.2 is exactly representable in binary; the nearest doubles add up to a value just above 0.3, so equality fails.
  2. How should you compare floats? With an absolute and relative tolerance, not equality.
  3. How do you store currency? Integer cents or a decimal type, never binary floats.
  4. What is the largest integer a double stores exactly? 2^53, about 9 quadrillion; beyond that, gaps appear.

Interviewers look for: a clear explanation of binary representation, tolerance-based comparison, and knowing the money and large-integer pitfalls.

Quick check

Which is the safest way to total prices like 19.99 and 5.01 in code?

Finished this lesson?

Further resources

Related lessons