6

Mathematically, 0.9 recurring can be shown to be equal to 1. This question however, is not about infinity, convergence, or the maths behind this.

The above assumption can be represented using doubles in C# with the following.

var oneOverNine = 1d / 9d;
var resultTimesNine = oneOverNine * 9d;

Using the code above, (resultTimesNine == 1d) evaluates to true.

When using decimals instead, the evaluation yields false, yet, my question is not about the disparate precision of double and decimal.

Since no type has infinite precision, how and why does double maintain such an equality where decimal does not? What is happening literally 'between the lines' of code above, with regards to the manner in which the oneOverNine variable is stored in memory?

Tom Bowers
  • 4,951
  • 3
  • 30
  • 43
  • I'm assuming rounding happens. For doubles towards `1.0`, for floats away from it. – millimoose Oct 17 '13 at 20:05
  • 1
    What version of C#, compiler and OS? Asking because using VS2010 on a 64-bit Win7 machine yields `true` for me for both floats and doubles, whether doing x86 or x64. – Joel Rondeau Oct 17 '13 at 20:19
  • @JoelRondeau Fair question, on my current machine (VS2010, Win7 x64), I get the same results as you, but false for decimal, so again, precision differences, but interesting for comparison. – Tom Bowers Oct 17 '13 at 20:33
  • Getting different answers from the same code on different machines reminded me of this Lippert answer: http://stackoverflow.com/a/8795656/154766 – Joel Rondeau Oct 17 '13 at 20:43

2 Answers2

11

It depends on the rounding used to get the closest representable value to 1/9. It could go either way. You can investigate the issue of representability at Rob Kennedy's useful page: http://pages.cs.wisc.edu/~rkennedy/exact-float

But don't think that somehow double is able to achieve exactness. It isn't. If you try with 2/9, 3/9 etc. you will find cases where the rounding goes the other way. The bottom line is that 1/9 is not exactly representable in binary floating point. And so rounding happens and your calculations are subject to rounding errors.

David Heffernan
  • 601,492
  • 42
  • 1,072
  • 1,490
2

What is happening literally 'between the lines' of code above, with regards to the manner in which the oneOverNine variable is stored in memory?

What you're asking about is called IEEE 754. This is the spec that C#, it's underlying .Net runtime, and most other programming platforms use to store and manipulate decimal values. This is because support for IEEE 754 is typically implemented directly at the CPU/chipset-level, making it both far more performant than an alternative implemented solely in software and far easier when building compilers, because the operations will map almost directly to specific CPU instructions.

Joel Coehoorn
  • 399,467
  • 113
  • 570
  • 794