[racket] Decimal rounding problem
At Thu, 29 Nov 2012 14:44:38 +0000, Greg Graham wrote:
> I am trying to report GPA calculation results to 2 decimal places, so I thought
> real->decimal-string would do the trick. However, the following behavior
> surprised me:
>
> > (real->decimal-string 3.225 2)
> "3.23"
> > (real->decimal-string 4.225 2)
> "4.22"
>
> I would like the second answer to be "4.23", which is what a student would
> expect to see if they did the calculations themselves. The documentation for
> real->decimal-string says that it first converts the argument to an exact
> number. I suspect the problem has something to do with this:
>
> > (inexact->exact 4.225)
> 4 126663739519795/562949953421312
> > (/ 126663739519795.0 562949953421312.0)
> 0.22499999999999964
>
> Is there another rounding function that would just round the floating point
> without going through the conversion to exact?
There are a couple of issues here.
First, the inexact number 4.225 is actually slightly smaller than the
exact value 4.225:
> (< 4.225 #e4.225)
#t
> (pretty-print-exact-as-decimal #t)
> (inexact->exact 4.225)
4.2249999999999996447286321199499070644378662109375
So, that's one reason to argue that "4.22" is the right answer.
Even if you use the exact number 4.225, though, you get a "2" as the
last digit:
> (real->decimal-string #e4.225 2)
"4.22"
That's because rounding in Racket is to "even" --- a different
convention than the one taught to most students, but one that it often
preferred in computing (because it often reduces accumulated error, as I
understand it).