FAQ
Cover
\
Build
\
Source
\
Bugs
\
Reference
\
Libraries
\
Tools
The bugs database has moved
here
.
Bug 1059 : Multiplying floats returns erroneous values.
Last modified: 2008-11-24 20:34
P
roject:
processing
trash
Version:
unspecified
Co
m
ponent:
android
book
core
libraries
pde
reference
tools
web
Status:
RESOLVED
Resolution:
INVALID -
Pr
i
ority:
P2
Severity:
normal
Platform
All
O
S:
All
Windows
Mac OS
Linux
Other
Reporter:
nullkraft
Assigned To:
fry
Attachment
Type
Created
Size
Actions
Description
: Opened: 2008-11-24 20:09
I'm using Processing 0157 for Linux, Intel X86, and I have NOT linked to
another Java. The problem I see is with the following code:
float digits = 1132.0 * 0.0001;
text("Digits: " + digits, 50, 50);
Which displays :
Digits: 0.113199994
which is incorrect, but when I invert 0.0001 and divide by 10000 instead I
get the following :
1132.0 / 10000 = 0.1132
which is correct. They are both technically the same operation and should
have the same results. I tested other values from 1119.0 through 1132.0
with some values returning incorrect results and others returning the
correct results. I also tried :
1132 * 0.8356 = 945.89923 which should have been :
945.8992
Additional Comment
#1 From fry 2008-11-24 20:15
floats are 32 bit floating point numbers. they are not perfect, they are
approximations. they offer around 4 to 6 digits of accuracy depending on
the case in which they're used, so you're going to get poor results if you
multiply a smallish number by 0.0001.
use a double if you need better numerical accuracy, then convert to float
before drawing since none of the APIs use doubles.
Additional Comment
#2 From fry 2008-11-24 20:16
***
Bug 1058
has been marked as a duplicate of this bug. ***
Additional Comment
#3 From nullkraft 2008-11-24 20:34
Actually, I thought about that too. But after testing larger/smaller
sequential numbers and getting the correct results I came to the conclusion
that there was something amiss. I don't know that it's even in Processing,
maybe it's in Java. I just thought I'd post it in case there's a real
problem. The fact that I can get the correct results by inverting and
multiplying is more than sufficient for my needs.