Bug 495 : Rounding Errors in the DXF export
Last modified: 2007-02-17 12:23




Status:
RESOLVED
Resolution:
FIXED -
Priority:
P2
Severity:
normal

 

Reporter:
Stephen Williams
Assigned To:
fry

Attachment Type Created Size Actions

Description:   Opened: 2007-01-22 05:03
I have detected rounding errors in the DXF exporter. There are two problems
with the code at the moment. The first problem is that in the call to ...

writer.println(PApplet.nf(val, 0, 4));

.. in the write() function 4 decimals is not always enough especially if
the object is created small. This formatting is really not necessary (or is
it?) If I need the values to be rounded I can do it before calling vertex()
and it may increase the file size by adding .0000 to if the value is an
integer. The other problem is that since I am using a German operating
system, the numbers are formatted as...

-400,0000

instead of...

-400.000000


I suggest simply removing the PApplet.nf() call, i.e.

writer.println(val);
Additional Comment #1 From fry 2007-01-29 19:15
32-bit floats will only give you four, maybe five digits of accuracy in the
decimal place.

i'd recommend scaling the image larger.
Additional Comment #2 From Stephen Williams 2007-01-30 04:51
I am using doubles in a modified version so this doesn't effect me but the
main problem is that on German operating system the decimals will be
ignored as 100,1 will be parsed as 1 and not 100.1 this leave a rounding
error 0.1. The Germans use commas as a decimal point character and for this
reason nf(0.1, 0, 1) = 0,1 and not 0.1



Additional Comment #3 From fry 2007-02-17 12:22
that's more of a problem. i'll just leave the formatting off.