 |
JDVyska
Beginner
Joined: 18 Oct 2002
Posts: 13
Location: USA
|
 Posted: Tue Sep 23, 2003 7:33 pm
funky floating
|
I'm in need to use tenths for the first time, so I began to play with floating point numbers... However, when doing division, I get some *bizarre* results. To demo the prob, I used this command:
#VAR TEST {%random(2,18)};#SHOW @TEST "|" [@TEST / 10.0]
Here's some sample results:
9 | 0.899999976158142
5 | 0.5
2 | 0.200000002980232
15 | 1.5
16 | 1.60000002384186
7 | 0.699999988079071
17 | 1.70000004768372
10 | 1
What in the heck is that all about? And I can't work around it, since %round only rounds to integers.
Jeremy |
|
|
|
|
JDVyska
Beginner
Joined: 18 Oct 2002
Posts: 13
Location: USA
|
Posted: Tue Sep 23, 2003 8:08 pm
|
OK, so my cryptology friend explained that for floating point, that is correct, which wasn't helpful though correct.
But, I realized that the rounding functionality I was looking for was rolled into the %format command. Heh. Since that wasn't clear through searching through the forums, I don't feel too bad for these two posts.
Jeremy |
|
|
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
