Back
Login
Register
Login
Register
Millisecond Forums
Home
»
Millisecond Forums
»
Inquisit 3
»
RE: Floting point problem
Floting point problem
Like
96
Floting point problem
View
Flat Ascending
Flat Descending
Threaded
Options
Subscribe to topic
Print This Topic
RSS Feed
Goto Topics Forum
Author
Message
mrg4
mrg4
posted 10 Years Ago
ANSWER
Topic Details
Share Topic
Group: Forum Members
Posts: 19,
Visits: 218
Hello,
we have a task where the participant earns points according to which key he presses.
In one case the participant receives 1 point, in the other case he gets 0.1 points. After each trial a feedback is given, saying:
"You received 0.1 / 1 points. Your total score is: XY.XY"
We therefore defined one value element for the gain in the recent trial and one counting the total score.
In case the participant receives 1 point, everything works perfecty fine, but due to the fact that the computer works with floating point, after the first time 0.1 points are added to the score, our feedback looks like e.g.:
"You received 0.10000000000000001 points. Your total score is: 5.10000000000000001"
We already tried to solve the problem by using rounding, but none of our attempts worked. We now use 0.5 / 5 points instead of 0.1 / 1, and it works perfectly.
This is thought to be a temporary solution, so we're hoping anyone is able to help us.
Thank you very much in advance.
Best regards
Tags
binary
floating
rounding
Like
96
Dave
Dave
posted 10 Years Ago
ANSWER
Post Details
Share Post
Group: Administrators
Posts: 13K,
Visits: 104K
This is a completely natural consequence of how computers represent real numbers (cf.
http://www.millisecond.com/forums/FindPost7984.aspx
). Your best bet is to use the format() function to limit the output of digits after the dot.
-> format("%.2f", values.myvalue)
Like
105
GO
Merge Selected
Merge into selected topic...
Merge into merge target...
Merge into a specific topic ID...
Open Merge
Like
96
Post Quoted Reply
Reading This Topic
Login
Login
Remember Me
Reset Password
Resend Validation Email
Login
Facebook
Explore
Messages
Mentions
Search