Ruby Returns Incorrect Integer Or Decimal Places When Dividing Two Numbers

Posted by Weston Ganger on August 11, 2016

I was noticing a weird issue in ruby where I had a division statement that was returning an integer when it should have returned a decimal value. Check this out.

12 / 5      #=> 2 <-- This is the wrong value, it should be 2.4

Turns out this is because its doing integer division so it returns an integer. To make it return a decimal value you should add a decimal point to one of the number.

12.0 / 5    #=> 2.4
12 / 5.0    #=> 2.4
12.to_f / 5 #=> 2.4

This still may not product the correct value sometimes. There are odd cases where floats will return the wrong values.

0.1 + 0.2    #=> 0.30000000000000004
0.05 + 0.93  #=> 0.9800000000000001

Wait a second, what? Yes this is extremely basic math and its failing. This is why they always say use Decimals not Floats when doing accurate calculations. So here is how to get the correct value.

number = 0.1 + BigDecimal('0.2') # Note the value passed to BigDecimal is a string, this is important to maintain accuracy
number.to_s('F')  #=> 0.3     '# F' specifies using floating point notation instead of engineering

Related External Links:

Posted in Rails and Tagged with math 

Need help on your next project or application?

I specialize in Ruby-on-Rails, AngularJS, Javascript, Bootstrap, and Hybrid Mobile Apps with Cordova & Ionic.

Contact Me

Recommended Posts