FauxPseudo @lemmy.world to Skeptic@lemmy.worldEnglish · 1 year ago"Do your own research" they say.lemmy.worldimagemessage-square100fedilinkarrow-up1704arrow-down127
arrow-up1677arrow-down1image"Do your own research" they say.lemmy.worldFauxPseudo @lemmy.world to Skeptic@lemmy.worldEnglish · 1 year agomessage-square100fedilink
minus-squarecynar@lemmy.worldlinkfedilinkEnglisharrow-up16·1 year agoFor those confused 2.25+2.25=4.5 rounds to 2+2=5 2.5+2.5=5 truncates to 2+2=5 Both can crop up in programming, depending on the situation.
minus-squareEcho DotlinkfedilinkEnglisharrow-up4·1 year ago2.25 + 2.25 = 4.5 If you add two floats together then the output is a float, if you add an int and a float together the output is a float. Computers will always perform the calculation as is, unless you explicitly tell them to perform a rounding operation.
minus-squarecynar@lemmy.worldlinkfedilinkEnglisharrow-up6·1 year agoHowever, if you stuff them into an int at the last minute, you can get that effect. Under the hood, it’s floats. On the output, it’s ints. It’s obvious and silly with small examples. The problem can creep in when you are using larger libraries or frameworks.
minus-squareafraid_of_zombies@lemmy.worldlinkfedilinkEnglisharrow-up1arrow-down1·1 year agoA few months back I had a floating point that had a single 1 like 16 digits past the decimal place and I couldn’t get rid of it.
For those confused
2.25+2.25=4.5 rounds to 2+2=5
2.5+2.5=5 truncates to 2+2=5
Both can crop up in programming, depending on the situation.
2.25 + 2.25 = 4.5
If you add two floats together then the output is a float, if you add an int and a float together the output is a float. Computers will always perform the calculation as is, unless you explicitly tell them to perform a rounding operation.
However, if you stuff them into an int at the last minute, you can get that effect.
Under the hood, it’s floats. On the output, it’s ints.
It’s obvious and silly with small examples. The problem can creep in when you are using larger libraries or frameworks.
A few months back I had a floating point that had a single 1 like 16 digits past the decimal place and I couldn’t get rid of it.