• Eximius@lemmy.world
    link
    fedilink
    English
    arrow-up
    11
    ·
    6 months ago

    Did nobody read the manual?

    IEEE 754 double precision: The 53-bit significand precision gives from 15 to 17 significant decimal digits precision.

    • Blackmist
      link
      fedilink
      English
      arrow-up
      2
      ·
      6 months ago

      I’m not sure where the 17 comes from. It’s 15.

      • hades@lemm.ee
        link
        fedilink
        English
        arrow-up
        4
        ·
        6 months ago

        The “15 to 17” part is worded somewhat confusingly, but it’s not wrong.

        The number of bits contained in a double is equivalent to ~15.95 decimal digits. If you want to store exactly a decimal number with a fixed number of significant digits, floor(15.95) = 15 digits is the most you can hope for. However, if you want to store exactly a double by writing it out as a decimal number, you need 17 digits.