Skip to content
  • James Bottomley's avatar
    string_helpers: fix precision loss for some inputs · 564b026f
    James Bottomley authored
    It was noticed that we lose precision in the final calculation for some
    inputs.  The most egregious example is size=3000 blk_size=1900 in units
    of 10 should yield 5.70 MB but in fact yields 3.00 MB (oops).
    
    This is because the current algorithm doesn't correctly account for
    all the remainders in the logarithms.  Fix this by doing a correct
    calculation in the remainders based on napier's algorithm.
    
    Additionally, now we have the correct result, we have to account for
    arithmetic rounding because we're printing 3 digits of precision.  This
    means that if the fourth digit is five or greater, we have to round up,
    so add a section to ensure correct rounding.  Finally account for all
    possible inputs correctly, including zero for block size.
    
    Fixes: b9f28d86
    
    
    Signed-off-by: default avatarJames Bottomley <JBottomley@Odin.com>
    Reported-by: default avatarVitaly Kuznetsov <vkuznets@redhat.com>
    Cc: <stable@vger.kernel.org>	[delay until after 4.4 release]
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    564b026f