]> git.tdb.fi Git - ext/subsurface.git/commit
Do proper rounding in interpolated pressure calculations
authorLinus Torvalds <torvalds@linux-foundation.org>
Sun, 20 Nov 2011 18:27:33 +0000 (10:27 -0800)
committerLinus Torvalds <torvalds@linux-foundation.org>
Sun, 20 Nov 2011 18:27:33 +0000 (10:27 -0800)
commita643e740dc32d474cc4f0ff3f0187418463e44e1
tree17a17d070cab4f80e84ceb41231664455a375bfd
parentbb421a416dd894979e2335efea8db06b681f0dc8
Do proper rounding in interpolated pressure calculations

We do all the pressures in mbar, which has plenty of precision for
interpolated pressures - even when we then do our discrete integration
over many samples.

However, when we calculate those interpolated pressure points, we should
make sure that we round the result correctly, otherwise the consistent
rounding errors (from truncating the FP value into our integer mbar
values) will result in a final pressure that is noticeably off in ugly
ways (ie "end pressure set by hand to 750 mbar, but shown as 748").

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
profile.c