mirrors/cosmopolitan
1
0
Fork 0
mirror of https://github.com/jart/cosmopolitan.git synced 2025-07-14 23:09:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Fix printf-family long double prec/rounding issues (#1283)

Browse source 
Currently, in cosmopolitan, there is no handling of the current rounding
mode for long double conversions, such that round-to-nearest gets always
used, regardless of the current rounding mode. %Le also improperly calls
gdtoa with a too small precision (which led to relatively similar bugs).

This patch fixes these issues, in particular by modifying the FPI object
passed to gdtoa such that it is modifiable (so that __fmt can adjust its
rounding field to correspond to FLT_ROUNDS (note that this is not needed
for dtoa, which checks FLT_ROUNDS directly)) and ors STRTOG_Neg into the
kind field in both of the __fmt_dfpbits and __fmt_ldfpbits functions, as
the gdtoa function also depends on it to be able to accurately round any
negative arguments. The change to kind also requires a few other changes
to make sure kind's upper bits (which include STRTOG_Neg) are masked off
when attempting to only examine the lower bits' value. Furthermore, this
patch also makes exactly one change in gdtoa, which appears to be needed
to fix rounding issues with FE_TOWARDZERO (this seems like a gdtoa bug).

The patch also adds a few tests for these issues, along with also taking
the opportunity to clean up some of the previous tests to do the asserts
in the right order (i.e. with the first argument as the expected result,
and the second one being used as the value that it is compared against).
This commit is contained in:
Gabriel Ravier 2024-09-08 03:26:04 +02:00 • committed by GitHub
parent f882887178
commit 4754f200ee
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 110 additions and 60 deletions
Download patch file Download diff file Expand all files Collapse all files

4
third_party/gdtoa/gdtoa.c vendored
View file

@ -293,7 +293,9 @@ gdtoa(const FPI *fpi, int be, ULong *bits, int *kindp, int mode, int ndigits, in
else if ( (rdir = fpi->rounding - 1) !=0) {
if (rdir < 0)
rdir = 2;
if (kind & STRTOG_Neg)
// note that we check for fpi->rounding == 0 as in that case we
// must *always* round towards 0, i.e. downwards, with rdir = 2
if (kind & STRTOG_Neg && fpi->rounding != 0)
rdir = 3 - rdir;
}
/* Now rdir = 0 ==> round near, 1 ==> round up, 2 ==> round down. */