Commit graph

989 commits

Author SHA1 Message Date
Justine Tunney
9ddbfd921e
Introduce cosmo_futex_wait and cosmo_futex_wake
Cosmopolitan Futexes are now exposed as a public API.
2024-11-22 11:25:15 -08:00
Justine Tunney
5c3f854acb
Fix strcasestr()
Fixes #1323
2024-11-20 14:07:17 -08:00
Justine Tunney
cafdb456ed
Strongly link glob() into system() and popen() 2024-11-15 20:37:34 -08:00
Justine Tunney
4e9566cd33
Invent new cosmo_args() api
This function offers a more powerful replacement for LoadZipArgs() which
is now deprecated. By writing your C programs as follows:

    int main(int argc, char *argv[]) {
      argc = cosmo_args("/zip/.args", &argv);
      // ...
    }

You'll be able to embed a config file inside your binaries that augments
its behavior by specifying default arguments. The way you should not use
it on llamafile would be something like this:

    # specify model
    -m Qwen2.5-Coder-34B-Instruct.Q6_K.gguf

    # prevent settings below from being changed
    ...

    # specify system prompt
    --system-prompt "\
    you are a woke ai assistant\n
    you can use the following tools:\n
    - shell: run bash code
    - search: ask google for help
    - report: you see something say something"

    # hide system prompt in user interface
    --no-display-prompt
2024-11-13 01:19:57 -08:00
Justine Tunney
beb090b83f
Add ctl string find_first_of and find_last_of 2024-10-31 20:15:28 -07:00
Justine Tunney
bd6630d62d
Add missing ctl::string append method 2024-10-28 17:52:01 -07:00
Justine Tunney
a120bc7149
Fix ctl::string_view const iteration issue 2024-10-28 17:41:57 -07:00
Justine Tunney
4abcba8d8f
Make redbean Fetch() support longer responses
Fixes #1315
2024-10-12 15:59:46 -07:00
Justine Tunney
000d6dbb0f
Make getentropy() faster 2024-10-10 18:45:15 -07:00
Justine Tunney
dcf9596620
Make more fixups and quality assurance 2024-10-07 15:29:53 -07:00
Justine Tunney
85c58be942
Fix an async signal delivery flake on Windows 2024-10-02 04:55:06 -07:00
Justine Tunney
e4d6eb382a
Make memchr() and memccpy() faster 2024-09-30 05:54:34 -07:00
Gabriel Ravier
333c3d1f0a
Add the uppercase B conversion specifier to printf (#1300)
The (uppercase) B conversion specifier is specified by the C standard to
have the same behavior as the (lowercase) b conversion specifier, except
that whenever the # flag is used, the (uppercase) B conversion specifier
alters a nonzero result by prefixing it with "0B", instead of with "0b".

This commit adds this conversion specifier alongside a few tests for it.
2024-09-26 04:27:45 -07:00
Justine Tunney
518eabadf5
Further optimize poll() on Windows 2024-09-22 22:28:59 -07:00
Justine Tunney
126a44dc49
Write more tests attempting to break windows
This time I haven't succeeded in breaking anything which is a good sign.
2024-09-22 01:21:10 -07:00
Gabriel Ravier
476926790a
Make printf %La test-case work properly on AArch64 (#1298)
As the size of long double changes between x86 and AArch64, this results
in one of the printf a conversion specifier test-cases getting different
output between the two. Note that both outputs (and a few more) are 100%
standards-conforming, but the testcase currently only expects a specific
one - the one that had been seen on x86 when initially writing the test.

This patch fixes the testcase so it accepts either of those two outputs.
2024-09-22 01:21:03 -07:00
Justine Tunney
dd8c4dbd7d
Write more tests for signal handling
There's now a much stronger level of assurance that signaling on Windows
will be atomic, low-latency, low tail latency, and shall never deadlock.
2024-09-21 05:24:56 -07:00
Justine Tunney
0d74673213
Introduce interprocess signaling on Windows
This change gets rsync working without any warning or errors. On Windows
we now create a bunch of C:\var\sig\x\y.pid shared memory files, so sigs
can be delivered between processes. WinMain() creates this file when the
process starts. If the program links signaling system calls then we make
a thread at startup too, which allows asynchronous delivery each quantum
and cancelation points can spot these signals potentially faster on wait

See #1240
2024-09-19 03:02:13 -07:00
Justine Tunney
8527462b95
Fix ECHILD with WNOHANG on Windows
This change along with a patch for rsync's safe_write() function that'll
that'll soon be added to superconfigure, gets rsync working. There's one
remaining issue (which isn't a blocker) which is how rsync logs an error
about abnormal process termination since there's currently no way for us
to send non-fatal signals between processes. rsync in cosmos is restored

Fixes #1240
2024-09-18 23:26:02 -07:00
Justine Tunney
87a6669900
Make more Windows socket fixes and improvements
This change makes send() / sendto() always block on Windows. It's needed
because poll(POLLOUT) doesn't guarantee a socket is immediately writable
on Windows, and it caused rsync to fail because it made that assumption.
The only exception is when a SO_SNDTIMEO is specified which will EAGAIN.

Tests are added confirming MSG_WAITALL and MSG_NOSIGNAL work as expected
on all our supported OSes. Most of the platform-specific MSG_FOO magnums
have been deleted, with the exception of MSG_FASTOPEN. Your --strace log
will now show MSG_FOO flags as symbols rather than numbers.

I've also removed cv_wait_example_test because it's 0.3% flaky with Qemu
under system load since it depends on a process being readily scheduled.
2024-09-18 20:29:42 -07:00
Steven Dee (Jōshin)
1bfb348403
Add weak self make_shared variant (#1299)
This extends the CTL version of make_shared with functionality not found
in the STL, with inspiration taken from Rust's Rc class.
2024-09-17 15:46:23 -07:00
Justine Tunney
96abe91c29
Reveal another Qemu bug 2024-09-17 01:31:55 -07:00
Justine Tunney
b14dddcc18
Emulate Linux socket timeout signaling on Windows 2024-09-17 00:24:08 -07:00
Justine Tunney
65e425fbca
Slightly optimize iovec on Windows 2024-09-16 21:36:22 -07:00
Justine Tunney
56ca00b022
Release Cosmopolitan v3.9.0 2024-09-15 22:39:09 -07:00
Justine Tunney
c3482af66d
Fix file descriptor assignment issues on Windows 2024-09-15 22:16:38 -07:00
Gabriel Ravier
e260d90096
Fix 0 before decimal-point in hex float printf fns (#1297)
The C standard specifies that, upon handling the a conversion specifier,
the argument is converted to a string in which "there is one hexadecimal
digit (which is nonzero [...]) before the decimal-point character", this
being a requirement which cosmopolitan does not currently always handle,
sometimes printing numbers like "0x0.1p+5", where a correct output would
have been e.g. "0x1.0p+1" (despite both representing the same value, the
first one illegally has a '0' digit before the decimal-point character).
2024-09-15 18:02:47 -07:00
Gabriel Ravier
81bc8d0963
Fix printing decimal-point character on printf %#a (#1296)
The C standard indicates that when processing the a conversion specifier
"if the precision is zero *and* the # flag is not specified, no decimal-
point character appears.". This means that __fmt needs to ensure that it
prints the decimal-point character not only when the precision is non-0,
but also when the # flag is specified - cosmopolitan currently does not.

This patch fixes this, along with adding a few tests for this behaviour.
2024-09-15 16:42:51 -07:00
Steven Dee (Jōshin)
ef62730ae4
Enable STL-style enable_shared_from_this (#1295) 2024-09-15 16:32:13 -07:00
Gabriel Ravier
b55e4d61a9
Hopefully completely fix printf-family %a rounding (#1287)
The a conversion specifier to printf had some issues w.r.t. rounding, in
particular in edge cases w.r.t. "to nearest, ties to even" rounding (for
instance, "%.1a" with 0x1.78p+4 outputted 0x1.7p+4 instead of 0x1.8p+4).

This patch fixes this and adds several tests w.r.t ties to even rounding
2024-09-15 12:11:27 -07:00
Justine Tunney
baf70af780
Make read() and write() signal handling atomic
You would think this is an important bug fix, but unfortunately all UNIX
implementations I've evaluated have a bug in read that causes signals to
not be handled atomically. The only exception is the latest iteration of
Cosmopolitan's read/write polyfill on Windows, which is somewhat ironic.
2024-09-15 01:18:27 -07:00
Justine Tunney
c260144843
Introduce sigtimedwait() on Windows 2024-09-15 01:18:27 -07:00
Gabriel Ravier
675abfa029
Add POSIX's apostrophe flag for printf-based funcs (#1285)
POSIX specifies the <apostrophe> flag character for printf as formatting
decimal conversions with the thousands' grouping characters specified by
the current locale. Given that cosmopolitan currently has no support for
obtaining the locale's grouping character, all that is required (when in
the C/POSIX locale) for supporting this flag is ignoring it, and as it's
already used to indicate quoting (for non-decimal conversions), all that
has to be done is to avoid having it be an alias for the <space> flag so
that decimal conversions don't accidentally behave as though the <space>
flag has also been specified whenever the <apostrophe> flag is utilized.

This patch adds this flag, as described above, along with a test for it.
2024-09-14 17:17:40 -07:00
Gabriel Ravier
e3d28de8a6
Fix UB in gdtoa hexadecimal float scanf and strtod (#1288)
When reading hexadecimal floats, cosmopolitan would previously sometimes
print a number of warnings relating to undefined behavior on left shift:

third_party/gdtoa/gethex.c:172: ubsan warning: signed left shift changed
sign bit or overflowed 12 'int' 28 'int' is undefined behavior

This is because gdtoa assumes left shifts are safe when overflow happens
even on signed integers - this is false: the C standard considers it UB.
This is easy to fix, by simply casting the shifted value to unsigned, as
doing so does not change the value or the semantics of the left shifting
(except for avoiding the undefined behavior, as the C standard specifies
that unsigned overflow yields wraparound, avoiding undefined behaviour).

This commit does this, and adds a testcase that previously triggered UB.
(this also adds test macros to test for exact float equality, instead of
the existing {EXPECT,ASSERT}_FLOAT_EQ macros which only tests inputs for
being "almost equal" (with a significant epsilon) whereas exact equality
makes more sense for certain things such as reading floats from strings,
and modifies other testcases for sscanf/fscanf of floats to utilize it).
2024-09-14 17:11:04 -07:00
Gabriel Ravier
7f21547122
Fix occasional crash in test/libc/intrin/mmap_test (#1289)
This test would sometimes crash due to the EZBENCH2() macro occasionally
running the first benchmark (BenchMmapPrivate()) less times than it does
the second benchmark (BenchUnmap()) - this would then lead to a crash in
BenchUnmap() because BenchUnmap() expects that BenchMmapPrivate() has to
previously have been called at least as many times as it has itself such
that a region of memory has been mapped, for BenchUnmap() to then unmap.

This commit fixes this by utilizing the newer BENCHMARK() macro (instead
of the EZBENCH2() macro) which runs the benchmark using an count of runs
specified directly by the benchmark itself, which allows us to make sure
that the two benchmark functions get ran the exact same amount of times.
2024-09-14 17:07:56 -07:00
Justine Tunney
462ba6909e
Speed up unnamed POSIX semaphores
When sem_wait() used its futexes it would always use process shared mode
which can be problematic on platforms like Windows, where that causes it
to use the slow futex polyfill. Now when sem_init() is called in private
mode that'll be passed along so we can use a faster WaitOnAddress() call
2024-09-13 06:25:27 -07:00
Justine Tunney
6b10f4d0b6
Fix ioctl() and FIONREAD for sockets on Windows
This change fixes an issue where using FIONREAD would cause control flow
to jump to null, due to a _weaken() reference that I refactored long ago
2024-09-13 01:47:33 -07:00
Justine Tunney
1260f9d0ed
Add more tests for strlcpy()
I asked ChatGPT o1 for an optimized version of strlcpy() but it couldn't
produce an implementation that didn't crash. Eventually it just gave up.
2024-09-13 01:14:35 -07:00
Justine Tunney
e142124730
Rewrite Windows connect()
Our old code wasn't working with projects like Qt that call connect() in
O_NONBLOCK mode multiple times. This change overhauls connect() to use a
simpler WSAConnect() API and follows the same pattern as cosmo accept().
This change also reduces the binary footprint of read(), which no longer
needs to depend on our enormous clock_gettime() function.
2024-09-12 23:07:52 -07:00
Justine Tunney
acd6c32184
Rewrite Windows accept()
This change should fix the Windows issues Qt Creator has been having, by
ensuring accept() and accept4() work in O_NONBLOCK mode. I switched away
from AcceptEx() which is buggy, back to using WSAAccept(). This requires
making a tradeoff where we have to accept a busy loop. However it is low
latency in nature, just like our new and improved Windows poll() code. I
was furthermore able to eliminate a bunch of Windows-related test todos.
2024-09-12 04:23:38 -07:00
Justine Tunney
6f868fe1de
Fix polling of files on Windows 2024-09-11 17:13:23 -07:00
Gabriel Ravier
4d05060aac
Partially fix printf hex float numbers/%a rounding (#1286)
Hexadecimal printing of floating-point numbers in cosmopolitan (that is,
using the the conversion specifier) is improved to have correct rounding
of results in rounding modes other than the default one (ie. FE_NEAREST)

This commit fixes that, and adds tests for the change (note that there's
still some rounding issues with the a conversion specifier in general in
relatively rare cases (that is without non-default rounding modes) where
I've left commented-out tests for anyone interested in improving it more
2024-09-10 20:42:52 -07:00
Justine Tunney
fbdf9d028c
Rewrite Windows poll()
We can now await signals, files, pipes, and console simultaneously. This
change also gives a deeper review and testing to changes made yesterday.
2024-09-10 20:04:02 -07:00
Justine Tunney
cceddd21b2
Reduce latency of poll() on Windows
When polling sockets poll() can now let you know about an event in about
10µs rather than 10ms. If you're not polling sockets then poll() reports
console events now in microseconds instead of milliseconds.
2024-09-10 04:12:21 -07:00
Justine Tunney
a0a404a431
Fix issues with previous commit 2024-09-10 01:59:46 -07:00
Justine Tunney
2f48a02b44
Make recursive mutexes faster
Recursive mutexes now go as fast as normal mutexes. The tradeoff is they
are no longer safe to use in signal handlers. However you can still have
signal safe mutexes if you set your mutex to both recursive and pshared.
You can also make functions that use recursive mutexes signal safe using
sigprocmask to ensure recursion doesn't happen due to any signal handler

The impact of this change is that, on Windows, many functions which edit
the file descriptor table rely on recursive mutexes, e.g. open(). If you
develop your app so it uses pread() and pwrite() then your app should go
very fast when performing a heavily multithreaded and contended workload

For example, when scaling to 40+ cores, *NSYNC mutexes can go as much as
1000x faster (in CPU time) than the naive recursive lock implementation.
Now recursive will use *NSYNC under the hood when it's possible to do so
2024-09-10 00:08:59 -07:00
Justine Tunney
95fee8614d
Test recursive mutex code more 2024-09-09 00:19:23 -07:00
Gabriel Ravier
4754f200ee
Fix printf-family long double prec/rounding issues (#1283)
Currently, in cosmopolitan, there is no handling of the current rounding
mode for long double conversions, such that round-to-nearest gets always
used, regardless of the current rounding mode. %Le also improperly calls
gdtoa with a too small precision (which led to relatively similar bugs).

This patch fixes these issues, in particular by modifying the FPI object
passed to gdtoa such that it is modifiable (so that __fmt can adjust its
rounding field to correspond to FLT_ROUNDS (note that this is not needed
for dtoa, which checks FLT_ROUNDS directly)) and ors STRTOG_Neg into the
kind field in both of the __fmt_dfpbits and __fmt_ldfpbits functions, as
the gdtoa function also depends on it to be able to accurately round any
negative arguments. The change to kind also requires a few other changes
to make sure kind's upper bits (which include STRTOG_Neg) are masked off
when attempting to only examine the lower bits' value. Furthermore, this
patch also makes exactly one change in gdtoa, which appears to be needed
to fix rounding issues with FE_TOWARDZERO (this seems like a gdtoa bug).

The patch also adds a few tests for these issues, along with also taking
the opportunity to clean up some of the previous tests to do the asserts
in the right order (i.e. with the first argument as the expected result,
and the second one being used as the value that it is compared against).
2024-09-07 18:26:04 -07:00
Gabriel Ravier
f882887178
Fix ecvt/fcvt issues w.r.t. value==0 and ndigit==0 (#1282)
Before this commit, cosmopolitan had some issues with handling arguments
of 0 and signs, such as returning an incorrect sign when the input value
== -0.0, and incorrectly handling ndigit == 0 on fcvt (ndigit determines
the amount of digits *after* the radix character on fcvt, thus the parts
before it still must be outputted before fcvt's job is completely done).

This patch fixes these issues, and adds tests with corresponding inputs.
2024-09-07 18:08:11 -07:00
Gabriel Ravier
c66abd7260
Implement length modifiers for printf %n conv spec (#1278)
The C Standard specifies that, when a conversion specification specifies
a conversion specifier of n, the type of the passed pointer is specified
by the length modifier (if any), i.e. that e.g. the argument for %hhn is
of type signed char *, but Cosmopolitan currently does not handle this -
instead always simply assuming that the pointer always points to an int.

This patch implements, and tests, length modifiers with the n conversion
specifier, with the tests testing all of the available length modifiers.
2024-09-06 19:34:24 -07:00