This change makes some miracle modifications to the System Five system
call support, which lets us have safe, correct, and atomic handling of
thread cancellations. It all turned out to be cheaper than anticipated
because it wasn't necessary to modify the system call veneers. We were
able to encode the cancellability of each system call into the magnums
found in libc/sysv/syscalls.sh. Since cancellations are so waq, we are
also supporting a lovely Musl Libc mask feature for raising ECANCELED.
This change includes a fix to Fetch() where an out of bounds memory read
could happen, when the reverse proxied endpoint omits the content-length
header. This caused a bunch of NUL chars to appear on TurfWar's /statusz
since it wouldn't actually overrun the buffer, and if it did it would've
been caught by MODE=asan builds.
All tests pass now under WSL2. They should pass under WSL1 too, but only
WSL2 is integrated into the test fleet right now. This change also fills
in some gaps in the error numbers.
Fixes#665
- ASAN memory morgue is now lockless
- Make C11 atomics header more portable
- Rewrote pthread keys support to be lockless
- Simplify Python's unicode table unpacking code
- Make crash report write(2) closer to being atomic
- Make it possible to strace/ftrace a single thread
- ASAN now checks nul-terminated strings fast and properly
- Windows fork() now restores TLS memory of calling thread
- Invent iso8601us() for faster timestamps
- Improve --strace descriptions of sigset_t
- Rebuild the Landlock Make bootstrap binary
- Introduce MODE=sysv for non-Windows builds
- Permit OFD fcntl() locks under pledge(flock)
- redbean can now protect your kernel from ddos
- Have vfork() fallback to sys_fork() not fork()
- Change kmalloc() to not die when out of memory
- Improve documentation for some termios functions
- Rewrite putenv() and friends to conform to POSIX
- Fix linenoise + strace verbosity issue on Windows
- Fix regressions in our ability to show backtraces
- Change redbean SetHeader() to no-op if value is nil
- Improve fcntl() so SQLite locks work in non-WAL mode
- Remove some unnecessary work during fork() on Windows
- Create redbean-based SSL reverse proxy for IPv4 TurfWar
- Fix ape/apeinstall.sh warning when using non-bash shells
- Add ProgramTrustedIp(), and IsTrustedIp() APIs to redbean
- Support $PWD, $UID, $GID, and $EUID in command interpreter
- Introduce experimental JTqFpD APE prefix for non-Windows builds
- Invent blackhole daemon for firewalling IP addresses via UNIX named socket
- Add ProgramTokenBucket(), AcquireToken(), and CountTokens() APIs to redbean
If threads are being used, then fork() will now acquire and release and
runtime locks so that fork() may be safely used from threads. This also
makes vfork() thread safe, because pthread mutexes will do nothing when
the process is a child of vfork(). More torture tests have been written
to confirm this all works like a charm. Additionally:
- Invent hexpcpy() api
- Rename nsync_malloc_() to kmalloc()
- Complete posix named semaphore implementation
- Make pthread_create() asynchronous signal safe
- Add rm, rmdir, and touch to command interpreter builtins
- Invent sigisprecious() and modify sigset functions to use it
- Add unit tests for posix_spawn() attributes and fix its bugs
One unresolved problem is the reclaiming of *NSYNC waiter memory in the
forked child processes, within apps which have threads waiting on locks
This lets our system() and popen() commands function sort of like
BusyBox and ToyBox. By default the Cosmopolitan Shell is lightweight.
But if you use STATIC_YOINK then you can pull the individual commands
you want into the linkage, and they'll be included in a single binary.
For example the demo binary embeds `tr` and `sed` and ends up ~140kb.
- SQLite file locking now works on Windows
- SQLite will now use fdatasync() on non-Apple platforms
- Fix Ctrl-C handler on Windows to not crash with TLS
- Signals now work in multithreaded apps on Windows
- fcntl() will now accurately report EINVAL errors
- fcntl() now has excellent --strace logging
- Token bucket replenish now go 100x faster
- *NSYNC cancellations now work on Windows
- Support closefrom() on NetBSD
- use PAGE_RSRV bit (originally only for blinkenlights),
rather than PAGE_V bit, to indicate that a virtual address
page has been reserved — this should allow a program to
create & reserve inaccessible "guard pages"
- mark page table entries for non-code pages with PAGE_XD bit,
which should be supported on (circa) post-2004 x86-64 CPUs
The cosmopolitan command interpreter now has 13 builtin commands,
variable support, support for ; / && / || syntax, asynchronous support,
and plenty of unit tests with bug fixes.
This change fixes a bug in posix_spawn() with null envp arg. strace
logging now uses atomic writes for scatter functions. Breaking change
renaming GetCpuCount() to _getcpucount(). TurfWar is now updated to use
the new token bucket algorithm. WIN32 affinity masks now inherit across
fork() and execve().
libc/intrin/interrupts.S should not be linked in unless
an IDT (or TSS) is explicitly requested somewhere (i.e.
it should probably not be a mandatory module).
This change addresses various open source compatibility issues, so that
we pass 313/411 of the tests in https://github.com/jart/libc-test where
earlier today we were passing about 30/411 of them, due to header toil.
Please note that Glibc only passes 341/411 so 313 today is pretty good!
- Make the conformance of libc/isystem/ headers nearly perfect
- Import more of the remaining math library routines from Musl
- Fix inconsistencies with type signatures of calls like umask
- Write tests for getpriority/setpriority which work great now
- conform to `struct sockaddr *` on remaining socket functions
- Import a bunch of uninteresting stdlib functions e.g. rand48
- Introduce readdir_r, scandir, pthread_kill, sigsetjmp, etc..
Follow the instructions in our `tool/scripts/cosmocc` toolchain to run
these tests yourself. You use `make CC=cosmocc` on the test repository
- Change IDT code so kprintf() isn't mandatory dependency
- Document current intentions around pthread_cancel()
- Make _npassert() an _unassert() in MODE=tiny
- fix rare thread exit race condition on openbsd
- pthread_getattr_np() now supplies detached status
- child threads may now pthread_join() the main thread
- introduce sigandset(), sigorset(), and sigisemptyset()
- introduce pthread_cleanup_push() and pthread_cleanup_pop()
- correctly check that virtual region we want to use is
unmapped, rather than accidentally clobbering existing pages
- avoid placing mmap'd memory at null virtual address
* [metal] Refactoring: separate out sys_writev_vga() and _vga_init() modules
* [metal] Read VGA info from BDA before long mode entry, not after
If using a pre-existing VGA text console, the VGA initialization
code now retrieves the cursor position & character height from
the BIOS data area while still in real mode — rather than
reading from the BIOS data area only after entering long mode.
(This should help make the code more correct, if Cosmopolitan
were to support UEFI graphics output in the future. If the
program were booted via UEFI, then the long mode IsMetal()
code would still be activated, but the BIOS data area might
not have been initialized in that case.)
This change also means that there are now a few more fields
in the `struct mman`.
* [metal] VGA console can now show "screen of death" upon a crash
There is now a new function _klog_vga(), which can be called
by kprintf() to output system messages — e.g. information about
CPU exceptions — on the VGA screen.
* [metal] CPU exception handler now dumps cr2 value
* [metal] Add demo of program crash reporting w/ bare metal VGA TTY
* [metal] Reduce size of "screen of death" code
You can now do things like implement mutexes using futexes in your
redbean lua code. This provides the fastest possible inter-process
communication for your production systems when SQLite alone as ipc
or things like pipes aren't sufficient.
* Test output of colors for VGA graphics modes in examples/vga.c
* [metal] Character output in VGA graphics modes is mostly working
* [metal] Mention magic key to switch video mode, at bootup
