- Invent iso8601us() for faster timestamps
- Improve --strace descriptions of sigset_t
- Rebuild the Landlock Make bootstrap binary
- Introduce MODE=sysv for non-Windows builds
- Permit OFD fcntl() locks under pledge(flock)
- redbean can now protect your kernel from ddos
- Have vfork() fallback to sys_fork() not fork()
- Change kmalloc() to not die when out of memory
- Improve documentation for some termios functions
- Rewrite putenv() and friends to conform to POSIX
- Fix linenoise + strace verbosity issue on Windows
- Fix regressions in our ability to show backtraces
- Change redbean SetHeader() to no-op if value is nil
- Improve fcntl() so SQLite locks work in non-WAL mode
- Remove some unnecessary work during fork() on Windows
- Create redbean-based SSL reverse proxy for IPv4 TurfWar
- Fix ape/apeinstall.sh warning when using non-bash shells
- Add ProgramTrustedIp(), and IsTrustedIp() APIs to redbean
- Support $PWD, $UID, $GID, and $EUID in command interpreter
- Introduce experimental JTqFpD APE prefix for non-Windows builds
- Invent blackhole daemon for firewalling IP addresses via UNIX named socket
- Add ProgramTokenBucket(), AcquireToken(), and CountTokens() APIs to redbean
If threads are being used, then fork() will now acquire and release and
runtime locks so that fork() may be safely used from threads. This also
makes vfork() thread safe, because pthread mutexes will do nothing when
the process is a child of vfork(). More torture tests have been written
to confirm this all works like a charm. Additionally:
- Invent hexpcpy() api
- Rename nsync_malloc_() to kmalloc()
- Complete posix named semaphore implementation
- Make pthread_create() asynchronous signal safe
- Add rm, rmdir, and touch to command interpreter builtins
- Invent sigisprecious() and modify sigset functions to use it
- Add unit tests for posix_spawn() attributes and fix its bugs
One unresolved problem is the reclaiming of *NSYNC waiter memory in the
forked child processes, within apps which have threads waiting on locks
- SQLite file locking now works on Windows
- SQLite will now use fdatasync() on non-Apple platforms
- Fix Ctrl-C handler on Windows to not crash with TLS
- Signals now work in multithreaded apps on Windows
- fcntl() will now accurately report EINVAL errors
- fcntl() now has excellent --strace logging
- Token bucket replenish now go 100x faster
- *NSYNC cancellations now work on Windows
- Support closefrom() on NetBSD
The cosmopolitan command interpreter now has 13 builtin commands,
variable support, support for ; / && / || syntax, asynchronous support,
and plenty of unit tests with bug fixes.
This change fixes a bug in posix_spawn() with null envp arg. strace
logging now uses atomic writes for scatter functions. Breaking change
renaming GetCpuCount() to _getcpucount(). TurfWar is now updated to use
the new token bucket algorithm. WIN32 affinity masks now inherit across
fork() and execve().
This change addresses various open source compatibility issues, so that
we pass 313/411 of the tests in https://github.com/jart/libc-test where
earlier today we were passing about 30/411 of them, due to header toil.
Please note that Glibc only passes 341/411 so 313 today is pretty good!
- Make the conformance of libc/isystem/ headers nearly perfect
- Import more of the remaining math library routines from Musl
- Fix inconsistencies with type signatures of calls like umask
- Write tests for getpriority/setpriority which work great now
- conform to `struct sockaddr *` on remaining socket functions
- Import a bunch of uninteresting stdlib functions e.g. rand48
- Introduce readdir_r, scandir, pthread_kill, sigsetjmp, etc..
Follow the instructions in our `tool/scripts/cosmocc` toolchain to run
these tests yourself. You use `make CC=cosmocc` on the test repository
- fix rare thread exit race condition on openbsd
- pthread_getattr_np() now supplies detached status
- child threads may now pthread_join() the main thread
- introduce sigandset(), sigorset(), and sigisemptyset()
- introduce pthread_cleanup_push() and pthread_cleanup_pop()
You can now do things like implement mutexes using futexes in your
redbean lua code. This provides the fastest possible inter-process
communication for your production systems when SQLite alone as ipc
or things like pipes aren't sufficient.
- Shutdown process now has optimal cancellation latency
- Fairer techniques for shedding connections under load
- We no longer need to call poll() which is now removed
It can now handle 240k SQLite write QPS at 3ms 99 percentile latency.
We're still working out the kinks since it's brand new. But we've got
this running in production already!
This change also found a few POSIX compliance bugs with errnos. Another
bug was discovered where, on Windows, pread() and pwrite() could modify
the file position in cases where ReadFile() returned an error e.g. when
seeking past the end of file. We also have more tests!
This makes it possible for us to use system() and popen() with paths
that redirect to filenames that contain spaces, e.g.
system("echo.com hello >\"hello there.txt\"")
It's difficult to solve this problem, because WIN32 only allows passing
one single argument when launching programs and each program is allowed
to tokenize that however it wants. Most software follows the convention
of cmd.exe which is poorly documented and positively byzantine.
In the future we're going to solve this by not using cmd.exe at all and
instead embedding the cocmd.com interpreter into the system() function.
In the meantime, our documentation has been updated to help recalibrate
any expectation the user might hold regarding the security of using the
Windows command interpreter.
Fixes#644
This change also introduces partial faccessat() support for zipos and
makes some slight breaking changes in errno results. close() is fixed
to use `EBADF` rather than `EINVAL` and we're now using `ENOTSUP` not
`EOPNOTSUPP` to indicate that zipos doesn't support a system call yet
This change improves copy_file_range(), sendfile(), splice(), openpty(),
closefrom(), close_range(), fadvise() and posix_fadvise() in addition to
writing tests that confirm things like errno and seeking behavior across
platforms. We now less aggressively polyfill behavior with some of these
functions when the platform support isn't available. Please see:
https://justine.lol/cosmopolitan/functions.html
Since we're now on Windows 8, we can have clone() work as advertised on
Windows, where it sends a futex wake to the child tid. It's also likely
we no longer need to work around thread flakes on OpenBSD, in _wait0().
Doing this makes binaries tinier, since we don't need to have all the
extra code for supporting a 32-bit address space. It also benefits us
because we're able to use WIN32 futexes, which makes locking simpler.
b69f3d2488 is what officially ended our
Windows 7 support. This change is merely a formalization. You can use
old versions of Cosmo now and forevermore if you need Windows 7 since
our repository is hermetic and vendors all its dependencies.
Won't fix#617
- Fix preadv() and pwritev() for old distros
- Introduce _npassert() and _unassert() macros
- Prove that file locks work properly on Windows
- Support fcntl(F_DUPFD_CLOEXEC) on more systems
This makes breaking changes to add underscores to many non-standard
function names provided by the c library. MODE=tiny is now tinier and
we now use smaller locks that are better for tiny apps in this mode.
Some headers have been renamed to be in the same folder as the build
package, so it'll be easier to know which build dependency is needed.
Certain old misguided interfaces have been removed. Intel intrinsics
headers are now listed in libc/isystem (but not in the amalgamation)
to help further improve open source compatibility. Header complexity
has also been reduced. Lastly, more shell scripts are now available.
The organization of the source files is now much more rational.
Old experiments that didn't work out are now deleted. Naming of
things like files is now more intuitive.
This change fixes#496 where ASAN spotted a race condition that could
happen in multithreaded programs, with more than OPEN_MAX descriptors
when using ZipOS or Windows NT, which require tracking open file info
and this change fixes that table so it never relocates, thus allowing
us to continue to enjoy the benefits of avoiding locks while reading.
If your main module has this declaration:
STATIC_YOINK("vga_console");
Then a VGA driver will be linked into your executable which
displays your stdio characters on the PC display, whereas
before we could only use the serial port. Your display is an
ANSI terminal and it's still a work in progress.
OpenBSD's qsort() function is more secure than the ones used by
FreeBSD, NetBSD and MacOS. The best part is it goes faster too!
This change also imports the OpenBSD mergesort() and heapsort()
It now works most excellently across all supported operating
sytsems (earlier it didn't work on NT and XNU). Demo code is
available in examples/clock.c and this change also adds some
of the newer ANSI C time functions like timespec_get(), plus
timespec_getres() which hasn't even come out yet as it's C23
pthread_mutex_lock() now uses a better algorithm which goes much faster
in multithreaded environments that have lock contention. This comes at
the cost of adding some fixed-cost overhead to mutex invocations. That
doesn't matter for Cosmopolitan because our core libraries all encode
locking operations as NOP instructions when in single-threaded mode.
Overhead only applies starting the moment you first call clone().
This is the same as `unreachable` except it always traps violations,
even if we're not running in MODE=dbg. This is useful for impossible
conditions relating to system calls. It avoids terrifying bugs where
control falls through to an unrelated function.
This change fixes a nasty bug where SIG_IGN and SIG_DFL weren't working
as advertised on BSDs. This change also fixes the tkill() definition on
MacOS so it maps to __pthread_kill().
This change restores the .symtab symbol table files in our flagship
programs (e.g. redbean.com, python.com) needed to show backtraces. This
also rolls back earlier changes to zip.com w.r.t. temp directories since
the right way to do it turned out to be the -b DIR flag.
This change also improves the performance of zip.com. It turned out
mmap() wasn't being used, because zip.com was assuming a 4096-byte
granularity, but cosmo requires 65536. There was also a chance to speed
up stdio scanning using the unlocked functions.
Rosetta does something strange to the signal handling registers but
setting SA_SIGINFO prevents the issue from happening. Set the flag
on XNU to work around the issue.
- Polyfill pselect() on Windows
- Add -O NOFILE flag to pledge.com
- Polyfill ppoll() on NetBSD, XNU, and Windows
- Support negative numbers and errno in sizetol()
- Add .RSS, .NOFILE, and .MAXCORE to Landlock Make
- Fix issue with .PLEDGE preventing touching of output files
- Add __watch() function (like ftrace) for logging memory changes
We now guarantee TMPDIR will be defined on a per build rule basis. It'll
be an absolute path. It'll be secure and unique. It'll be rm -rf'd after
the last shell script line in your build rule is executed. If $TMPDIR is
already defined, then it'll be created as a subdirectory of your $TMPDIR
and then replace the variable with the new definition. The Landlock Make
repository will be updated with examples shortly after this change which
shall be known as Landlock Make 1.1.1.
See #530
It turned out that specifying all SRCS and INCS as dependencies on the
pattern rules for all headers, caused `make` memory usage to skyrocket
from 40mb ot 160mb. This change also reduces the build graph another 4%.
This change introduces the nointernet() function which may be called to
prevent a process and its descendants from communicating with publicly
routable Internet addresses. GNU Make has been modified to always call
this function. In the future Landlock Make will have a way to whitelist
subnets to override this behavior, or disable it entirely. Support is
available for Linux only. Our firewall does not require root access.
Calling nointernet() will return control to the caller inside a new
process that has a SECCOMP BPF filter installed, which traps network
related system calls. Your original process then becomes a permanent
ptrace() supervisor that monitors all processes and threads descending
from the returned child. Whenever a networking system call happens the
kernel will stop the process and wakes up the monitor, which then peeks
into the child memory to read the sockaddr_in to determine if it's ok.
The downside to doing this is that there can be only one supervisor at a
time using ptrace() on a process. So this firewall won't be enabled if
you run make under strace or inside gdb. It also makes testing tricky.
