This change boosts SSL handshake performance from 2,627 to ~10,000 per
second which is the same level of performance as NGINX at establishing
secure connections. That's impressive if we consider that redbean is a
forking frontend application server. This was accomplished by:
1. Enabling either SSL session caching or SSL tickets. We choose to
use tickets since they reduce network round trips too and that's
a more important metric than wrk'ing localhost.
2. Fixing mbedtls_mpi_sub_abs() which is the most frequently called
function. It's called about 12,000 times during an SSL handshake
since it's the basis of most arithmetic operations like addition
and for some strange reason it was designed to make two needless
copies in addition to calling malloc and free. That's now fixed.
3. Improving TLS output buffering during the SSL handshake only, so
that only a single is write and read system call is needed until
blocking on the ping pong.
redbean will now do a better job wiping sensitive memory from a child
process as soon as it's not needed. The nice thing about fork is it's
much faster than reverse proxying so the goal is to use the different
address spaces along with setuid() to minimize the risk that a server
key will be compromised in the event that application code is hacked.
redbean improvements:
- Explicitly disable corking
- Simulate Python regex API for Lua
- Send warmup requests in main process on startup
- Add Class-A granular IPv4 network classification
- Add /statusz page so you can monitor your redbean's health
- Fix regressions on OpenBSD/NetBSD caused by recent changes
- Plug Authorization header into Lua GetUser and GetPass APIs
- Recognize X-Forwarded-{For,Host} from local reverse proxies
- Add many additional functions to redbean Lua server page API
- Report resource usage of child processes on `/` listing page
- Introduce `-a` flag for logging child process resource usage
- Introduce `-t MILLIS` flag and `ProgramTimeout(ms)` init API
- Introduce `-H "Header: value"` flag and `ProgramHeader(k,v)` API
Cosmopolitan Libc improvements:
- Make strerror() simpler
- Make inet_pton() not depend on sscanf()
- Fix OpenExecutable() which broke .data section earlier
- Fix stdio in cases where it overflows kernel tty buffer
- Fix bugs in crash reporting w/o .com.dbg binary present
- Add polyfills for SO_LINGER, SO_RCVTIMEO, and SO_SNDTIMEO
- Polyfill TCP_CORK on BSD and XNU using TCP_NOPUSH magnums
New netcat clone in examples/nc.c:
While testing some of the failure conditions for redbean, I noticed that
BusyBox's `nc` command is pretty busted, if you use it as an interactive
tool, rather than having it be part of a pipeline. Unfortunately this'll
only work on UNIX since Windows doesn't let us poll on stdio and sockets
at the same time because I don't think they want tools like this running
on their platform. So if you want forbidden fruit, it's here so enjoy it
You can now build Cosmopolitan with Clang:
make -j8 MODE=llvm
o/llvm/examples/hello.com
The assembler and linker code is now friendly to LLVM too.
So it's not needed to configure Clang to use binutils under
the hood. If you love LLVM then you can now use pure LLVM.
You can now use cosmopolitan.h with an ANSI C89 compiler like MSVC. The
Cosmopolitan codebase itself won't support being compiled that way. But
you can build objects that link against Cosmopolitan using any compiler
and you can furthermore use tools like IntelliSense that can't even GNU
See also #40
- Support deterministic stacks on OpenBSD
- Support OpenBSD system call origin verification
- Fix overrun by one in chibicc string token allocator
- Get all chibicc tests passing under Address Sanitizer
This change enables Address Sanitizer systemically w/ `make MODE=dbg`.
Our version of Rust's `unsafe` keyword is named `noasan` which is used
for two functions that do aligned memory chunking, like `strcpy.c` and
we need to fix the tiny DEFLATE code, but that's it everything else is
fabulous you can have all the fischer price security blankets you need
Best of all is we're now able to use the ASAN data in Blinkenlights to
colorize the memory dumps. See the screenshot below of a test program:
https://justine.lol/blinkenlights/asan.png
Which is operating on float arrays stored on the stack, with red areas
indicating poisoned memory, and the green areas indicate valid memory.
We always favor calling functions like openat(), fstatat(), etc. because
Linux, XNU, FreeBSD, and OpenBSD all elected to support them, while some
systems like Android love them so much, that they stopped supporting the
old interfaces.
This change ensures that when dirfd is actually a dirfd and not AT_FDCWD
we'll do the right thing on Windows NT. We use an API that's been around
since Vista to accomplish that.
This change also adds exponential backoff to chdir() on Windows since it
seems almost as flaky on Windows 7 as the rmdir() function.
For the first time ever, all tests in this codebase now pass, when
run automatically on macos, freebsd, openbsd, rhel5, rhel7, alpine
and windows via the network using the runit and runitd build tools
- Fix vfork exec path etc.
- Add XNU opendir() support
- Add OpenBSD opendir() support
- Add Linux history to syscalls.sh
- Use copy_file_range on FreeBSD 13+
- Fix system calls with 7+ arguments
- Fix Windows with greater than 16 FDs
- Fix RUNIT.COM and RUNITD.COM flakiness
- Fix OpenBSD munmap() when files are mapped
- Fix long double so it's actually long on Windows
- Fix OpenBSD truncate() and ftruncate() thunk typo
- Let Windows fcntl() be used on socket files descriptors
- Fix Windows fstat() which had an accidental printf statement
- Fix RHEL5 CLOCK_MONOTONIC by not aliasing to CLOCK_MONOTONIC_RAW
This is wonderful. I never could have dreamed it would be possible
to get it working so well on so many platforms with tiny binaries.
Fixes#31Fixes#25Fixes#14
This program popped up on Hacker News recently. It's the only modern
compiler I've ever seen that doesn't have dependencies and is easily
modified. So I added all of the missing GNU extensions I like to use
which means it might be possible soon to build on non-Linux and have
third party not vendor gcc binaries.
A new rollup tool now exists for flattening out the headers in a way
that works better for our purposes than cpp. A lot of the API clutter
has been removed. APIs that aren't a sure thing in terms of general
recommendation are now marked internal.
There's now a smoke test for the amalgamation archive and gigantic
header file. So we can now guarantee you can use this project on the
easiest difficulty setting without the gigantic repository.
A website is being created, which is currently a work in progress:
https://justine.storage.googleapis.com/cosmopolitan/index.html
This change includes many bug fixes, for the NT polyfills, strings,
memory, boot, and math libraries which were discovered by adding more
tools for recreational programming, such as PC emulation. Lemon has also
been vendored because it works so well at parsing languages.
- Emulator can now test the αcτµαlly pδrταblε εxεcµταblε bootloader
- Whipped up a webserver named redbean. It services 150k requests per
second on a single core. Bundling assets inside zip enables extremely
fast serving for two reasons. The first is that zip central directory
lookups go faster than stat() system calls. The second is that both
zip and gzip content-encoding use DEFLATE, therefore, compressed
responses can be served via the sendfile() system call which does an
in-kernel copy directly from the zip executable structure. Also note
that red bean zip executables can be deployed easily to all platforms,
since these native executables work on Linux, Mac, BSD, and Windows.
- Address sanitizer now works very well
I wanted a tiny scriptable meltdown proof way to run userspace programs
and visualize how program execution impacts memory. It helps to explain
how things like Actually Portable Executable works. It can show you how
the GCC generated code is going about manipulating matrices and more. I
didn't feel fully comfortable with Qemu and Bochs because I'm not smart
enough to understand them. I wanted something like gVisor but with much
stronger levels of assurances. I wanted a single binary that'll run, on
all major operating systems with an embedded GPL barrier ZIP filesystem
that is tiny enough to transpile to JavaScript and run in browsers too.
https://justine.storage.googleapis.com/emulator625.mp4
