There's a new program named ape/ape-m1.c which will be used to build an
embeddable binary that can load ape and elf executables. The support is
mostly working so far, but still chasing down ABI issues.
- Introduce epoll_pwait()
- Rewrite -ftrapv and ffs() libraries in C code
- Use more FreeBSD code in math function library
- Get significantly more tests passing on qemu-aarch64
- Fix many Musl long double functions that were broken on AARCH64
This makes breaking changes to add underscores to many non-standard
function names provided by the c library. MODE=tiny is now tinier and
we now use smaller locks that are better for tiny apps in this mode.
Some headers have been renamed to be in the same folder as the build
package, so it'll be easier to know which build dependency is needed.
Certain old misguided interfaces have been removed. Intel intrinsics
headers are now listed in libc/isystem (but not in the amalgamation)
to help further improve open source compatibility. Header complexity
has also been reduced. Lastly, more shell scripts are now available.
- 10.5% reduction of o//depend dependency graph
- 8.8% reduction in latency of make command
- Fix issue with temporary file cleanup
There's a new -w option in compile.com that turns off the recent
Landlock output path workaround for "good commands" which do not
unlink() the output file like GNU tooling does.
Our new GNU Make unveil sandboxing appears to have zero overhead
in the grand scheme of things. Full builds are pretty fast since
the only thing that's actually slowed us down is probably libcxx
make -j16 MODE=rel
RL: took 85,732,063µs wall time
RL: ballooned to 323,612kb in size
RL: needed 828,560,521µs cpu (11% kernel)
RL: caused 39,080,670 page faults (99% memcpy)
RL: 350,073 context switches (72% consensual)
RL: performed 0 reads and 11,494,960 write i/o operations
pledge() and unveil() no longer consider ENOSYS to be an error.
These functions have also been added to Python's cosmo module.
This change also removes some WIN32 APIs and System Five magnums
which we're not using and it's doubtful anyone else would be too
The whole repository is now buildable with GNU Make Landlock sandboxing.
This proves that no Makefile targets exist which touch files other than
their declared prerequisites. In order to do this, we had to:
1. Stop code morphing GCC output in package.com and instead run a
newly introduced FIXUPOBJ.COM command after GCC invocations.
2. Disable all the crumby Python unit tests that do things like create
files in the current directory, or rename() files between folders.
This ended up being a lot of tests, but most of them are still ok.
3. Introduce an .UNSANDBOXED variable to GNU Make to disable Landlock.
We currently only do this for things like `make tags`.
4. This change deletes some GNU Make code that was preventing the
execve() optimization from working. This means it should no longer
be necessary in most cases for command invocations to be indirected
through the cocmd interpreter.
5. Missing dependencies had to be declared in certain places, in cases
where they couldn't be automatically determined by MKDEPS.COM
6. The libcxx header situation has finally been tamed. One of the
things that makes this difficult is MKDEPS.COM only wants to
consider the first 64kb of a file, in order to go fast. But libcxx
likes to have #include lines buried after huge documentation.
7. An .UNVEIL variable has been introduced to GNU Make just in case
we ever wish to explicitly specify additional things that need to
be whitelisted which aren't strictly prerequisites. This works in
a manner similar to the recently introduced .EXTRA_PREREQS feature.
There's now a new build/bootstrap/make.com prebuilt binary available. It
should no longer be possible to write invalid Makefile code.
This change switches most of the core locks to be re-entrant, in order
to reduce the chance of deadlocking code that does, clever things with
asynchronous signal handlers. This change implements it it in pthreads
so we're one step closer to having a standardized threading primitives
- Document redbean's argon2 module
- Fix regressions in cthreads library
- Make testlib work better with threads
- Give the cthreads library lots of love
- Remove some of the stdio assembly code
- Implement getloadavg() across platforms
- Code size optimizations for errnos, etc.
- Only check for signals in main thread on Windows
- Make errnos for dup2 / dup3 consistent with posix
This change also fixes a bug in the argon2 module, where the NUL
terminator was being included in the hash encoded ascii string. This
shouldn't require any database migrations to folks who found this module
and productionized it, since the argon2 library treats it as a c string.
- Fix some minor issues in ar.com
- Have execve() look for `ape` command
- Rewrite NT paths using /c/ rather /??/c:/
- Replace broken GCC symlinks with .sym files
- Rewrite $PATH environment variables on startup
- Make $(APE_NO_MODIFY_SELF) the default bootloader
- Add all build command dependencies to build/bootstrap
- Get the repository mostly building from source on non-Linux
- Document more compiler flags
- Expose new __print_maps() api
- Better overflow checking in mmap()
- Improve the shell example somewhat
- Fix minor runtime bugs regarding stacks
- Make kill() on fork()+execve()'d children work
- Support CLONE_CHILD_CLEARTID for proper joining
- Fix recent possible deadlock regression with --ftrace
- add vdso dump utility
- tests now log stack usage
- rename g_ftrace to __ftrace
- make internal spinlocks go faster
- add conformant c11 atomics library
- function tracing now logs stack usage
- make function call tracing thread safe
- add -X unsecure (no ssl) mode to redbean
- munmap() has more consistent behavior now
- pacify fsync() calls on python unit tests
- make --strace flag work better in redbean
- start minimizing and documenting compiler flags
- Get threads working on NetBSD
- Get threads working on OpenBSD
- Fix Emacs config for Emacs v28
- Improve --strace logging of sigset_t
- Improve --strace logging of struct stat
- Improve memory safety of DescribeThing functions
- Refactor auto stack allocation into LIBC_RUNTIME
- Introduce shell.com example which works on Windows
- Refactor __strace_thing into DescribeThing functions
- Document the CHECK macros and improve them in NDEBUG mode
- Rewrite MAP_STACK so it uses FreeBSD behavior across platforms
- Deprecate and discourage the use of MAP_GROWSDOWN (it's weird)
- Improve serialization
- Add Benchmark() API to redbean
- Refactor UNIX API to be assert() friendly
- Make the redbean Lua REPL print data structures
- Fix recent regressions in linenoise reverse search
- Add -i flag so redbean can be a language interpreter
- Get clone() working on FreeBSD
- Increase some Python build quotas
- Add more atomic builtins to chibicc
- Fix ASAN poisoning of alloca() memory
- Make MODE= mandatory link path tinier
- Improve the examples folder a little bit
- Start working on some more resource limits
- Make the linenoise auto-complete UI as good as GNU readline
- Update compile.com, avoiding AVX codegen on non-AVX systems
- Make sure empty path to syscalls like opendir raises ENOENT
- Correctly polyfill ENOENT vs. ENOTDIR on the New Technology
- Port bestline's paredit features to //third_party/linenoise
- Remove workarounds for RHEL 5.0 bugs that were fixed in 5.1
- Update a couple unicode data files
- Disable strace during logger calls
- SQLite now uses pread() / pwrite()
- pread() past EOF on NT now returns 0
- Make the NT mmap() and fork() code elegant
- Give NT a big performance boost with memory
- Add many more mmap() tests to prove it works
You can now do epic things like this:
puts(_gc(xasprintf("%d", 123)));
The _gc() API is shorthand for _defer() which works like Go's keyword:
const char *s = xasprintf("%d", 123);
_defer(free, s);
puts(s);
Be sure to always use -fno-omit-frame-pointer which makes code fast too.
Enjoy! See also #114
We can put this back the moment someone requests it. Pain-free garbage
collection for the C language is pretty cool. All it does is overwrite
the return address with a trampoline that calls free(). It's not clear
what it should be named if it's made a public API.
- Get ASAN working on Windows.
- Deleting directories and then recreating them with the same name in a
short period of time appears to be a no-no on Windows.
- There's no reason to call FlushFileBuffers on close() for pipes, and
it's harmful since it might block indefinitely for no good reason.
This change enables Address Sanitizer systemically w/ `make MODE=dbg`.
Our version of Rust's `unsafe` keyword is named `noasan` which is used
for two functions that do aligned memory chunking, like `strcpy.c` and
we need to fix the tiny DEFLATE code, but that's it everything else is
fabulous you can have all the fischer price security blankets you need
Best of all is we're now able to use the ASAN data in Blinkenlights to
colorize the memory dumps. See the screenshot below of a test program:
https://justine.lol/blinkenlights/asan.png
Which is operating on float arrays stored on the stack, with red areas
indicating poisoned memory, and the green areas indicate valid memory.
A new rollup tool now exists for flattening out the headers in a way
that works better for our purposes than cpp. A lot of the API clutter
has been removed. APIs that aren't a sure thing in terms of general
recommendation are now marked internal.
There's now a smoke test for the amalgamation archive and gigantic
header file. So we can now guarantee you can use this project on the
easiest difficulty setting without the gigantic repository.
A website is being created, which is currently a work in progress:
https://justine.storage.googleapis.com/cosmopolitan/index.html
This is done without using Microsoft's internal APIs. MAP_PRIVATE
mappings are copied to the subprocess via a pipe, since Microsoft
doesn't want us to have proper COW pages. MAP_SHARED mappings are
remapped without needing to do any copying. Global variables need
copying along with the stack and the whole heap of anonymous mem.
This actually improves the reliability of the redbean http server
although one shouldn't expect 10k+ connections on a home computer
that isn't running software built to serve like Linux or FreeBSD.
This change includes many bug fixes, for the NT polyfills, strings,
memory, boot, and math libraries which were discovered by adding more
tools for recreational programming, such as PC emulation. Lemon has also
been vendored because it works so well at parsing languages.
I wanted a tiny scriptable meltdown proof way to run userspace programs
and visualize how program execution impacts memory. It helps to explain
how things like Actually Portable Executable works. It can show you how
the GCC generated code is going about manipulating matrices and more. I
didn't feel fully comfortable with Qemu and Bochs because I'm not smart
enough to understand them. I wanted something like gVisor but with much
stronger levels of assurances. I wanted a single binary that'll run, on
all major operating systems with an embedded GPL barrier ZIP filesystem
that is tiny enough to transpile to JavaScript and run in browsers too.
https://justine.storage.googleapis.com/emulator625.mp4
