- Fix sigsuspend() on XNU
- Fix strsignal() on non-Linux
- Add unit tests for strsignal()
- Add unit tests for setitimer()
- Add unit tests for sigsuspend()
- Rewrite setitimer() for New Technology
- Rewrite nanosleep() for New Technology
- Polyfill SIGALRM on the New Technology
- select(0,0,0,0) on NT now calls pause()
- Remove some NTDLL calls that aren't needed
- Polyfill SA_NOCLDWAIT on the New Technology
- Polyfill SA_RESETHAND on the New Technology
- Polyfill sigprocmask() on the New Technology
- Polyfill SIGCHLD+SIG_IGN on the New Technology
- Polyfill SA_RESTART masking on the New Technology
- Deliver console signals from main thread on New Technology
- Document SA_RESTART behavior w/ @sarestartable / @norestart
- System call trace in MODE=dbg now prints inherited FDs and signal mask
This is similar to the --ftrace (c function call trace) flag, except
it's less noisy since it only logs system calls to stderr. Having this
flag is valuable because (1) system call tracing tells us a lot about
the behavior of complex programs and (2) it's usually very hard to get
system call tracing on various operating systems, e.g. strace, ktrace,
dtruss, truss, nttrace, etc. Especially on Apple platforms where even
with the special boot trick, debuggers still aren't guaranteed to work.
make -j8 o//examples
o//examples/hello.com --strace
This is enabled by default in MODE=, MODE=opt, and MODE=dbg. In MODE=dbg
extra information will be printed.
make -j8 MODE=dbg o/dbg/examples
o/dbg/examples/hello.com --strace |& less
This change also changes:
- Rename IsText() → _istext()
- Rename IsUtf8() → _isutf8()
- Fix madvise() on Windows NT
- Fix empty string case of inet_ntop()
- vfork() wrapper now saves and restores errno
- Update xsigaction() to yoink syscall support
- Python static hello world now 1.8mb
- Python static fully loaded now 10mb
- Python HTTPS client now uses MbedTLS
- Python REPL now completes import stmts
- Increase stack size for Python for now
- Begin synthesizing posixpath and ntpath
- Restore Python \N{UNICODE NAME} support
- Restore Python NFKD symbol normalization
- Add optimized code path for Intel SHA-NI
- Get more Python unit tests passing faster
- Get Python help() pagination working on NT
- Python hashlib now supports MbedTLS PBKDF2
- Make memcpy/memmove/memcmp/bcmp/etc. faster
- Add Mersenne Twister and Vigna to LIBC_RAND
- Provide privileged __printf() for error code
- Fix zipos opendir() so that it reports ENOTDIR
- Add basic chmod() implementation for Windows NT
- Add Cosmo's best functions to Python cosmo module
- Pin function trace indent depth to that of caller
- Show memory diagram on invalid access in MODE=dbg
- Differentiate stack overflow on crash in MODE=dbg
- Add stb_truetype and tools for analyzing font files
- Upgrade to UNICODE 13 and reduce its binary footprint
- COMPILE.COM now logs resource usage of build commands
- Start implementing basic poll() support on bare metal
- Set getauxval(AT_EXECFN) to GetModuleFileName() on NT
- Add descriptions to strerror() in non-TINY build modes
- Add COUNTBRANCH() macro to help with micro-optimizations
- Make error / backtrace / asan / memory code more unbreakable
- Add fast perfect C implementation of μ-Law and a-Law audio codecs
- Make strtol() functions consistent with other libc implementations
- Improve Linenoise implementation (see also github.com/jart/bestline)
- COMPILE.COM now suppresses stdout/stderr of successful build commands
The ZIP filesystem has a breaking change. You now need to use /zip/ to
open() / opendir() / etc. assets within the ZIP structure of your APE
binary, instead of the previous convention of using zip: or zip! URIs.
This is needed because Python likes to use absolute paths, and having
ZIP paths encoded like URIs simply broke too many things.
Many more system calls have been updated to be able to operate on ZIP
files and file descriptors. In particular fcntl() and ioctl() since
Python would do things like ask if a ZIP file is a terminal and get
confused when the old implementation mistakenly said yes, because the
fastest way to guarantee native file descriptors is to dup(2). This
change also improves the async signal safety of zipos and ensures it
doesn't maintain any open file descriptors beyond that which the user
has opened.
This change makes a lot of progress towards adding magic numbers that
are specific to platforms other than Linux. The philosophy here is that,
if you use an operating system like FreeBSD, then you should be able to
take advantage of FreeBSD exclusive features, even if we don't polyfill
them on other platforms. For example, you can now open() a file with the
O_VERIFY flag. If your program runs on other platforms, then Cosmo will
automatically set O_VERIFY to zero. This lets you safely use it without
the need for #ifdef or ifstatements which detract from readability.
One of the blindspots of the ASAN memory hardening we use to offer Rust
like assurances has always been that memory passed to the kernel via
system calls (e.g. writev) can't be checked automatically since the
kernel wasn't built with MODE=asan. This change makes more progress
ensuring that each system call will verify the soundness of memory
before it's passed to the kernel. The code for doing these checks is
fast, particularly for buffers, where it can verify 64 bytes a cycle.
- Correct O_LOOP definition on NT
- Introduce program_executable_name
- Add ASAN guards to more system calls
- Improve termios compatibility with BSDs
- Fix bug in Windows auxiliary value encoding
- Add BSD and XNU specific errnos and open flags
- Add check to ensure build doesn't talk to internet
The most exciting improvement is dynamic pages will soon be able to use
the executable itself as an object store. it required a heroic technique
for overcoming ETXTBSY restrictions which lets us open the executable in
read/write mode, which means (1) wa can restore the APE header, and (2)
we can potentially containerize redbean extension code so that modules
you download for your redbean online will only impact your redbean.
Here's a list of breaking changes to redbean:
- Remove /tool/net/ prefix from magic ZIP paths
- GetHeader() now returns NIL if header is absent
Here's a list of fixes and enhancements to redbean:
- Support 64-bit ZIP archives
- Record User-Agent header in logs
- Add twelve error handlers to accept()
- Display octal st_mode on listing page
- Show ZIP file comments on listing page
- Restore APE MZ header on redbean startup
- Track request count on redbean index page
- Report server uptime on redbean index page
- Don't bind server socket using SO_REUSEPORT
- Fix#151 where Lua LoadAsset() could free twice
- Report rusage accounting when workers exit w/ -vv
- Use ZIP iattr field as text/plain vs. binary hint
- Add ParseUrl() API for parsing things like a.href
- Add ParseParams() API for parsing HTTP POST bodies
- Add IsAcceptablePath() API for checking dots, etc.
- Add IsValidHttpToken() API for validating sane ASCII
- Add IsAcceptableHostPort() for validating HOST[:PORT]
- Send 400 response to HTTP/1.1 requests without a Host
- Send 403 response if ZIP or file isn't other readable
- Add virtual hosting that tries prepending Host to path
- Route requests based on Host in Request-URI if present
- Host routing will attempt to remove or add the www. prefix
- Sign-extend UNIX timestamps and don't adjust FileTime zone
Here's some of the improvements made to Cosmopolitan Libc:
- Fix ape.S indentation
- Improve consts.sh magnums
- Write pretty good URL parser
- Improve rusage accounting apis
- Bring mremap() closer to working
- Added ZIP APIs which will change
- Check for overflow in reallocarray()
- Remove overly fancy linkage in strerror()
- Fix GDB attach on crash w/ OpenBSD msyscall()
- Make sigqueue() portable to most UNIX distros
- Make integer serialization macros more elegant
- Bring back 34x tprecode8to16() performance boost
- Make malloc() more resilient to absurdly large sizes
- Polyfill ucontext_t on FreeBSD/OpenBSD/NetBSD
- Add tests confirming signals can edit CPU state
- Work towards supporting ZIP filesystem on bare metal
- Add more tinymath unit tests for POSIX conformance
- Add X87 and SSE status flags to crash report
- Fix some bugs in blinkenlights
- Fix llvm build breakage
Your Actually Portable Executables now contains a simple virtual memory
that works similarly to the Linux Kernel in the sense that it maps your
physical memory to negative addresses. This is needed to support mmap()
and malloc(). This functionality has zero code size impact. For example
the MODE=tiny LIFE.COM executable is still only 12KB in size.
The APE bootloader code has also been simplified to improve readibility
and further elevate the elegance by which we're able to support so many
platforms thereby enhancing verifiability so that we may engender trust
in this bootloading process.
You can now build Cosmopolitan with Clang:
make -j8 MODE=llvm
o/llvm/examples/hello.com
The assembler and linker code is now friendly to LLVM too.
So it's not needed to configure Clang to use binutils under
the hood. If you love LLVM then you can now use pure LLVM.
- Get ASAN working on Windows.
- Deleting directories and then recreating them with the same name in a
short period of time appears to be a no-no on Windows.
- There's no reason to call FlushFileBuffers on close() for pipes, and
it's harmful since it might block indefinitely for no good reason.
- Remove XD bit in page tables
- Fix cylinder+head+sector arithmetic
- Implement fstat() for serial file descriptors on metal
Here's how to boot an Actually Portable Executable in QEMU:
make -j12 o//tool/viz/deathstar.com
qemu-system-x86_64 -serial stdio -fda o//tool/viz/deathstar.com
Here's a screenshot of DEATHSTAR.COM booted in QEMU:
https://justine.lol/cosmopolitan/cosmo-metal-qemu.png
Thus metal support is in much better shape now, but still incomplete.
Only a few system calls have been polyfilled. To figure out which ones
your program needs, simply boot it in the blinkenlights emulator with a
breakpoint, and press CTRL-C to continue to the system call breakpoint.
If it doesn't break then you should be good. (Note: to emulate normally
you can press 'c' and use CTRL-T and ALT-T to tune the speed.)
m=tiny
make -j12 SILENT=0 MODE=$m \
o/$m/tool/build/blinkenlights.com \
o/$m/tool/viz/deathstar.com
o/$m/tool/build/blinkenlights.com \
-r -t -b systemfive.linux \
o/$m/tool/viz/deathstar.com
Thank @Theldus for the bug report that made this change possible.
Fixes#20 which explains this change further.
A new rollup tool now exists for flattening out the headers in a way
that works better for our purposes than cpp. A lot of the API clutter
has been removed. APIs that aren't a sure thing in terms of general
recommendation are now marked internal.
There's now a smoke test for the amalgamation archive and gigantic
header file. So we can now guarantee you can use this project on the
easiest difficulty setting without the gigantic repository.
A website is being created, which is currently a work in progress:
https://justine.storage.googleapis.com/cosmopolitan/index.html
I wanted a tiny scriptable meltdown proof way to run userspace programs
and visualize how program execution impacts memory. It helps to explain
how things like Actually Portable Executable works. It can show you how
the GCC generated code is going about manipulating matrices and more. I
didn't feel fully comfortable with Qemu and Bochs because I'm not smart
enough to understand them. I wanted something like gVisor but with much
stronger levels of assurances. I wanted a single binary that'll run, on
all major operating systems with an embedded GPL barrier ZIP filesystem
that is tiny enough to transpile to JavaScript and run in browsers too.
https://justine.storage.googleapis.com/emulator625.mp4
