If you build a static ELF executable in `ld -q` mode (which leaves rela
sections inside the binary) then you can run it through the elf2pe.com
program afterwards, which will turn it into a PE executable. We have a
new trick for defining WIN32 DLL imports in C without any assembly code.
This also achieves the optimally tiny and perfect PE binary structure.
We need this because it isn't possible to have a GNU ld linker script
generate a PE file where the virtual pointer and the file pointer can
drift apart. This post-linker can do that. One cool benefit is we can
now use a smaller 512-byte alignment in the file, and an even bigger
64kb alignment for the segment virtual addresses, and the executable
ends up being smaller.
Another program introduced by this change is pecheck.com which can do
extensive linting of PE static executables to help explain why Windows
won't load it.
- Blocking read operations on the Windows Console can now EINTR
- Blocking read operations on Windows pipes now EINTR more reliably
- setitimer() will no longer be inherited across fork() on Windows
- It's now possible to use ECHO when the console is in raw mode
- The ECHOCTL flag now works correctly on the Windows Console
- The ICRNL flag now works correctly on the Windows Console
- pread() and pwrite() will now raise ESPIPE on Windows
- Opening /dev/tty on Windows is improved (untested)
- Overlapped I/O is now implemented in a better way
- Remove PAGESIZE constant
- Fix realloc() documentation
- Fix ttyname_r() error reporting
- Make forking more reliable on Windows
- Make execvp() a few microseconds faster
- Make system() a few microseconds faster
- Tighten up the socket-related magic numbers
- Loosen restrictions on mmap() offset alignment
- Improve GetProgramExecutableName() with getenv("_")
- Use mkstemp() as basis for mktemp(), tmpfile(), tmpfd()
- Fix flakes in pthread_cancel_test, unix_test, fork_test
- Fix recently introduced futex stack overflow regression
- Let sockets be passed as stdio to subprocesses on Windows
- Improve security of bind() on Windows w/ SO_EXCLUSIVEADDRUSE
This change addresses everything from stack smashing to %SYSTEMROOT%
breaking socket(). Issues relating to compile.com not reporting text
printed to stderr has been resolved for Windows builds.
This change ports APE Loader to Linux AARCH64, so that Raspberry Pi
users can run programs like redbean, without the executable needing
to modify itself. Progress has also slipped into this change on the
issue of making progress better conforming to user expectations and
industry standards regarding which symbols we're allowed to declare
This version has better error messages and safety checks. It supports
loading static position-independent executables. It correctly handles
more kinds of weird ELF program header layouts. A force flag has been
added to avoid system execve(). Finally the longstanding misalignment
with our ELF PT_NOTE section has been addressed.
lz4toasm should now more easily accept LZ4 files output by
compressor programs that do not support the extracted-size
field, such as Stephan Brumme's smallz4.
This patch also proposes to add a new lz4len() function to
the libc: it parses an LZ4 compressed block to compute the
unpacked content size, without really unpacking the block.
Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
- This commit mints a new release of APE Loader v1.2 which supports
loading ELF programs with a non-contiguous virtual address layout
even though we've never been able to take advantage of it, due to
how `objcopy -SO binary` fills any holes left by PT_LOAD. This'll
change soon, since we'll have a new way of creating APE binaries.
- The undiamonding trick with our ioctl() implementation is removed
since POSIX has been killing ioctl() for years and they've done a
much better job. One problem it resolves, is that ioctl(FIONREAD)
wasn't working earlier and that caused issues when building Emacs
- Fix unused local variable errors
- Remove yoinks from sigaction() header
- Add nox87 and aarch64 to github actions
- Fix cosmocc -fportcosmo in linking mode
- It's now possible to build `make m=llvm o/llvm/libc`
- compile.com now polyfills -march=native which gcc/clang removed
- Guarantee zero Windows code is linked into non-Windows binaries
- MODE=tinylinux binaries are now back to being as tiny as ~4kb
- Improve the runtime's stack allocation / alignment hack
- GitHub Actions now tests Linux modes for assurance
- Fix handling of precision in hex float formatting
- Enhance the cocmd interpreter for system() and popen()
- Manually ran the Lua unit tests, which are now passing
- Let stdio i/o operations happen when file is in error state
- We're now saving and restoring xmm in ftrace out of paranoia
It turned out that Landlock Make hasn't been applying sandboxing for a
while, due to a mistyped if statement for `$(USE_SYSTEM_TOOLCHAIN)` it
should have had the opposite meaning. Regressions in the build configs
have been fixed. The rmrf() function works better now. The rm.com tool
works according to POSIX with the exception of supporting prompts.
This change fixes stderr to be unbuffered. Added hardware AES on ARM64
to help safeguard against timing attacks. The curl.com command will be
somewhat more pleasant to use.
The *NSYNC linked list API is good enough that it deserves to be part of
the C libray, so this change writes an improved version of it which uses
that offsetof() trick from the Linux Kernel. We vendor all of the *NSYNC
tests in third_party which helped confirm the needed refactoring is safe
This change also deletes more old code that didn't pan out. My goal here
is to work towards a vision where the Cosmopolitan core libraries become
less experimental and more focused on curation. This better reflects the
current level of quality we've managed to achieve.
After going through the MODE=dbg and MODE=zero build modes, a bunch of
little issues were identified, which have been addressed. Fixing those
issues created even more troubles for the project, because it improved
our ability to detect latent problems which are getting fixed so fast.
The build/bootstrap/ar.com program is now tinier. This change reduces
its size from 140kb to 53kb. Nothing was traded away. Cosmopolitan Ar
performance is now 2x better than llvm-ar largely thanks to using the
copy_file_range() system call. This change homebrews a new allocation
API that addresses the shortcomings of the C standard library design.
Using these new balloc() and reballoc() functions I managed to reduce
memory consumption so much that Cosmpolitan Ar should now use roughly
100x fewer bytes of peak resident memory compared to llvm-ar. Correct
behavior with better compatibility has been assured. Binary output is
now pretty much bit-identical to llvm-ar, as of this change. This can
and should be the living proof we need to show that a better world is
possible for software.
- Found some bugs in LLVM compiler-rt library
- The useless LIBC_STUBS package is now deleted
- Improve the overflow checking story even further
- Get chibicc tests working in MODE=dbg mode again
- The libc/isystem/ headers now have correctly named guards
Since 8ff48201ca we no longer need the
hack where, when running .com.dbg files, we scanned for the embedded
.com file offset, and then computed zip offsets realtive to that. It
wasn't very reliable in the first place, and was causing issues with
running our new .com.dbg executables, which are true zip files.
- More timspec_*() and timeval_*() APIs have been introduced.
- The copyfd() function is now simplified thanks to POSIX rules.
- More Cosmo-specific APIs have been moved behind the COSMO define.
- The setitimer() polyfill for Windows NT is now much higher quality.
- Fixed build error for MODE=aarch64 due to -mstringop-strategy=loop.
- This change introduces `make MODE=nox87 toolchain` which makes it
possible to build programs using your cosmocc toolchain that don't
have legacy fpu instructions. This is useful, for example, if you
want to have a ~22kb tinier blink virtual machine.
This change fixes an issue with the tcflow() magic numbers that was
causing bash to freeze up on Linux. While auditing termios polyfills,
several other issues were identified with XNU/BSD compatibility.
Out of an abundance of caution this change undefines as much surface
area from libc/calls/struct/termios.h as possible, so that autoconf
scripts are less likely to detect non-POSIX teletypewriter APIs that
haven't been polyfilled by Cosmopolitan.
This is a *breaking change* for your static archives in /opt/cosmos if
you use the cosmocc toolchain. That's because this change disables the
ioctl() undiamonding trick for code outside the monorepo, specifically
because it'll lead to brittle ABI breakages like this. If you're using
the cosmocc toolchain, you'll need to rebuild libraries like ncurses,
readline, etc. Yes diamonds cause bloat. To work around that, consider
using tcgetwinsize() instead of ioctl(TIOCGWINSZ) since it'll help you
avoid pulling every single ioctl-related polyfill into the linkage.
The cosmocc script was specifying -DNDEBUG for some reason. It's fixed.
This change takes an entirely new approach to the incremental linking of
pkzip executables. The assets created by zipobj.com are now treated like
debug data. After a .com.dbg is compiled, fixupobj.com should be run, so
it can apply fixups to the offsets and move the zip directory to the end
of the file. Since debug data doesn't get objcopy'd, a new tool has been
introduced called zipcopy.com which should be run after objcopy whenever
a .com file is created. This is all automated by the `cosmocc` toolchain
which is rapidly becoming the new recommended approach.
This change also introduces the new C23 checked arithmetic macros.
This change improves the way internal APIs are being hidden behind the
`COSMO` define. The cosmo.h header will take care of defining that, so
that a separate define statement isn't needed. This change also does a
lot more to define which APIs are standard, and which belong to Cosmo.
This change integrates e58abc1110b335a3341e8ad5821ad8e3880d9bb2 from
https://github.com/ahgamut/musl-cross-make/ which fixes the issues we
were having with our C language extension for symbolic constants. This
change also performs some code cleanup and bug fixes to getaddrinfo().
It's now possible to compile projects like ncurses, readline and python
without needing to patch anything upstream, except maybe a line or two.
Pretty soon it should be possible to build a Linux distro on Cosmo.
- Work towards improving non-optimized build support
- Introduce MODE=zero which is -O0 without ASAN/UBSAN
- Use system GCC when ~/.cosmo.mk has USE_SYSTEM_TOOLCHAIN=1
- Have package.com check .privileged code doesn't call non-privileged
This change greatly reduces the number of modules that need to be
compiled. The only issue right now is that sometimes when viewing
symbol table entries, the aliased symbol is chosen.
This change implements a new approach to function call logging, that's
based on the GCC flag: -fpatchable-function-entry. Read the commentary
in build/config.mk to learn how it works.
This change progresses our AARCH64 support:
- The AARCH64 build and tests are now passing
- Add 128-bit floating-point support to printf()
- Fix clone() so it initializes cosmo's x28 TLS register
- Fix TLS memory layout issue with aarch64 _Alignas vars
- Revamp microbenchmarking tools so they work on aarch64
- Make some subtle improvements to aarch64 crash reporting
- Make kisdangerous() memory checks more accurate on aarch64
- Remove sys_open() since it's not available on Linux AARCH64
This change makes general improvements to Cosmo and Redbean:
- Introduce GetHostIsa() function in Redbean
- You can now feature check using pledge(0, 0)
- You can now feature check using unveil("",0)
- Refactor some more x86-specific asm comments
- Refactor and write docs for some libm functions
- Make the mmap() API behave more similar to Linux
- Fix WIFSIGNALED() which wrongly returned true for zero
- Rename some obscure cosmo keywords from noFOO to dontFOO
