Commit graph

162 commits

Author SHA1 Message Date
Justine Tunney
afc58a8b41
Upgrade cosmopolitan gcc and binutils to 2.35.2
This change upgrades to the latest portcosmo gcc patch
6728fe1a25185560603ca312a8d4352af2a4e515 which lets us avoid needing to
define __tmpcosmo_FOO constants. We're now using an appropriate binutils
version for GCC 11. The older binutils sometimes wasn't able to print
backtraces, due to not being able to find a .debug_ranges section.

This is breaking change for /opt/cosmos libraries :'( due to this weird
"error: need linked-to section for --gc-sections" that pops up.

Please run `make clean` in the monorepo before rebuilding.
2023-06-18 00:54:14 -07:00
Justine Tunney
562a1384cd
Make blink support conditionally linkable into APE 2023-06-17 07:55:35 -07:00
Justine Tunney
c3440d040c
Make improvements
- More timspec_*() and timeval_*() APIs have been introduced.
- The copyfd() function is now simplified thanks to POSIX rules.
- More Cosmo-specific APIs have been moved behind the COSMO define.
- The setitimer() polyfill for Windows NT is now much higher quality.
- Fixed build error for MODE=aarch64 due to -mstringop-strategy=loop.
- This change introduces `make MODE=nox87 toolchain` which makes it
  possible to build programs using your cosmocc toolchain that don't
  have legacy fpu instructions. This is useful, for example, if you
  want to have a ~22kb tinier blink virtual machine.
2023-06-15 14:50:53 -07:00
Justine Tunney
4778cd4d27
Fix bugs in termios library and cleanup code
This change fixes an issue with the tcflow() magic numbers that was
causing bash to freeze up on Linux. While auditing termios polyfills,
several other issues were identified with XNU/BSD compatibility.

Out of an abundance of caution this change undefines as much surface
area from libc/calls/struct/termios.h as possible, so that autoconf
scripts are less likely to detect non-POSIX teletypewriter APIs that
haven't been polyfilled by Cosmopolitan.

This is a *breaking change* for your static archives in /opt/cosmos if
you use the cosmocc toolchain. That's because this change disables the
ioctl() undiamonding trick for code outside the monorepo, specifically
because it'll lead to brittle ABI breakages like this. If you're using
the cosmocc toolchain, you'll need to rebuild libraries like ncurses,
readline, etc. Yes diamonds cause bloat. To work around that, consider
using tcgetwinsize() instead of ioctl(TIOCGWINSZ) since it'll help you
avoid pulling every single ioctl-related polyfill into the linkage.

The cosmocc script was specifying -DNDEBUG for some reason. It's fixed.
2023-06-14 19:30:52 -07:00
Justine Tunney
2eb7148474
Create a fake syscall() function due to Python 2023-06-09 05:18:38 -07:00
Justine Tunney
23e235b7a5
Fix bugs in cosmocc toolchain
This change integrates e58abc1110b335a3341e8ad5821ad8e3880d9bb2 from
https://github.com/ahgamut/musl-cross-make/ which fixes the issues we
were having with our C language extension for symbolic constants. This
change also performs some code cleanup and bug fixes to getaddrinfo().
It's now possible to compile projects like ncurses, readline and python
without needing to patch anything upstream, except maybe a line or two.
Pretty soon it should be possible to build a Linux distro on Cosmo.
2023-06-08 23:44:03 -07:00
Justine Tunney
22f81a8d50
Improve cosmocc / cosmoc++ toolchain scripts
- Get out of the red zone
- Generate --ftrace nops unless -Os is passed
- Intercept -o path to generate .com / .com.dbg appropriately
2023-06-08 14:29:22 -07:00
Justine Tunney
7558549d44
Test m=aarch64 on GitHub Actions 2023-06-05 15:23:00 -07:00
Justine Tunney
4aa1d09b9e
Improve aarch64 native support some more
This change introduces partial support for automating remote testing of
aarch64 binaries on Raspberry Pi and Apple Silicon.
2023-06-04 08:58:47 -07:00
Justine Tunney
bcf9af94bf
Get threads working well on MacOS Arm64
- Now using 10x better GCD semaphores
- We now generate Linux-like thread ids
- We now use fast system clock / sleep libraries
- The APE M1 loader now generates Linux-like stacks
2023-06-04 01:57:10 -07:00
Justine Tunney
1422e96b4e
Introduce native support for MacOS ARM64
There's a new program named ape/ape-m1.c which will be used to build an
embeddable binary that can load ape and elf executables. The support is
mostly working so far, but still chasing down ABI issues.
2023-05-20 04:17:03 -07:00
Justine Tunney
cc1732bc42
Make AARCH64 harder, better, faster, stronger
- Perform some housekeeping on scalar math function code
- Import ARM's Optimized Routines for SIMD string processing
- Upgrade to latest Chromium zlib and enable more SIMD optimizations
2023-05-15 02:15:34 -07:00
Justine Tunney
fd34ef732d
Make considerably more progress on AARCH64
- Utilities like pledge.com now build
- kprintf() will no longer balk at 48-bit addresses
- There's a new aarch64-dbg build mode that should work
- gc() and defer() are mostly pacified; avoid using them on aarch64
- THIRD_PART_STB now has Arm Neon intrinsics for fast image handling
2023-05-12 22:42:57 -07:00
Justine Tunney
414667b1c9
Get TEST_LIBC_STR passing on AARCH64
It's now possible to run commands like:

    make -j8 m=aarch64 o/aarch64/test/libc/str

Which will cross-compile and run the test suites in a qemu-aarch64
binary that's vendored in the third_party/qemu/ folder within your
x86_64 build environment.
2023-05-12 18:09:23 -07:00
Justine Tunney
f312f706f4
Bring MODE=tiny binary sizes down to 20kb minimum
aarch64 binaries start at 4kb.
2023-05-10 04:20:48 -07:00
Justine Tunney
5a455eaa0b
Work on magic numbers for aarch64 2023-05-10 04:20:48 -07:00
Justine Tunney
e5e3cdf447
Get LIBC_RUNTIME and LIBC_CALLS building on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney
ca2860947f
Make progress towards aarch64 build 2023-05-10 04:20:46 -07:00
Gabriel Ravier
f7bccf5513
Fix pledge rpath support for old getdents syscall (#804)
The rpath pledge as currently implemented in cosmopolitan does not
allow for usage of the old getdents syscall (0x4e), which is different
from the newer getdents syscall (0xd9) solely in that it does not
support 64-bit filesystems.

This means that, for example, old statically linked binaries cannot
use `readdir` and other such functions which use this syscall instead
of the more modern one, even though there is no threat in allowing
that syscall alongside the more modern one (except that the binary may
have issues with 64-bit filesystems, but that's a separate problem).

This patch fixes this.
2023-04-17 16:15:01 -07:00
Gavin Hayes
ff9c15f48a
Add APE fexecve() support (#733) 2023-02-22 18:58:23 -08:00
Justine Tunney
4b8874ceb9
Make some simple fixes 2023-02-03 02:09:15 -08:00
Justine Tunney
cf19e3e46a
Fix SO_LINGER on Apple 2023-02-02 20:05:37 -08:00
tkchia
73507d5f4e
Add some definitions for Linux frame buffer/console ioctl APIs (#712)
Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2023-01-01 16:28:03 -08:00
Justine Tunney
120079b0a6
Remove evil constants from cosmopolitan.h
- `I` a.k.a. `_Complex_I` (fixes #705)
- `B0`, `B50`, etc. (fixes #403)
2022-12-17 00:42:45 -08:00
Justine Tunney
ed161b240e
Clean up some code 2022-12-11 14:30:59 -08:00
Justine Tunney
c995838e5c
Make improvements
- Clean up sigaction() code
- Add a port scanner example
- Introduce a ParseCidr() API
- Clean up our futex abstraction code
- Fix a harmless integer overflow in ParseIp()
- Use kernel semaphores on NetBSD to make threads much faster
2022-11-07 02:26:06 -08:00
Justine Tunney
3f0bcdc3ef
Improve cancellations, randomness, and time
- Exhaustively document cancellation points
- Rename SIGCANCEL to SIGTHR just like BSDs
- Further improve POSIX thread cancellations
- Ensure asynchronous cancellations work correctly
- Elevate the quality of getrandom() and getentropy()
- Make futexes cancel correctly on OpenBSD 6.x and 7.x
- Add reboot.com and shutdown.com to examples directory
- Remove underscore prefix from awesome timespec_*() APIs
- Create assertions that help verify our cancellation points
- Remove bad timespec APIs (cmp generalizes eq/ne/gt/gte/lt/lte)
2022-11-05 23:45:32 -07:00
Justine Tunney
022536cab6
Make futexes cancellable by pthreads 2022-11-04 18:36:34 -07:00
Justine Tunney
2278327eba
Implement support for POSIX thread cancellations
This change makes some miracle modifications to the System Five system
call support, which lets us have safe, correct, and atomic handling of
thread cancellations. It all turned out to be cheaper than anticipated
because it wasn't necessary to modify the system call veneers. We were
able to encode the cancellability of each system call into the magnums
found in libc/sysv/syscalls.sh. Since cancellations are so waq, we are
also supporting a lovely Musl Libc mask feature for raising ECANCELED.
2022-11-04 01:04:43 -07:00
Justine Tunney
14d036b68d
Add WSL to test fleet
All tests pass now under WSL2. They should pass under WSL1 too, but only
WSL2 is integrated into the test fleet right now. This change also fills
in some gaps in the error numbers.

Fixes #665
2022-11-02 06:49:42 -07:00
Justine Tunney
f7ff77d865
Make fixes and improvements
- Invent iso8601us() for faster timestamps
- Improve --strace descriptions of sigset_t
- Rebuild the Landlock Make bootstrap binary
- Introduce MODE=sysv for non-Windows builds
- Permit OFD fcntl() locks under pledge(flock)
- redbean can now protect your kernel from ddos
- Have vfork() fallback to sys_fork() not fork()
- Change kmalloc() to not die when out of memory
- Improve documentation for some termios functions
- Rewrite putenv() and friends to conform to POSIX
- Fix linenoise + strace verbosity issue on Windows
- Fix regressions in our ability to show backtraces
- Change redbean SetHeader() to no-op if value is nil
- Improve fcntl() so SQLite locks work in non-WAL mode
- Remove some unnecessary work during fork() on Windows
- Create redbean-based SSL reverse proxy for IPv4 TurfWar
- Fix ape/apeinstall.sh warning when using non-bash shells
- Add ProgramTrustedIp(), and IsTrustedIp() APIs to redbean
- Support $PWD, $UID, $GID, and $EUID in command interpreter
- Introduce experimental JTqFpD APE prefix for non-Windows builds
- Invent blackhole daemon for firewalling IP addresses via UNIX named socket
- Add ProgramTokenBucket(), AcquireToken(), and CountTokens() APIs to redbean
2022-10-19 07:19:19 -07:00
Justine Tunney
60cb435cb4
Implement pthread_atfork()
If threads are being used, then fork() will now acquire and release and
runtime locks so that fork() may be safely used from threads. This also
makes vfork() thread safe, because pthread mutexes will do nothing when
the process is a child of vfork(). More torture tests have been written
to confirm this all works like a charm. Additionally:

- Invent hexpcpy() api
- Rename nsync_malloc_() to kmalloc()
- Complete posix named semaphore implementation
- Make pthread_create() asynchronous signal safe
- Add rm, rmdir, and touch to command interpreter builtins
- Invent sigisprecious() and modify sigset functions to use it
- Add unit tests for posix_spawn() attributes and fix its bugs

One unresolved problem is the reclaiming of *NSYNC waiter memory in the
forked child processes, within apps which have threads waiting on locks
2022-10-16 12:25:13 -07:00
Justine Tunney
5af19b7eed
Make some foss compatibility improvements 2022-10-14 13:59:34 -07:00
Justine Tunney
997ce29ddc
Elevate Windows production worthiness
- SQLite file locking now works on Windows
- SQLite will now use fdatasync() on non-Apple platforms
- Fix Ctrl-C handler on Windows to not crash with TLS
- Signals now work in multithreaded apps on Windows
- fcntl() will now accurately report EINVAL errors
- fcntl() now has excellent --strace logging
- Token bucket replenish now go 100x faster
- *NSYNC cancellations now work on Windows
- Support closefrom() on NetBSD
2022-10-13 13:44:41 -07:00
Justine Tunney
e557058ac8
Improve cosmo's conformance to libc-test
This change addresses various open source compatibility issues, so that
we pass 313/411 of the tests in https://github.com/jart/libc-test where
earlier today we were passing about 30/411 of them, due to header toil.
Please note that Glibc only passes 341/411 so 313 today is pretty good!

- Make the conformance of libc/isystem/ headers nearly perfect
- Import more of the remaining math library routines from Musl
- Fix inconsistencies with type signatures of calls like umask
- Write tests for getpriority/setpriority which work great now
- conform to `struct sockaddr *` on remaining socket functions
- Import a bunch of uninteresting stdlib functions e.g. rand48
- Introduce readdir_r, scandir, pthread_kill, sigsetjmp, etc..

Follow the instructions in our `tool/scripts/cosmocc` toolchain to run
these tests yourself. You use `make CC=cosmocc` on the test repository
2022-10-10 17:52:41 -07:00
Justine Tunney
467a332e38
Introduce sigtimedwait() and sigwaitinfo()
This change also invents sigcountset() and strsignal_r() and improves
the quality of siginfo_t handling.
2022-10-10 07:39:44 -07:00
Justine Tunney
672ccda37c
Clean up some sleep code 2022-10-08 03:00:48 -07:00
Justine Tunney
3b4fcd8575
Fix corner case in Linux stack mappings
We need to make sure no existing mappings exist between the
MAP_GROWSDOWN page and the guard page, since otherwise it's
not going to be able to grow down thus causing difficult to
troubleshoot failures.
2022-10-03 13:43:52 -07:00
Justine Tunney
7549a5755e
Support futexes on FreeBSD 2022-10-02 11:57:13 -07:00
Justine Tunney
acd8900071
Add fexecve() and map O_EXEC to O_PATH on Linux 2022-10-02 09:15:46 -07:00
Justine Tunney
6cc9e08f1b
Introduce cosmocc and cosmoc++ toolchain commands 2022-09-20 04:34:03 -07:00
Justine Tunney
c7a8cd21e9
Improve system call wrappers
This change improves copy_file_range(), sendfile(), splice(), openpty(),
closefrom(), close_range(), fadvise() and posix_fadvise() in addition to
writing tests that confirm things like errno and seeking behavior across
platforms. We now less aggressively polyfill behavior with some of these
functions when the platform support isn't available. Please see:

https://justine.lol/cosmopolitan/functions.html
2022-09-19 15:06:25 -07:00
Justine Tunney
134ffee519
Change support vector to Windows 8+
Doing this makes binaries tinier, since we don't need to have all the
extra code for supporting a 32-bit address space. It also benefits us
because we're able to use WIN32 futexes, which makes locking simpler.

b69f3d2488 is what officially ended our
Windows 7 support. This change is merely a formalization. You can use
old versions of Cosmo now and forevermore if you need Windows 7 since
our repository is hermetic and vendors all its dependencies.

Won't fix #617
2022-09-15 03:55:05 -07:00
Justine Tunney
3f49889841
Make important improvements
- Fix preadv() and pwritev() for old distros
- Introduce _npassert() and _unassert() macros
- Prove that file locks work properly on Windows
- Support fcntl(F_DUPFD_CLOEXEC) on more systems
2022-09-14 22:39:08 -07:00
Justine Tunney
1ad2f530f9
Add script.com and whois.com courtesy of FreeBSD 2022-09-13 20:26:13 -07:00
Justine Tunney
8f5678882d
Make dos errno multimapping linkage tinier
640 bytes for old kDos2Errno table
182 bytes for new kDos2Errno under hello2.com (MODE=fastbuild)
122 bytes for new kDos2Errno under hello2.com (MODE=tiny)
2022-09-13 06:25:50 -07:00
Justine Tunney
b5cb71ab84
Use *NSYNC for POSIX threads locking APIs
Condition variables, barriers, and r/w locks now work very well.
2022-09-11 11:04:50 -07:00
Justine Tunney
e97f1a99cf
Fix pthread stacks with larger guard size 2022-09-09 06:41:22 -07:00
Justine Tunney
2d17ab016c
Perform more low-level code cleanup 2022-09-09 04:07:08 -07:00
Justine Tunney
b73e35c6fa
Improve open source compatibility
This change tunes the default stack size for the outside world to 8mb
while at the same time, reducing Cosmopolitan's default stack size to
64kb. You can override the stack size using STATIC_STACK_SIZE(). Your
build scripts should point to o//ape/public/ape.lds

This change also fixes the definition of SOMAXCONN and removes AF_RDS
since it's not polyfilled and Python 3.11 complained.
2022-09-08 03:19:35 -07:00