Commit graph

267 commits

Author SHA1 Message Date
Justine Tunney
893cc06fc2 Fix libunwind stubs 2022-06-26 05:29:25 -07:00
Justine Tunney
5cd0e876bd Scrub double-conversion and add quick wrapper 2022-06-26 05:03:25 -07:00
Justine Tunney
61257d48d4 Make some quick fixes and cleanup 2022-06-26 02:58:36 -07:00
Justine Tunney
fbc053e018 Make fixes and improvements
- Introduce __assert_disable global
- Improve strsignal() thread safety
- Make system call tracing thread safe
- Fix SO_RCVTIMEO / SO_SNDTIMEO on Windows
- Refactor DescribeFoo() functions into one place
- Fix fork() on Windows when TLS and MAP_STACK exist
- Round upwards in setsockopt(SO_RCVTIMEO) on Windows
- Disable futexes on OpenBSD which seem extremely broken
- Implement a better kludge for monotonic time on Windows
2022-06-25 21:09:09 -07:00
Justine Tunney
f79b2fe23a Change assumptions about PROT_EXEC implying PROT_READ
On all operating systems tested so far, PROT_EXEC without PROT_READ
always makes memory readable. This turned out to not be the case on
Chromebooks, which likely means they have the capability of running
programs which aren't able to read their own code.
2022-06-23 14:13:19 -07:00
Justine Tunney
a4601a24d3 Perform some code cleanup 2022-06-23 10:21:07 -07:00
Justine Tunney
3fc23c6cbc Add getcontext() and setcontext()
Fixes #413
2022-06-23 06:01:44 -07:00
Theta Nil
2b54f1bcf6
Migrate from Travis to GitHub Actions (#441) 2022-06-23 04:39:00 -07:00
Justine Tunney
1235ad6006 Add temporary stubs for libunwind 2022-06-22 07:11:08 -07:00
Justine Tunney
9545062175 Add pthread tls api 2022-06-22 07:08:17 -07:00
Justine Tunney
fc097ac275 Add unix domain socket support to redbean 2022-06-22 03:04:25 -07:00
Justine Tunney
f317a47cd8 Fixup and polish chibicc some more
Fixes #433
2022-06-20 04:32:25 -07:00
Justine Tunney
d5312b60f7 Make improvements to locking
This change makes pthread_mutex_lock() as fast as _spinlock() by
default. Thread instability issues on NetBSD have been resolved.
Improvements made to gdtoa thread code. Crash reporting will now
synchronize between threads in a slightly better way.
2022-06-19 01:30:12 -07:00
Justine Tunney
c1cfca8ae1 Make fixes and improvements
- Polyfill UTIME_OMIT on XNU
- Refactor Lua build code so it's better
- Add unix module to lua.com (Discord request)
- Add unix.utimensat() and unix.futimens() to redbean
- Avoid creating double slash path in linenoise (#428)
- Remove double slashes in NT paths automatically (#428)
- Make strerror() smarter about showing NT errors (#428)

Fixes #428
2022-06-18 01:46:12 -07:00
Justine Tunney
d0d9cd38c5 Write tests and fixes for utimensat() 2022-06-17 02:43:00 -07:00
Justine Tunney
c06ffd458c Write some lock contention tests 2022-06-16 09:06:09 -07:00
Justine Tunney
e466dd0553 Add torture test for zipos file descriptors
This change hardens the code for opening /zip/ files using the system
call interface. Thread safety and signal safety has been improved for
file descriptors in general. We now document fixed addresses that are
needed for low level allocations.
2022-06-15 16:29:49 -07:00
Justine Tunney
1c387727fe Add long double printf formatting support
You can now easily print numbers that have 80 bits of precision. This
change also imports many of the test vectors from the gdtoa codebase.
2022-06-14 02:18:03 -07:00
Justine Tunney
a3865ecc3c Make more fixes and improvements
- Fix Makefile flaking due to ZIPOBJ_FLAGS generation
- Make printf() floating point and gdtoa thread safe
- Polish up the runit / runitd programs some more
- Prune some more makefile dependencies
2022-06-13 11:02:13 -07:00
Justine Tunney
3c285337a2 Fix %c with nul character
Fixes #417
2022-06-12 22:28:19 -07:00
Justine Tunney
8b72490431 Make mutex calling code 10x tinier
Calls to lock/unlock functions are now NOPs by default. The first time
clone() is called, they get turned into CALL instructions. Doing this
caused funcctions like fputc() to shrink from 85 bytes to 45+4 bytes.
Since the ANSI solution of `(__threaded && lock())` inlines os much
superfluous binary content into functions all over the place.
2022-06-12 20:17:12 -07:00
Justine Tunney
8cdec62f5b Apply even more fixups
- Finish cleaning up the stdio unlocked APIs
- Make __cxa_finalize() properly thread safe
- Don't log locks if threads aren't being used
- Add some more mutex guards to places using _mmi
- Specific lock names now appear in the --ftrace logs
- Fix mkdeps.com generating invalid Makefiles sometimes
- Simplify and fix bugs in the test runner infrastructure
- Fix issue where sometimes some functions wouldn't be logged
2022-06-12 11:57:00 -07:00
Justine Tunney
4ddfc47d6e Make some more fixups 2022-06-12 09:37:17 -07:00
Justine Tunney
6070a53e89 Make a few fixups 2022-06-12 07:32:14 -07:00
Justine Tunney
e96aceae41 Bump support up to FreeBSD 13 and NetBSD 9.2
These releases are really exciting since they contained the patches we
worked to get upstreamed. It means that their /bin/sh interpreters all
work fine with Actually Portable Executable now.
2022-06-11 10:36:17 -07:00
Justine Tunney
c260345e06 Make locks more reliable
This change switches most of the core locks to be re-entrant, in order
to reduce the chance of deadlocking code that does, clever things with
asynchronous signal handlers. This change implements it it in pthreads
so we're one step closer to having a standardized threading primitives
2022-06-11 02:07:20 -07:00
Justine Tunney
5ea618f0af Make clock_gettime() faster on FreeBSD 2022-06-11 00:05:06 -07:00
Justine Tunney
c6d8e516b2 Print warning when microbenchmarking w/ powersave
RDTSC on Linux has so much jitter when the CPU is in powersave mode
causing things like microbenchmarks to have a 1000% margin of error
2022-06-10 21:07:03 -07:00
Justine Tunney
adac64a52b Reduce makefile dependencies by 10%
The includes in libc/calls/calls.h have now been refactored so that
functions with struct parameters are declared in libc/calls/struct/
2022-06-08 20:01:28 -07:00
Justine Tunney
701564de19 Perform better fixups of NT paths in execve()
This change ensures we do a better job translating /c/foo.bar paths into
c:/foo.bar paths on Windows when generating the CreateProcess() cmd line
thus fixing a regression that happened in the last two months when using
the help() feature of Actually Portable Python in the CMD.EXE shell.
2022-06-08 18:27:19 -07:00
Justine Tunney
da6d610056 Use Lua auto buffers when possible 2022-05-29 19:04:30 -07:00
Justine Tunney
13ee75150c Improve redbean plus code size optimizations
This change turns symbol table compression back on using Puff, which
noticeably reduces the size of programs like redbean and Python. The
redbean web server receives some minor API additions for controlling
things like SSL in addition to filling gaps in the documentation.
2022-05-29 08:21:19 -07:00
Justine Tunney
425ff5dff0 Refactor some code
- Write tests for cthreads
- Fix bugs in pe2.com tool
- Fix ASAN issue with GetDosEnviron()
- Consolidate the cthread header files
- Some code size optimizations for MODE=
- Attempted to squash a tls linker warning
- Attempted to get futexes working on FreeBSD
2022-05-28 14:09:21 -07:00
Justine Tunney
de5de19004 Make improvements
- Document redbean's argon2 module
- Fix regressions in cthreads library
- Make testlib work better with threads
- Give the cthreads library lots of love
- Remove some of the stdio assembly code
- Implement getloadavg() across platforms
- Code size optimizations for errnos, etc.
- Only check for signals in main thread on Windows
- Make errnos for dup2 / dup3 consistent with posix

This change also fixes a bug in the argon2 module, where the NUL
terminator was being included in the hash encoded ascii string. This
shouldn't require any database migrations to folks who found this module
and productionized it, since the argon2 library treats it as a c string.
2022-05-28 00:28:09 -07:00
Justine Tunney
cb67223051 Add malloc logging tool
STATIC_YOINK("enable_memory_log");
2022-05-26 23:19:22 -07:00
Justine Tunney
10b97ca630 Add test for sendfile() and reduce branches 2022-05-25 22:29:10 -07:00
Justine Tunney
8f12cd980d Fix execution bug w/o binfmt_misc
The longjmp relocation in the master boot record coincidentally had a
quote character in it, which caused the master boot record to be used
when interpreter by the shell. The solution, is to move the grub stub
below the long mode loader so the real mode loader relocation is near
the master boot record. This change includes a regression test.
2022-05-25 17:24:02 -07:00
Justine Tunney
d230a01222 Make build hermetic without shell scripts
- Fix some minor issues in ar.com
- Have execve() look for `ape` command
- Rewrite NT paths using /c/ rather /??/c:/
- Replace broken GCC symlinks with .sym files
- Rewrite $PATH environment variables on startup
- Make $(APE_NO_MODIFY_SELF) the default bootloader
- Add all build command dependencies to build/bootstrap
- Get the repository mostly building from source on non-Linux
2022-05-25 13:55:57 -07:00
Justine Tunney
d44ff6ce1f Make improvements
- Implement openpty()
- Add `--assimilate` flag to APE bootloader
- Restore Linux vDSO clock_gettime() support
- Use `$(APE_NO_MODIFY_SELF)` on more programs
2022-05-24 10:58:48 -07:00
Justine Tunney
cef50f2a6b Write test for stack overflow detection 2022-05-23 16:32:16 -07:00
Justine Tunney
99e67c348b Reduce Makefile dependencies by 4% 2022-05-23 15:07:01 -07:00
Justine Tunney
312ed5c67c Fix some issues and do some code cleanup 2022-05-23 10:15:53 -07:00
Justine Tunney
1f229e4efc Use re-entrant locks on stdio 2022-05-22 08:28:33 -07:00
Justine Tunney
4e9662cbc7 Write tests for new APE loader and fix bugs
- Add FreeBSD-specific mmap() flags
- Reduce size of the APE loader from 8kb to 4kb
- Work towards fixing the Makefile build on WSL
- Automate testing of APE no-modify-self behaviors
- Make the ape.S shell script code cleaner and tinier
- Improve the APE sanity check to test behavior better
- Fixed issue with ShowCrashReports() sigaltstack() on BSDs
- Delete symbols for S_MODE magnums which wasted compile time

If you checked out yesterday's APE commit, please run:

    rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape"

Because this change fixes certain aspects of the new ABI. We don't have
automated migrations for APE loader versions yet. Thanks! You can also
download prebuilt binaries here:

- https://justine.lol/ape.elf    (Linux/FreeBSD/NetBSD/OpenBSD)
- https://justine.lol/ape.macho  (Apple)

Install the appropriate one as `/usr/bin/ape`.
2022-05-22 05:45:38 -07:00
Justine Tunney
db0d8dd806 Support Linux binfmt_misc and APE loading on Apple
The "no modify self" variant of Actually Portable Executable is now
supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then
ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file
which are installed on the fly to ${TMPDIR:-/tmp}, which enables us
launch the executable, without needing to copy the whole executable

To prevent it from copying a tiny executable to your temp directory
you need to install the `ape` command (renamed from ape-loader), to
a system path. For example:

    # FreeBSD / NetBSD / OpenBSD
    make -j8 o//ape/ape
    cp o//ape/ape /usr/bin/ape

    # Mac OS
    # make -j8 o//ape/ape.macho
    curl https://justine.lol/ape.macho >/usr/bin/ape
    chmod +x /usr/bin/ape

On Linux you can get even more performance with the new binfmt_misc
support which makes launching non-modifying APE binaries as fast as
launching ELF executables. Running the following command:

    # Linux
    ape/apeinstall.sh

Will copy APE loader to /usr/bin/ape and register with binfmt_misc
Lastly, this change also fixes a really interesting race condition
with OpenBSD thread joining.
2022-05-21 09:28:25 -07:00
Justine Tunney
7838edae88 Fix many thread and file descriptor issues on Windows
The greenbean web server now works nearly perfectly on Windows with over
1000 threads. But some synchronization issues still remain which prevent
us from going over nine thousand.
2022-05-20 20:57:19 -07:00
Justine Tunney
ec2cb88058 Make fixes and improvements
- Document more compiler flags
- Expose new __print_maps() api
- Better overflow checking in mmap()
- Improve the shell example somewhat
- Fix minor runtime bugs regarding stacks
- Make kill() on fork()+execve()'d children work
- Support CLONE_CHILD_CLEARTID for proper joining
- Fix recent possible deadlock regression with --ftrace
2022-05-19 16:57:49 -07:00
Justine Tunney
6e52cba37a Fix stdio regression
This change fixes a nasty regression caused by
80b211e314 which deadlocked.

This change also causes MbedTLS to prefer the ChaCha ciphersuite on
older CPUs that don't have AES hardware instructions.
2022-05-19 00:51:15 -07:00
Justine Tunney
9208c83f7a Make some systemic improvements
- add vdso dump utility
- tests now log stack usage
- rename g_ftrace to __ftrace
- make internal spinlocks go faster
- add conformant c11 atomics library
- function tracing now logs stack usage
- make function call tracing thread safe
- add -X unsecure (no ssl) mode to redbean
- munmap() has more consistent behavior now
- pacify fsync() calls on python unit tests
- make --strace flag work better in redbean
- start minimizing and documenting compiler flags
2022-05-18 16:52:36 -07:00
Justine Tunney
3d99ebb68c Add raw parameter to redbean lua compress 2022-05-17 10:41:23 -07:00