Commit graph

752 commits

Author SHA1 Message Date
Justine Tunney
517267a577 Reduce MKDEPS.COM latency from 60ms to 18ms 2022-06-11 12:23:47 -07:00
Justine Tunney
e96aceae41 Bump support up to FreeBSD 13 and NetBSD 9.2
These releases are really exciting since they contained the patches we
worked to get upstreamed. It means that their /bin/sh interpreters all
work fine with Actually Portable Executable now.
2022-06-11 10:36:17 -07:00
Justine Tunney
1ae6cfc198 Update build binaries for WSL
If you use WSL then you may want to delete /tmp/ape, o/tmp/ape, and
/usr/bin/ape before running make again. That should resolve issues.
2022-06-11 05:25:03 -07:00
Justine Tunney
e32288d4ea Fix WSL regression with APE loader
Microsoft's version of Linux requires that the ELF program headers say
that the executable has 4096 byte alignment. However, it doesn't force
us to pad the binary with NOPs to a page-aligned size.
2022-06-11 05:17:26 -07:00
Justine Tunney
c260345e06 Make locks more reliable
This change switches most of the core locks to be re-entrant, in order
to reduce the chance of deadlocking code that does, clever things with
asynchronous signal handlers. This change implements it it in pthreads
so we're one step closer to having a standardized threading primitives
2022-06-11 02:07:20 -07:00
Justine Tunney
5ea618f0af Make clock_gettime() faster on FreeBSD 2022-06-11 00:05:06 -07:00
Justine Tunney
5deda43766 Handle wildcard certificates in redbean 2022-06-10 21:59:14 -07:00
Justine Tunney
c6d8e516b2 Print warning when microbenchmarking w/ powersave
RDTSC on Linux has so much jitter when the CPU is in powersave mode
causing things like microbenchmarks to have a 1000% margin of error
2022-06-10 21:07:03 -07:00
Justine Tunney
41c86fe86b Fix build error in MODE=dbg 2022-06-10 20:25:29 -07:00
Justine Tunney
8a967ae6fe Fix header dependencies in third_party/make 2022-06-10 13:07:11 -07:00
Justine Tunney
af4687cc3f Add Info-ZIP unzip.com 2022-06-10 05:09:11 -07:00
Justine Tunney
1531f1a4a7 Add CTRL-U shortcut to QuickJS REPL 2022-06-10 03:39:05 -07:00
Justine Tunney
fde939291d Fix an UBSAN warning 2022-06-10 03:33:05 -07:00
Justine Tunney
f8520e10b2 Add ProgramMaxPayloadSize() API to Redbean
This way we don't have to use the flag and can configure it using
.init.lua instead.
2022-06-10 03:28:46 -07:00
Justine Tunney
2528536cff Fix some plinko documentation 2022-06-10 03:28:24 -07:00
Justine Tunney
a41669dec6 Make dlmalloc a little faster
This change also documents the libc arena allocator.
2022-06-09 21:25:21 -07:00
Justine Tunney
fa1e8a3e65 Make emacs not croak when editing dlmalloc 2022-06-09 17:45:30 -07:00
Justine Tunney
3c7ae0fc72 Add w3c html tidy 2022-06-09 06:51:08 -07:00
Justine Tunney
ecc8962555 Improve performance of runit.com
We now have O(1) deflates rather than O(n).
2022-06-08 21:52:47 -07:00
Justine Tunney
e1a40783da Fix fadvise() on Windows 2022-06-08 20:33:13 -07:00
Justine Tunney
adac64a52b Reduce makefile dependencies by 10%
The includes in libc/calls/calls.h have now been refactored so that
functions with struct parameters are declared in libc/calls/struct/
2022-06-08 20:01:28 -07:00
Justine Tunney
4e7ce1538c Fix linenoise flickering on CTRL+L 2022-06-08 18:40:51 -07:00
Justine Tunney
701564de19 Perform better fixups of NT paths in execve()
This change ensures we do a better job translating /c/foo.bar paths into
c:/foo.bar paths on Windows when generating the CreateProcess() cmd line
thus fixing a regression that happened in the last two months when using
the help() feature of Actually Portable Python in the CMD.EXE shell.
2022-06-08 18:27:19 -07:00
Justine Tunney
cfb5d3e406 Fix issue with --strace log 2022-06-08 17:20:19 -07:00
Justine Tunney
e011973593 Remove -no-canonical-prefixes flag 2022-05-29 19:14:56 -07:00
Justine Tunney
da6d610056 Use Lua auto buffers when possible 2022-05-29 19:04:30 -07:00
Justine Tunney
13ee75150c Improve redbean plus code size optimizations
This change turns symbol table compression back on using Puff, which
noticeably reduces the size of programs like redbean and Python. The
redbean web server receives some minor API additions for controlling
things like SSL in addition to filling gaps in the documentation.
2022-05-29 08:21:19 -07:00
Justine Tunney
425ff5dff0 Refactor some code
- Write tests for cthreads
- Fix bugs in pe2.com tool
- Fix ASAN issue with GetDosEnviron()
- Consolidate the cthread header files
- Some code size optimizations for MODE=
- Attempted to squash a tls linker warning
- Attempted to get futexes working on FreeBSD
2022-05-28 14:09:21 -07:00
Gautham
909e54510d
Fix Python test_repl in MODE=dbg (#412) 2022-05-28 05:50:42 -07:00
Justine Tunney
de5de19004 Make improvements
- Document redbean's argon2 module
- Fix regressions in cthreads library
- Make testlib work better with threads
- Give the cthreads library lots of love
- Remove some of the stdio assembly code
- Implement getloadavg() across platforms
- Code size optimizations for errnos, etc.
- Only check for signals in main thread on Windows
- Make errnos for dup2 / dup3 consistent with posix

This change also fixes a bug in the argon2 module, where the NUL
terminator was being included in the hash encoded ascii string. This
shouldn't require any database migrations to folks who found this module
and productionized it, since the argon2 library treats it as a c string.
2022-05-28 00:28:09 -07:00
Justine Tunney
cb67223051 Add malloc logging tool
STATIC_YOINK("enable_memory_log");
2022-05-26 23:19:22 -07:00
Gautham
7e9fb0a9f1
Move importlib functions to within C (#408)
This offers a 10% speedup in Python startup time. It also
makes debugging using cosmopolitan tooling easier.
2022-05-26 23:20:59 -07:00
Justine Tunney
10b97ca630 Add test for sendfile() and reduce branches 2022-05-25 22:29:10 -07:00
Justine Tunney
8f12cd980d Fix execution bug w/o binfmt_misc
The longjmp relocation in the master boot record coincidentally had a
quote character in it, which caused the master boot record to be used
when interpreter by the shell. The solution, is to move the grub stub
below the long mode loader so the real mode loader relocation is near
the master boot record. This change includes a regression test.
2022-05-25 17:24:02 -07:00
Justine Tunney
1a29424c52 Fix ARG_MAX with chibicc documentation generator 2022-05-25 14:06:11 -07:00
Justine Tunney
d230a01222 Make build hermetic without shell scripts
- Fix some minor issues in ar.com
- Have execve() look for `ape` command
- Rewrite NT paths using /c/ rather /??/c:/
- Replace broken GCC symlinks with .sym files
- Rewrite $PATH environment variables on startup
- Make $(APE_NO_MODIFY_SELF) the default bootloader
- Add all build command dependencies to build/bootstrap
- Get the repository mostly building from source on non-Linux
2022-05-25 13:55:57 -07:00
Justine Tunney
d44ff6ce1f Make improvements
- Implement openpty()
- Add `--assimilate` flag to APE bootloader
- Restore Linux vDSO clock_gettime() support
- Use `$(APE_NO_MODIFY_SELF)` on more programs
2022-05-24 10:58:48 -07:00
Justine Tunney
cef50f2a6b Write test for stack overflow detection 2022-05-23 16:32:16 -07:00
Justine Tunney
99e67c348b Reduce Makefile dependencies by 4% 2022-05-23 15:07:01 -07:00
Justine Tunney
e8632cfc30 Revert recent change to ASAN module 2022-05-23 13:30:16 -07:00
Justine Tunney
312ed5c67c Fix some issues and do some code cleanup 2022-05-23 10:15:53 -07:00
Justine Tunney
1f229e4efc Use re-entrant locks on stdio 2022-05-22 08:28:33 -07:00
Justine Tunney
4e9662cbc7 Write tests for new APE loader and fix bugs
- Add FreeBSD-specific mmap() flags
- Reduce size of the APE loader from 8kb to 4kb
- Work towards fixing the Makefile build on WSL
- Automate testing of APE no-modify-self behaviors
- Make the ape.S shell script code cleaner and tinier
- Improve the APE sanity check to test behavior better
- Fixed issue with ShowCrashReports() sigaltstack() on BSDs
- Delete symbols for S_MODE magnums which wasted compile time

If you checked out yesterday's APE commit, please run:

    rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape"

Because this change fixes certain aspects of the new ABI. We don't have
automated migrations for APE loader versions yet. Thanks! You can also
download prebuilt binaries here:

- https://justine.lol/ape.elf    (Linux/FreeBSD/NetBSD/OpenBSD)
- https://justine.lol/ape.macho  (Apple)

Install the appropriate one as `/usr/bin/ape`.
2022-05-22 05:45:38 -07:00
ProducerMatt
056dc5f554
Extend quotas for various slow tests (#411) 2022-05-21 20:13:01 -07:00
Justine Tunney
db0d8dd806 Support Linux binfmt_misc and APE loading on Apple
The "no modify self" variant of Actually Portable Executable is now
supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then
ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file
which are installed on the fly to ${TMPDIR:-/tmp}, which enables us
launch the executable, without needing to copy the whole executable

To prevent it from copying a tiny executable to your temp directory
you need to install the `ape` command (renamed from ape-loader), to
a system path. For example:

    # FreeBSD / NetBSD / OpenBSD
    make -j8 o//ape/ape
    cp o//ape/ape /usr/bin/ape

    # Mac OS
    # make -j8 o//ape/ape.macho
    curl https://justine.lol/ape.macho >/usr/bin/ape
    chmod +x /usr/bin/ape

On Linux you can get even more performance with the new binfmt_misc
support which makes launching non-modifying APE binaries as fast as
launching ELF executables. Running the following command:

    # Linux
    ape/apeinstall.sh

Will copy APE loader to /usr/bin/ape and register with binfmt_misc
Lastly, this change also fixes a really interesting race condition
with OpenBSD thread joining.
2022-05-21 09:28:25 -07:00
Justine Tunney
7838edae88 Fix many thread and file descriptor issues on Windows
The greenbean web server now works nearly perfectly on Windows with over
1000 threads. But some synchronization issues still remain which prevent
us from going over nine thousand.
2022-05-20 20:57:19 -07:00
Justine Tunney
96781d0679 Fix mistake with previous commit 2022-05-20 15:08:02 -07:00
Justine Tunney
f4fc16754a Fix build invalidation issue with GNU Make
The `HDRS` and/or `INCS` variables need to be defined in order for
`mkdeps.com` to compute build dependencies for `o//depend`. We're using
INCS here since makeint.h is the evil kind of header that isn't actually
defining a library interface but rather is a hodgepodge of common code.
2022-05-20 14:20:54 -07:00
Justine Tunney
8141988e92 Shave 4kb off default mode binaries 2022-05-20 08:18:20 -07:00
Gautham
fb4382e9a4
os.realname variable for correct USER_SITE value (#410)
In site.py, Python uses os.name to decide where the USER_SITE (ie the
folder containing the user's locally installed packages) is located.
With cosmo we have set os.name as "posix" always, so we use a new
os.realname to decide the USER_SITE location.
2022-05-20 08:17:10 -07:00