Commit graph

411 commits

Author SHA1 Message Date
Justine Tunney
e466dd0553 Add torture test for zipos file descriptors
This change hardens the code for opening /zip/ files using the system
call interface. Thread safety and signal safety has been improved for
file descriptors in general. We now document fixed addresses that are
needed for low level allocations.
2022-06-15 16:29:49 -07:00
Justine Tunney
579080cd4c Improve generation of redbean static headers
This change fixes an issue where Cache-Control could be outputted twice
when Lua code calls SetHeader() from OnHttpRequest().
2022-06-14 22:01:13 -07:00
Justine Tunney
42b34c26f8 Release redbean 2.0.1 2022-06-14 20:21:34 -07:00
Paul Kulchenko
8b13e1acc8
Fixed redbean compression functions (#421) 2022-06-14 17:50:25 -07:00
Paul Kulchenko
58c77a9b1d
Fix recent regression in redbean helpers (#419) 2022-06-14 17:22:51 -07:00
Justine Tunney
a3865ecc3c Make more fixes and improvements
- Fix Makefile flaking due to ZIPOBJ_FLAGS generation
- Make printf() floating point and gdtoa thread safe
- Polish up the runit / runitd programs some more
- Prune some more makefile dependencies
2022-06-13 11:02:13 -07:00
Justine Tunney
4ddfc47d6e Make some more fixups 2022-06-12 09:37:17 -07:00
Justine Tunney
6070a53e89 Make a few fixups 2022-06-12 07:32:14 -07:00
Justine Tunney
517267a577 Reduce MKDEPS.COM latency from 60ms to 18ms 2022-06-11 12:23:47 -07:00
Justine Tunney
e96aceae41 Bump support up to FreeBSD 13 and NetBSD 9.2
These releases are really exciting since they contained the patches we
worked to get upstreamed. It means that their /bin/sh interpreters all
work fine with Actually Portable Executable now.
2022-06-11 10:36:17 -07:00
Justine Tunney
1ae6cfc198 Update build binaries for WSL
If you use WSL then you may want to delete /tmp/ape, o/tmp/ape, and
/usr/bin/ape before running make again. That should resolve issues.
2022-06-11 05:25:03 -07:00
Justine Tunney
c260345e06 Make locks more reliable
This change switches most of the core locks to be re-entrant, in order
to reduce the chance of deadlocking code that does, clever things with
asynchronous signal handlers. This change implements it it in pthreads
so we're one step closer to having a standardized threading primitives
2022-06-11 02:07:20 -07:00
Justine Tunney
5ea618f0af Make clock_gettime() faster on FreeBSD 2022-06-11 00:05:06 -07:00
Justine Tunney
5deda43766 Handle wildcard certificates in redbean 2022-06-10 21:59:14 -07:00
Justine Tunney
c6d8e516b2 Print warning when microbenchmarking w/ powersave
RDTSC on Linux has so much jitter when the CPU is in powersave mode
causing things like microbenchmarks to have a 1000% margin of error
2022-06-10 21:07:03 -07:00
Justine Tunney
af4687cc3f Add Info-ZIP unzip.com 2022-06-10 05:09:11 -07:00
Justine Tunney
f8520e10b2 Add ProgramMaxPayloadSize() API to Redbean
This way we don't have to use the flag and can configure it using
.init.lua instead.
2022-06-10 03:28:46 -07:00
Justine Tunney
2528536cff Fix some plinko documentation 2022-06-10 03:28:24 -07:00
Justine Tunney
ecc8962555 Improve performance of runit.com
We now have O(1) deflates rather than O(n).
2022-06-08 21:52:47 -07:00
Justine Tunney
adac64a52b Reduce makefile dependencies by 10%
The includes in libc/calls/calls.h have now been refactored so that
functions with struct parameters are declared in libc/calls/struct/
2022-06-08 20:01:28 -07:00
Justine Tunney
e011973593 Remove -no-canonical-prefixes flag 2022-05-29 19:14:56 -07:00
Justine Tunney
da6d610056 Use Lua auto buffers when possible 2022-05-29 19:04:30 -07:00
Justine Tunney
13ee75150c Improve redbean plus code size optimizations
This change turns symbol table compression back on using Puff, which
noticeably reduces the size of programs like redbean and Python. The
redbean web server receives some minor API additions for controlling
things like SSL in addition to filling gaps in the documentation.
2022-05-29 08:21:19 -07:00
Justine Tunney
425ff5dff0 Refactor some code
- Write tests for cthreads
- Fix bugs in pe2.com tool
- Fix ASAN issue with GetDosEnviron()
- Consolidate the cthread header files
- Some code size optimizations for MODE=
- Attempted to squash a tls linker warning
- Attempted to get futexes working on FreeBSD
2022-05-28 14:09:21 -07:00
Justine Tunney
de5de19004 Make improvements
- Document redbean's argon2 module
- Fix regressions in cthreads library
- Make testlib work better with threads
- Give the cthreads library lots of love
- Remove some of the stdio assembly code
- Implement getloadavg() across platforms
- Code size optimizations for errnos, etc.
- Only check for signals in main thread on Windows
- Make errnos for dup2 / dup3 consistent with posix

This change also fixes a bug in the argon2 module, where the NUL
terminator was being included in the hash encoded ascii string. This
shouldn't require any database migrations to folks who found this module
and productionized it, since the argon2 library treats it as a c string.
2022-05-28 00:28:09 -07:00
Justine Tunney
cb67223051 Add malloc logging tool
STATIC_YOINK("enable_memory_log");
2022-05-26 23:19:22 -07:00
Justine Tunney
d230a01222 Make build hermetic without shell scripts
- Fix some minor issues in ar.com
- Have execve() look for `ape` command
- Rewrite NT paths using /c/ rather /??/c:/
- Replace broken GCC symlinks with .sym files
- Rewrite $PATH environment variables on startup
- Make $(APE_NO_MODIFY_SELF) the default bootloader
- Add all build command dependencies to build/bootstrap
- Get the repository mostly building from source on non-Linux
2022-05-25 13:55:57 -07:00
Justine Tunney
d44ff6ce1f Make improvements
- Implement openpty()
- Add `--assimilate` flag to APE bootloader
- Restore Linux vDSO clock_gettime() support
- Use `$(APE_NO_MODIFY_SELF)` on more programs
2022-05-24 10:58:48 -07:00
Justine Tunney
cef50f2a6b Write test for stack overflow detection 2022-05-23 16:32:16 -07:00
Justine Tunney
312ed5c67c Fix some issues and do some code cleanup 2022-05-23 10:15:53 -07:00
Justine Tunney
1f229e4efc Use re-entrant locks on stdio 2022-05-22 08:28:33 -07:00
Justine Tunney
4e9662cbc7 Write tests for new APE loader and fix bugs
- Add FreeBSD-specific mmap() flags
- Reduce size of the APE loader from 8kb to 4kb
- Work towards fixing the Makefile build on WSL
- Automate testing of APE no-modify-self behaviors
- Make the ape.S shell script code cleaner and tinier
- Improve the APE sanity check to test behavior better
- Fixed issue with ShowCrashReports() sigaltstack() on BSDs
- Delete symbols for S_MODE magnums which wasted compile time

If you checked out yesterday's APE commit, please run:

    rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape"

Because this change fixes certain aspects of the new ABI. We don't have
automated migrations for APE loader versions yet. Thanks! You can also
download prebuilt binaries here:

- https://justine.lol/ape.elf    (Linux/FreeBSD/NetBSD/OpenBSD)
- https://justine.lol/ape.macho  (Apple)

Install the appropriate one as `/usr/bin/ape`.
2022-05-22 05:45:38 -07:00
Justine Tunney
4245da19e2 Remove some old dead code from ftrace 2022-05-20 04:01:50 -07:00
Justine Tunney
ec2cb88058 Make fixes and improvements
- Document more compiler flags
- Expose new __print_maps() api
- Better overflow checking in mmap()
- Improve the shell example somewhat
- Fix minor runtime bugs regarding stacks
- Make kill() on fork()+execve()'d children work
- Support CLONE_CHILD_CLEARTID for proper joining
- Fix recent possible deadlock regression with --ftrace
2022-05-19 16:57:49 -07:00
Justine Tunney
6e52cba37a Fix stdio regression
This change fixes a nasty regression caused by
80b211e314 which deadlocked.

This change also causes MbedTLS to prefer the ChaCha ciphersuite on
older CPUs that don't have AES hardware instructions.
2022-05-19 00:51:15 -07:00
Justine Tunney
9208c83f7a Make some systemic improvements
- add vdso dump utility
- tests now log stack usage
- rename g_ftrace to __ftrace
- make internal spinlocks go faster
- add conformant c11 atomics library
- function tracing now logs stack usage
- make function call tracing thread safe
- add -X unsecure (no ssl) mode to redbean
- munmap() has more consistent behavior now
- pacify fsync() calls on python unit tests
- make --strace flag work better in redbean
- start minimizing and documenting compiler flags
2022-05-18 16:52:36 -07:00
Justine Tunney
2ad1c9078d Improve dependency generation for libcxx headers
This change also updates redbean to have more informative error messages
based on the ones that are actually showing up in production.
2022-05-17 11:38:35 -07:00
Justine Tunney
3d99ebb68c Add raw parameter to redbean lua compress 2022-05-17 10:41:23 -07:00
Justine Tunney
8bfb70ca3f Add Compress() and Uncompress() to redbean 2022-05-16 16:49:20 -07:00
Justine Tunney
55de4ca6b5 Support thread local storage 2022-05-16 13:20:08 -07:00
Paul Kulchenko
e5e141d9b5
Update redbean unix module documentation for consistency (#409) 2022-05-14 23:25:54 -07:00
Justine Tunney
c446af799e Compress network file transfers 2022-05-14 23:17:22 -07:00
Justine Tunney
54e6f564c1 Let ctrl-c interrupt lua server pages in repl mode 2022-05-14 11:47:16 -07:00
Justine Tunney
80b211e314 Add raw memory visualization tool to redbean
This change introduces a `-W /dev/pts/1` flag to redbean. What it does
is use the mincore() system call to create a dual-screen terminal
display that lets you troubleshoot the virtual address space. This is
useful since page faults are an important thing to consider when using a
forking web server. Now we have a colorful visualization of which pages
are going to fault and which ones are resident in memory.

The memory monitor, if enabled, spawns as a thread that just outputs
ANSI codes to the second terminal in a loop. In order to make this
happen using the new clone() polyfill, stdio is now thread safe.

This change also introduces some new demo pages to redbean. It also
polishes the demos we already have, to look a bit nicer and more
presentable for the upcoming release, with better explanations too.
2022-05-14 04:33:58 -07:00
Justine Tunney
578cb21591 Add syscalls to Blinkenlights and fix bugs 2022-05-13 13:31:21 -07:00
Justine Tunney
4e62cefa6e Add zlib gzip functions
These will now be included in the cosmopolitan.a releases. It took a bit
of time because, these functions depend on heavyweight parts of the libc
that wouldn't be appropriate for the core zlib library to depend upon.

Fixes #345
2022-05-12 12:17:40 -07:00
Justine Tunney
4499f98e76 Add /.args feature to Redbean/Lua/SQLite/Python/QuickJS
You now have some ability to truly make an executable yours, by adding a
`.args` file to the root of the zip structure. If this is specified,
then you'll be overriding the default CLI args.

This will be a great feature for folks who want to distribute their own
apps, using the interpreter executable, but have the executable appears
to be just your app rather than being the interpreter.
2022-05-12 11:04:47 -07:00
Justine Tunney
0f6251f4d2 Make redbean unix.open default to O_RDONLY 2022-05-12 09:43:01 -07:00
Justine Tunney
e7611a8476 Make improvements
- Get threads working on NetBSD
- Get threads working on OpenBSD
- Fix Emacs config for Emacs v28
- Improve --strace logging of sigset_t
- Improve --strace logging of struct stat
- Improve memory safety of DescribeThing functions
- Refactor auto stack allocation into LIBC_RUNTIME
- Introduce shell.com example which works on Windows
- Refactor __strace_thing into DescribeThing functions
- Document the CHECK macros and improve them in NDEBUG mode
- Rewrite MAP_STACK so it uses FreeBSD behavior across platforms
- Deprecate and discourage the use of MAP_GROWSDOWN (it's weird)
2022-05-12 06:45:36 -07:00
Justine Tunney
dd9ab01d25 Revert "Use 64-bit years"
This reverts commit cfc3a953ae.
2022-05-12 06:45:36 -07:00