Commit graph

318 commits

Author SHA1 Message Date
Justine Tunney
3f0bcdc3ef
Improve cancellations, randomness, and time
- Exhaustively document cancellation points
- Rename SIGCANCEL to SIGTHR just like BSDs
- Further improve POSIX thread cancellations
- Ensure asynchronous cancellations work correctly
- Elevate the quality of getrandom() and getentropy()
- Make futexes cancel correctly on OpenBSD 6.x and 7.x
- Add reboot.com and shutdown.com to examples directory
- Remove underscore prefix from awesome timespec_*() APIs
- Create assertions that help verify our cancellation points
- Remove bad timespec APIs (cmp generalizes eq/ne/gt/gte/lt/lte)
2022-11-05 23:45:32 -07:00
Justine Tunney
37d40e087f
Ignore SIGSYS on BSD by default 2022-11-03 09:32:12 -07:00
Justine Tunney
5e60e5ad10
Release redbean v2.2 2022-11-02 09:52:54 -07:00
Justine Tunney
6b06a8176d
Fix some glitches in redbean
This change includes a fix to Fetch() where an out of bounds memory read
could happen, when the reverse proxied endpoint omits the content-length
header. This caused a bunch of NUL chars to appear on TurfWar's /statusz
since it wouldn't actually overrun the buffer, and if it did it would've
been caught by MODE=asan builds.
2022-11-02 09:46:34 -07:00
Justine Tunney
e522aa3a07
Make more threading improvements
- ASAN memory morgue is now lockless
- Make C11 atomics header more portable
- Rewrote pthread keys support to be lockless
- Simplify Python's unicode table unpacking code
- Make crash report write(2) closer to being atomic
- Make it possible to strace/ftrace a single thread
- ASAN now checks nul-terminated strings fast and properly
- Windows fork() now restores TLS memory of calling thread
2022-11-01 23:28:26 -07:00
Paul Kulchenko
d7b88734cd
Make redbean compress w/o explicit content-type (#671) 2022-11-01 23:15:20 -07:00
Justine Tunney
da336b3ea8
Make some more fixes to prod 2022-10-19 13:10:24 -07:00
Justine Tunney
69bee64a59
Make some last minute production changes 2022-10-19 10:00:29 -07:00
Justine Tunney
f7ff77d865
Make fixes and improvements
- Invent iso8601us() for faster timestamps
- Improve --strace descriptions of sigset_t
- Rebuild the Landlock Make bootstrap binary
- Introduce MODE=sysv for non-Windows builds
- Permit OFD fcntl() locks under pledge(flock)
- redbean can now protect your kernel from ddos
- Have vfork() fallback to sys_fork() not fork()
- Change kmalloc() to not die when out of memory
- Improve documentation for some termios functions
- Rewrite putenv() and friends to conform to POSIX
- Fix linenoise + strace verbosity issue on Windows
- Fix regressions in our ability to show backtraces
- Change redbean SetHeader() to no-op if value is nil
- Improve fcntl() so SQLite locks work in non-WAL mode
- Remove some unnecessary work during fork() on Windows
- Create redbean-based SSL reverse proxy for IPv4 TurfWar
- Fix ape/apeinstall.sh warning when using non-bash shells
- Add ProgramTrustedIp(), and IsTrustedIp() APIs to redbean
- Support $PWD, $UID, $GID, and $EUID in command interpreter
- Introduce experimental JTqFpD APE prefix for non-Windows builds
- Invent blackhole daemon for firewalling IP addresses via UNIX named socket
- Add ProgramTokenBucket(), AcquireToken(), and CountTokens() APIs to redbean
2022-10-19 07:19:19 -07:00
tkchia
7a06760e6f
Rename LINUX to _HOSTLINUX etc. to reduce clashes (#655)
Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-10-10 20:31:25 -07:00
Justine Tunney
e557058ac8
Improve cosmo's conformance to libc-test
This change addresses various open source compatibility issues, so that
we pass 313/411 of the tests in https://github.com/jart/libc-test where
earlier today we were passing about 30/411 of them, due to header toil.
Please note that Glibc only passes 341/411 so 313 today is pretty good!

- Make the conformance of libc/isystem/ headers nearly perfect
- Import more of the remaining math library routines from Musl
- Fix inconsistencies with type signatures of calls like umask
- Write tests for getpriority/setpriority which work great now
- conform to `struct sockaddr *` on remaining socket functions
- Import a bunch of uninteresting stdlib functions e.g. rand48
- Introduce readdir_r, scandir, pthread_kill, sigsetjmp, etc..

Follow the instructions in our `tool/scripts/cosmocc` toolchain to run
these tests yourself. You use `make CC=cosmocc` on the test repository
2022-10-10 17:52:41 -07:00
Justine Tunney
7ae556463a
Rename rand64() to _rand64() 2022-10-10 04:12:06 -07:00
Justine Tunney
1c0adb556d
Release redbean v2.1 2022-10-08 09:03:51 -07:00
Justine Tunney
b46f763ee7
Permit multiple certs with same common name
This is needed to support Cloudflare interop.
2022-09-27 18:17:04 -07:00
Justine Tunney
2cc1d5ac4c
Improve redbean method=get parameter handling 2022-09-19 20:04:39 -07:00
Justine Tunney
3665232d66
Release redbean 2.0.19 2022-09-17 02:18:19 -07:00
Justine Tunney
775944a2d0
Fix important bugs in redbean
This change upgrades to the latest Chromium Zlib, fixes bugs in redbean,
and introduces better support for reverse proxies like Cloudflare. This
change improves the security of redbean and it's recommended that users
upgrade to the release that'll follow. This change also updates the docs
to clarify how to use the security tools redbean provides e.g. pledge(),
unveil(), and the MODE=asan builds which improve memory safety.
2022-09-17 02:18:06 -07:00
Justine Tunney
6f7d0cb1c3
Pay off more technical debt
This makes breaking changes to add underscores to many non-standard
function names provided by the c library. MODE=tiny is now tinier and
we now use smaller locks that are better for tiny apps in this mode.
Some headers have been renamed to be in the same folder as the build
package, so it'll be easier to know which build dependency is needed.
Certain old misguided interfaces have been removed. Intel intrinsics
headers are now listed in libc/isystem (but not in the amalgamation)
to help further improve open source compatibility. Header complexity
has also been reduced. Lastly, more shell scripts are now available.
2022-09-12 23:36:56 -07:00
Justine Tunney
b5cb71ab84
Use *NSYNC for POSIX threads locking APIs
Condition variables, barriers, and r/w locks now work very well.
2022-09-11 11:04:50 -07:00
Justine Tunney
155b378a39
Tidy up the threading implementation
The organization of the source files is now much more rational.
Old experiments that didn't work out are now deleted. Naming of
things like files is now more intuitive.
2022-09-10 02:56:25 -07:00
Paul Kulchenko
5140897c27
Update redbean to include params in slash redirect (#604) 2022-09-08 19:14:54 -07:00
Justine Tunney
9f963dc597
Clean up some of the threading code 2022-09-08 12:31:56 -07:00
Justine Tunney
55c6297e13
Make more compatibility improvements 2022-09-06 07:04:13 -07:00
Justine Tunney
6a04bc3318
Release redbean 2.0.18 2022-09-04 06:57:59 -07:00
Paul Kulchenko
33b5b5b312
Remove path escaping from LaunchBrowser (#584)
This is so you can have `?query=args`.
2022-08-31 22:14:06 -07:00
Paul Kulchenko
164e2ab27b
Rename MapContentType to ProgramContentType (#570) 2022-08-24 09:35:17 -07:00
Paul Kulchenko
d2cec808dc
Fix redbean to check for file read result when -F option is used (#569) 2022-08-23 20:54:51 -07:00
Paul Kulchenko
91a3252434
Add MapContentType to redbean (#566) 2022-08-22 22:02:58 -07:00
Paul Kulchenko
04cd62c319
Update redbean GetResponseBody to take response compression into account (#561) 2022-08-21 22:26:41 -07:00
Paul Kulchenko
e204fc0820
Update redbean If-Modified-Since check to accept same timestamp (#562) 2022-08-21 22:03:30 -07:00
Justine Tunney
14e3025749 Release redbean 2.0.17 2022-08-21 18:35:53 -07:00
Justine Tunney
51fa5535ac Fix regression in Fetch() 2022-08-21 18:32:57 -07:00
Justine Tunney
bea0b55576 Fix If-Modified-Since regression 2022-08-21 18:05:05 -07:00
Paul Kulchenko
a51edaa0db
Remove .heartbeat.lua from redbean (#554)
It's been replaced by `OnServerHeartbeat`.
2022-08-20 01:31:58 -07:00
Justine Tunney
7ab15e0b23 Add .PLEDGE/.CPU/.MEMORY/etc. to Landlock Make 1.2 2022-08-14 20:16:44 -07:00
Justine Tunney
ead3fc2b31 Fix Landlock Make so it can read pattern rule vars
It turned out that specifying all SRCS and INCS as dependencies on the
pattern rules for all headers, caused `make` memory usage to skyrocket
from 40mb ot 160mb. This change also reduces the build graph another 4%.
2022-08-13 17:23:05 -07:00
Justine Tunney
7cf66bc161 Prevent Make from talking to public Internet
This change introduces the nointernet() function which may be called to
prevent a process and its descendants from communicating with publicly
routable Internet addresses. GNU Make has been modified to always call
this function. In the future Landlock Make will have a way to whitelist
subnets to override this behavior, or disable it entirely. Support is
available for Linux only. Our firewall does not require root access.

Calling nointernet() will return control to the caller inside a new
process that has a SECCOMP BPF filter installed, which traps network
related system calls. Your original process then becomes a permanent
ptrace() supervisor that monitors all processes and threads descending
from the returned child. Whenever a networking system call happens the
kernel will stop the process and wakes up the monitor, which then peeks
into the child memory to read the sockaddr_in to determine if it's ok.

The downside to doing this is that there can be only one supervisor at a
time using ptrace() on a process. So this firewall won't be enabled if
you run make under strace or inside gdb. It also makes testing tricky.
2022-08-12 21:51:39 -07:00
Justine Tunney
8a0a2c0c36 Fold LIBC_RAND into LIBC_STDIO/TINYMATH/INTRIN 2022-08-11 12:32:00 -07:00
Justine Tunney
05b8f82371 Fold LIBC_BITS into LIBC_INTRIN 2022-08-11 12:13:18 -07:00
Justine Tunney
625aa365f1 Refactor pledge() to be more configurable
The earlier iterations did too much guesswork when it came to things
like stderr logging and syscall origin verification. This change will
make things more conformant to existing practices. The __pledge_mode
extension now can be configured in a better way.

There's also a new `-q` flag added to pledge.com, e.g.

    o//tool/build/pledge.com -qv. ls

Is a good way to disable warnings about `tty` access attempts.
2022-08-11 11:35:30 -07:00
Justine Tunney
0277d7d6e9 Rewrite Linux pledge() code so it can be a payload
It's now possible to build our pledge() polyfill as a dynamic shared
object that can be injected into a glibc executable using LD_PRELOAD
2022-08-08 11:41:08 -07:00
Justine Tunney
5546559034 Improve pledge() usability and consistency
- We now kill the program on violations like OpenBSD
- We now print a message explaining which promise is needed
- This change also fixes a linkage bug with thread local storage
- Your sigaction() handlers should now be more thread safe

A new `__pledge_mode` global has been introduced to make pledge() more
customizable on Linux. For example:

    __attribute__((__constructor__)) static void init(void) {
      __pledge_mode = SECCOMP_RET_ERRNO | EPERM;
    }

Can be used to restore our old permissive pledge() behavior.
2022-08-07 16:18:33 -07:00
Justine Tunney
cf93ecbbb2 Prove that Makefile is fully defined
The whole repository is now buildable with GNU Make Landlock sandboxing.
This proves that no Makefile targets exist which touch files other than
their declared prerequisites. In order to do this, we had to:

  1. Stop code morphing GCC output in package.com and instead run a
     newly introduced FIXUPOBJ.COM command after GCC invocations.

  2. Disable all the crumby Python unit tests that do things like create
     files in the current directory, or rename() files between folders.
     This ended up being a lot of tests, but most of them are still ok.

  3. Introduce an .UNSANDBOXED variable to GNU Make to disable Landlock.
     We currently only do this for things like `make tags`.

  4. This change deletes some GNU Make code that was preventing the
     execve() optimization from working. This means it should no longer
     be necessary in most cases for command invocations to be indirected
     through the cocmd interpreter.

  5. Missing dependencies had to be declared in certain places, in cases
     where they couldn't be automatically determined by MKDEPS.COM

  6. The libcxx header situation has finally been tamed. One of the
     things that makes this difficult is MKDEPS.COM only wants to
     consider the first 64kb of a file, in order to go fast. But libcxx
     likes to have #include lines buried after huge documentation.

  7. An .UNVEIL variable has been introduced to GNU Make just in case
     we ever wish to explicitly specify additional things that need to
     be whitelisted which aren't strictly prerequisites. This works in
     a manner similar to the recently introduced .EXTRA_PREREQS feature.

There's now a new build/bootstrap/make.com prebuilt binary available. It
should no longer be possible to write invalid Makefile code.
2022-08-06 04:05:08 -07:00
Justine Tunney
acdf591833 Avoid long double timestamps in redbean 2022-08-05 19:24:05 -07:00
Justine Tunney
8d9ac3da50 Improve redbean memory locality
Doing this improves message latency and throughput, since we're
consolidating the 504 bytes of static memory which has the most
churn into a single place.
2022-08-05 17:34:53 -07:00
Justine Tunney
c9d7838213 Enter meltdown mode if ProgramMaxWorkers triggers 2022-08-05 15:42:17 -07:00
Paul Kulchenko
9c06067c84
Add max worker processing to redbean (#520)
* Add max worker processing
* Introduce ProgramMaxWorkers() API
2022-08-05 15:21:27 -07:00
Justine Tunney
9d7514a906 Release redbean 2.0.16 2022-08-05 15:05:57 -07:00
Justine Tunney
84e89a55d4 Fix redbean -A asset storing flag 2022-08-05 15:04:13 -07:00
Paul Kulchenko
449fb2fb59
Fix stderr being improperly closed during daemonization (#513)
This may happen when ProgramLogPath is used after a file descriptor is
already used by an earlier call (for example, to open an SQLite file).
2022-08-05 14:57:17 -07:00