Commit graph

107 commits

Author SHA1 Message Date
Justine Tunney
367d06d9e4 Fold LIBC_UNICODE into LIBC_STR 2022-08-13 08:42:32 -07:00
Justine Tunney
17aea99bb3 Fold LIBC_ALG into LIBC_MEM 2022-08-13 08:32:34 -07:00
Justine Tunney
7cf66bc161 Prevent Make from talking to public Internet
This change introduces the nointernet() function which may be called to
prevent a process and its descendants from communicating with publicly
routable Internet addresses. GNU Make has been modified to always call
this function. In the future Landlock Make will have a way to whitelist
subnets to override this behavior, or disable it entirely. Support is
available for Linux only. Our firewall does not require root access.

Calling nointernet() will return control to the caller inside a new
process that has a SECCOMP BPF filter installed, which traps network
related system calls. Your original process then becomes a permanent
ptrace() supervisor that monitors all processes and threads descending
from the returned child. Whenever a networking system call happens the
kernel will stop the process and wakes up the monitor, which then peeks
into the child memory to read the sockaddr_in to determine if it's ok.

The downside to doing this is that there can be only one supervisor at a
time using ptrace() on a process. So this firewall won't be enabled if
you run make under strace or inside gdb. It also makes testing tricky.
2022-08-12 21:51:39 -07:00
Justine Tunney
8a0a2c0c36 Fold LIBC_RAND into LIBC_STDIO/TINYMATH/INTRIN 2022-08-11 12:32:00 -07:00
Justine Tunney
05b8f82371 Fold LIBC_BITS into LIBC_INTRIN 2022-08-11 12:13:18 -07:00
Justine Tunney
625aa365f1 Refactor pledge() to be more configurable
The earlier iterations did too much guesswork when it came to things
like stderr logging and syscall origin verification. This change will
make things more conformant to existing practices. The __pledge_mode
extension now can be configured in a better way.

There's also a new `-q` flag added to pledge.com, e.g.

    o//tool/build/pledge.com -qv. ls

Is a good way to disable warnings about `tty` access attempts.
2022-08-11 11:35:30 -07:00
Justine Tunney
10fd8bdb70 Unbloat the build
This change resurrects ae5d06dc53
2022-08-11 00:15:29 -07:00
Justine Tunney
7b993d561c Use private keyword on makefile target variables 2022-08-10 18:36:11 -07:00
Justine Tunney
c1d99676c4 Revert "Unbloat build config"
This reverts commit ae5d06dc53.
2022-08-10 12:44:56 -07:00
Justine Tunney
ae5d06dc53 Unbloat build config
- 10.5% reduction of o//depend dependency graph
- 8.8% reduction in latency of make command
- Fix issue with temporary file cleanup

There's a new -w option in compile.com that turns off the recent
Landlock output path workaround for "good commands" which do not
unlink() the output file like GNU tooling does.

Our new GNU Make unveil sandboxing appears to have zero overhead
in the grand scheme of things. Full builds are pretty fast since
the only thing that's actually slowed us down is probably libcxx

    make -j16 MODE=rel
    RL: took 85,732,063µs wall time
    RL: ballooned to 323,612kb in size
    RL: needed 828,560,521µs cpu (11% kernel)
    RL: caused 39,080,670 page faults (99% memcpy)
    RL: 350,073 context switches (72% consensual)
    RL: performed 0 reads and 11,494,960 write i/o operations

pledge() and unveil() no longer consider ENOSYS to be an error.
These functions have also been added to Python's cosmo module.

This change also removes some WIN32 APIs and System Five magnums
which we're not using and it's doubtful anyone else would be too
2022-08-10 04:43:09 -07:00
Justine Tunney
0277d7d6e9 Rewrite Linux pledge() code so it can be a payload
It's now possible to build our pledge() polyfill as a dynamic shared
object that can be injected into a glibc executable using LD_PRELOAD
2022-08-08 11:41:08 -07:00
Justine Tunney
5546559034 Improve pledge() usability and consistency
- We now kill the program on violations like OpenBSD
- We now print a message explaining which promise is needed
- This change also fixes a linkage bug with thread local storage
- Your sigaction() handlers should now be more thread safe

A new `__pledge_mode` global has been introduced to make pledge() more
customizable on Linux. For example:

    __attribute__((__constructor__)) static void init(void) {
      __pledge_mode = SECCOMP_RET_ERRNO | EPERM;
    }

Can be used to restore our old permissive pledge() behavior.
2022-08-07 16:18:33 -07:00
Justine Tunney
cf93ecbbb2 Prove that Makefile is fully defined
The whole repository is now buildable with GNU Make Landlock sandboxing.
This proves that no Makefile targets exist which touch files other than
their declared prerequisites. In order to do this, we had to:

  1. Stop code morphing GCC output in package.com and instead run a
     newly introduced FIXUPOBJ.COM command after GCC invocations.

  2. Disable all the crumby Python unit tests that do things like create
     files in the current directory, or rename() files between folders.
     This ended up being a lot of tests, but most of them are still ok.

  3. Introduce an .UNSANDBOXED variable to GNU Make to disable Landlock.
     We currently only do this for things like `make tags`.

  4. This change deletes some GNU Make code that was preventing the
     execve() optimization from working. This means it should no longer
     be necessary in most cases for command invocations to be indirected
     through the cocmd interpreter.

  5. Missing dependencies had to be declared in certain places, in cases
     where they couldn't be automatically determined by MKDEPS.COM

  6. The libcxx header situation has finally been tamed. One of the
     things that makes this difficult is MKDEPS.COM only wants to
     consider the first 64kb of a file, in order to go fast. But libcxx
     likes to have #include lines buried after huge documentation.

  7. An .UNVEIL variable has been introduced to GNU Make just in case
     we ever wish to explicitly specify additional things that need to
     be whitelisted which aren't strictly prerequisites. This works in
     a manner similar to the recently introduced .EXTRA_PREREQS feature.

There's now a new build/bootstrap/make.com prebuilt binary available. It
should no longer be possible to write invalid Makefile code.
2022-08-06 04:05:08 -07:00
Justine Tunney
638c56e3a5 Propagate nil in Lua APIs more often 2022-08-05 17:34:13 -07:00
Justine Tunney
8593580d0a Fix lsqlite3.lversion() and document local changes
Fixes #507
2022-07-24 05:13:28 -07:00
Justine Tunney
f968e2a726 Improve pledge() and unveil() further
- Fix getpriority()
- Add AT_MINSIGSTKSZ
- Fix bugs in BPF code
- Show more stuff in printargs.com
- Write manual test for pledge.com
- pledge() now generates tinier BPF code
- Have pledge("exec") only enable execve()
- Fix pledge.com chroot setuid functionality
- Improve pledge.com unveiling of ape loader
2022-07-24 03:10:16 -07:00
Jared Miller
31ac58a57b
Adds RFC driven BASE64, MD5, and SHA tests (#504) 2022-07-23 19:01:32 -07:00
Justine Tunney
ce5cb8a2f8 Have redbean Lua repl show utf-8 when it's valid
Otherwise EncodeLua() will assume strings are binary and display them
using hex sequences.
2022-07-23 07:22:19 -07:00
Justine Tunney
31e4b0867b Pledge and unveil redbean's unit tests 2022-07-23 07:22:19 -07:00
Justine Tunney
48ce3ad7cc Do some work on redbean
- Rewrite Slurp() API to be like string.sub()
- Introduce a new Barf() API for creating files
- Update Redbean `-S` sandbox flag to do unveiling
2022-07-22 20:44:24 -07:00
Justine Tunney
516b68606f Add pretty printing to redbean serializers 2022-07-22 10:10:33 -07:00
Jared Miller
d4000bb8f7
Correct more typos (#500) 2022-07-21 20:53:30 -07:00
Justine Tunney
1837dc2e85 Make improvements
- Introduce path module to redbean
- Fix glitch with linenoise printing extra line on eof
- Introduce closefrom() and close_range() system calls
- Make file descriptor closing more secure in pledge.com
2022-07-21 03:36:42 -07:00
Jared Miller
7e2eae5c15
Remove trailing whitespace from all files (#497) 2022-07-20 20:31:16 -07:00
Justine Tunney
7f966de489 Limit pledge.com default virtual mem to total ram 2022-07-15 06:53:01 -07:00
Justine Tunney
baf51a4a23 Add utf-8 validation to ljson 2022-07-15 06:20:07 -07:00
Justine Tunney
28b9d9f781 Fix C stack remaining check in MODE=tiny 2022-07-14 07:23:15 -07:00
Justine Tunney
bcdd5c5f74 Fix ljson stack check 2022-07-14 07:07:17 -07:00
Justine Tunney
b707fca77a Make JSON parser perfectly conformant 2022-07-13 23:02:19 -07:00
Gautham
0cea6c560f
Make JSON parser nearly perfectly compliant (#483) 2022-07-13 07:38:23 -07:00
Gautham
12d9f7ade6
Make ljson more strict (#482) 2022-07-13 02:39:19 -07:00
Justine Tunney
e3cd476a9b Improve Lua and JSON serialization 2022-07-12 23:35:11 -07:00
Justine Tunney
3027d67037 Import more Musl math 2022-07-12 15:55:12 -07:00
Justine Tunney
6d52664aa7 Rename json test files to avoid build tuning 2022-07-12 13:24:25 -07:00
Justine Tunney
3f3e7e92d7 Audit every single JSON test 2022-07-12 12:40:38 -07:00
Justine Tunney
6ee18986e4 Further improve JSON serialization 2022-07-11 23:06:49 -07:00
Gautham
3c10fb5580
Fix quote escapes in JSON tests (#481) 2022-07-11 18:04:51 -07:00
Gautham
9308463b0f
Escape bad unicode in json tests (#480) 2022-07-11 15:16:44 -07:00
Gautham
594615a0ff
Add more json tests for ljson (#478) 2022-07-11 11:54:24 -07:00
ProducerMatt
84764ce7b8
lre_test.lua: Benchmark() fails on some systems (#474) 2022-07-10 23:05:50 -07:00
Gautham
c0b325bafa
Add json.org tests for DecodeJson (#473) 2022-07-10 08:35:51 -07:00
Justine Tunney
5fa77f1e8f Make _Thread_local more seamlessly working 2022-07-10 08:27:50 -07:00
Justine Tunney
e4d6e263d4 Rename ParseJson() to DecodeJson() for consistency 2022-07-09 17:34:41 -07:00
Justine Tunney
ee82cee432 Further improve json / lua serialization 2022-07-09 16:27:26 -07:00
Justine Tunney
7756e27993 Fix GitHub build 2022-07-09 12:00:57 -07:00
Justine Tunney
2189877856 Implement new JSON parser for redbean 2022-07-09 11:44:19 -07:00
Justine Tunney
d37536bd4b Fix oops w/ array serialization ordering 2022-07-09 10:29:26 -07:00
Justine Tunney
c9e68b0ebc Make redbean serialization deterministic 2022-07-09 04:09:51 -07:00
Justine Tunney
182d7720cf Fix GitHub actions
Our build was flaking due to ETXTBSY errors running multiple redbean
instances in parallel. This is due to the StoreAsset() support which
seems to cause enough problems it's worth making a *breaking change*
turning it off by default for now. There's a new -* flag, to restore
redbean's old self-modifying behavior.
2022-07-09 01:18:55 -07:00
Justine Tunney
1c83670229 Write more redbean unit tests
- Fix DescribeSigset()
- Introduce new unix.rmrf() API
- Fix redbean sigaction() doc example code
- Fix unix.sigaction() w/ more than two args
- Improve redbean re module API (non-breaking)
- Enhance Lua with Python string multiplication
- Make third parameter of unix.socket() default to 0
2022-07-08 23:10:02 -07:00