Commit graph

409 commits

Author SHA1 Message Date
Paul Kulchenko
449fb2fb59
Fix stderr being improperly closed during daemonization (#513)
This may happen when ProgramLogPath is used after a file descriptor is
already used by an earlier call (for example, to open an SQLite file).
2022-08-05 14:57:17 -07:00
Theta Nil
ab301401c7
Add gensvg demo to redbean (#524)
gensvg demo and mk for it
2022-08-03 05:21:41 -07:00
Paul Kulchenko
42bd79a461
Add OnServerHeartbeat to Redbean (#522) 2022-08-02 20:41:44 -07:00
Justine Tunney
8593580d0a Fix lsqlite3.lversion() and document local changes
Fixes #507
2022-07-24 05:13:28 -07:00
Justine Tunney
f968e2a726 Improve pledge() and unveil() further
- Fix getpriority()
- Add AT_MINSIGSTKSZ
- Fix bugs in BPF code
- Show more stuff in printargs.com
- Write manual test for pledge.com
- pledge() now generates tinier BPF code
- Have pledge("exec") only enable execve()
- Fix pledge.com chroot setuid functionality
- Improve pledge.com unveiling of ape loader
2022-07-24 03:10:16 -07:00
Paul Kulchenko
638e14bbf3
Improve Redbean shutdown (#506)
* Update redbean shutdown to call OnServerStop when all shutdown/logging is done

* Move closing file descriptors during daemonization earlier

This should fix using opened file descriptors, for example, SQLite DB
files and redbean itself when StoreAsset is used. Fixes #182.

* Move opening logs earlier to capture logs from Listen and .init.lua

* Move pidpath handling outside of daemonize, as it can be used independently
2022-07-23 18:58:31 -07:00
Justine Tunney
16fc83f9ce Explicitly disable Linux capabilities 2022-07-23 12:06:41 -07:00
Justine Tunney
ffedbfe14d Fix copy/paste error in redbean docs 2022-07-23 07:37:39 -07:00
Justine Tunney
3828c08aa0 Release redbean 2.0.15 2022-07-23 07:22:19 -07:00
Justine Tunney
31e4b0867b Pledge and unveil redbean's unit tests 2022-07-23 07:22:19 -07:00
Justine Tunney
48ce3ad7cc Do some work on redbean
- Rewrite Slurp() API to be like string.sub()
- Introduce a new Barf() API for creating files
- Update Redbean `-S` sandbox flag to do unveiling
2022-07-22 20:44:24 -07:00
Justine Tunney
742251dd92 Release redbean 2.0.14 2022-07-22 17:07:25 -07:00
Paul Kulchenko
b5904947e9
Add GetResponseBody to redbean (#502)
* Add GetResponseBody to redbean to get access to generated response
* Update GetStatus to return status code set by redbean itself (outside
  of Lua code)
2022-07-22 11:22:39 -07:00
Justine Tunney
516b68606f Add pretty printing to redbean serializers 2022-07-22 10:10:33 -07:00
Justine Tunney
84caee23ba Make sorted serialization faster
Redbean Lua and JSON serialization now goes faster because we're now
inserting object entries into tree data structure rather than making
an array and sorting it at the end. For example, when serializing an
object with 10,000 entries this goes twice as fast. However it still
goes slower than saying EncodeJson(x, {sorted=false}).
2022-07-22 04:19:01 -07:00
Justine Tunney
8b469389f6 Remove plenty of makefile misconfigurations 2022-07-21 09:20:59 -07:00
Justine Tunney
1837dc2e85 Make improvements
- Introduce path module to redbean
- Fix glitch with linenoise printing extra line on eof
- Introduce closefrom() and close_range() system calls
- Make file descriptor closing more secure in pledge.com
2022-07-21 03:36:42 -07:00
Jared Miller
7e2eae5c15
Remove trailing whitespace from all files (#497) 2022-07-20 20:31:16 -07:00
jared
ed205e98a1
WIP: Correct all typos (#498) 2022-07-20 14:01:15 -07:00
Justine Tunney
98254a7c1f Make pledge() and unveil() work amazingly
This change reconciles our pledge() implementation with the OpenBSD
kernel source code. We now a polyfill that's much closer to OpenBSD's
behavior. For example, it was discovered that "stdio" permits threads.
There were a bunch of Linux system calls that needed to be added, like
sched_yield(). The exec / execnative category division is now dropped.
We're instead using OpenBSD's "prot_exec" promise for launching APE
binaries and dynamic shared objects. We also now filter clone() flags.

The pledge.com command has been greatly improved. It now does unveiling
by default when Landlock is available. It's now smart enough to unveil a
superset of paths that OpenBSD automatically unveils with pledge(), such
as /etc/localtime. pledge.com also now checks if the executable being
launched is a dynamic shared object, in which case it unveils libraries.

These changes now make it possible to pledge curl on ubuntu 20.04 glibc:

    pledge.com -p 'stdio rpath prot_exec inet dns tty sendfd recvfd' \
        curl -s https://justine.lol/hello.txt

Here's what pledging curl on Alpine 3.16 with Musl Libc looks like:

    pledge.com -p 'stdio rpath prot_exec dns inet' \
        curl -s https://justine.lol/hello.txt

Here's what pledging curl.com w/ ape loader looks like:

    pledge.com -p 'stdio rpath prot_exec dns inet' \
        o//examples/curl.com https://justine.lol/hello.txt

The most secure sandbox, is curl.com converted to static ELF:

    o//tool/build/assimilate.com o//examples/curl.com
    pledge.com -p 'stdio rpath dns inet' \
        o//examples/curl.com https://justine.lol/hello.txt

A weird corner case needed to be handled when resolving symbolic links
during the unveiling process, that's arguably a Landlock bug. It's not
surprising since Musl and Glibc are also inconsistent here too.
2022-07-19 21:33:49 -07:00
Justine Tunney
bf59defc0c Fix GitHub Actions build
This is an unusual failure that seems to happen intermittently across
the various build modes. It should not be possible for life.elf to be
exiting with status zero.
2022-07-18 20:45:18 -07:00
Paul Kulchenko
574eba8352
Add redbean OnLogLatency hook (#495) 2022-07-18 20:17:14 -07:00
Justine Tunney
e81edf7b04 Improve pledge() and unveil()
The pledge.com command now supports the new [WIP] unveil() support. For
example, to strongly sandbox our command for listing directories.

    o//tool/build/assimilate.com o//examples/ls.com
    pledge.com -v /etc -p 'stdio rpath' o//examples/ls.com /etc

This file system sandboxing is going to be perfect for us, because APE
binaries are self-contained static executables that really don't use the
filesystem that much. On the other hand, with non-static executables,
sandboxing is going to be more difficult. For example, here's how to
sandbox the `ls` command on the latest Alpine:

    pledge.com -v rx:/lib -v /usr/lib -v /etc -p 'stdio rpath exec' ls /etc

This change fixes the `execpromises` API with pledge().

This change also adds unix.unveil() to redbean.

Fixes #494
2022-07-18 07:58:20 -07:00
Justine Tunney
5b11033d4d Add redbean -I flag for launching browser
It's now possible with any redbean (including redbean-original) to
launch the system web browser without having to use the Lua API. For
example, you can create an args file:

    echo -I/ >.args
    zip redbean-original.com .args

That will white-label redbean so it launches a specific page when you
double-click on the executable.

See https://github.com/jart/cosmopolitan/discussions/472
2022-07-17 06:12:57 -07:00
Justine Tunney
4d25f8c3c9 Add tcp syn packet fingerprinting to redbean
This change also fixes bugs in enoprotoopt reporting with setsockopt and
getsockopt error returns.
2022-07-17 02:43:49 -07:00
Wiebe
4700984456
Update Redbean help.txt (#492)
Fix typos and add previously undocumented functions:

- GetHttpReason
- IsHiddenPath
- IsAcceptablePath
- IsReasonablePath
- ProgramTimeout
2022-07-16 11:00:28 -07:00
Justine Tunney
6c49e36537 Release redbean 2.0.13 2022-07-15 07:47:36 -07:00
Justine Tunney
baf51a4a23 Add utf-8 validation to ljson 2022-07-15 06:20:07 -07:00
Justine Tunney
28b9d9f781 Fix C stack remaining check in MODE=tiny 2022-07-14 07:23:15 -07:00
Justine Tunney
b707fca77a Make JSON parser perfectly conformant 2022-07-13 23:02:19 -07:00
Justine Tunney
60164a7266 Add assimilate.com command for APE binaries 2022-07-13 20:56:24 -07:00
Gautham
0cea6c560f
Make JSON parser nearly perfectly compliant (#483) 2022-07-13 07:38:23 -07:00
Justine Tunney
1d490fcb94 Add pledge.com for launching commands in a sandbox 2022-07-13 04:31:46 -07:00
Gautham
12d9f7ade6
Make ljson more strict (#482) 2022-07-13 02:39:19 -07:00
Justine Tunney
30cc2c8dc1 Release redbean 2.0.12 2022-07-13 00:05:31 -07:00
Justine Tunney
e3cd476a9b Improve Lua and JSON serialization 2022-07-12 23:35:11 -07:00
Justine Tunney
3027d67037 Import more Musl math 2022-07-12 15:55:12 -07:00
Justine Tunney
3f3e7e92d7 Audit every single JSON test 2022-07-12 12:40:38 -07:00
Justine Tunney
6ee18986e4 Further improve JSON serialization 2022-07-11 23:06:49 -07:00
Gautham
11a1c62d11
Reduce JSON depth limit to 128 (#477)
- also remove tpyo from net.mk
- STACK_FRAME_UNLIMITED is not needed
- also let parser allow 0e1, 0e+1 as floats
2022-07-11 10:23:44 -07:00
Gautham
aeee2f9106
Add depth limit for JSON (#476) 2022-07-11 08:17:58 -07:00
Justine Tunney
3f015b1e51 Make some minor fixups to bug reporting, etc. 2022-07-11 05:58:24 -07:00
Justine Tunney
68ca49bfdd Improve APE install scripts and add uninstaller
See #350 thanks @tkchia
2022-07-10 14:13:45 -07:00
Justine Tunney
331fdd1d29 Fix some broken builds and broken hearts 2022-07-10 09:08:38 -07:00
Gautham
c0b325bafa
Add json.org tests for DecodeJson (#473) 2022-07-10 08:35:51 -07:00
Justine Tunney
5f4f6b0e69 Make _Thread_local work across platforms
We now rewrite the binary image at runtime on Windows and XNU to change
mov %fs:0,%reg instructions to use %gs instead. There's also simpler
threading API introduced by this change and it's called _spawn() and
_join(), which has replaced most clone() usage.
2022-07-10 04:01:17 -07:00
Justine Tunney
e4d6e263d4 Rename ParseJson() to DecodeJson() for consistency 2022-07-09 17:34:41 -07:00
Justine Tunney
28f0104330 Bump redbean up to 2.0.10 2022-07-09 17:28:14 -07:00
Justine Tunney
ee82cee432 Further improve json / lua serialization 2022-07-09 16:27:26 -07:00
Paul Kulchenko
9e86980191
Redbean fix listen leak (#470)
* Fix release of resources allocated to skipped sockets
* Update log messages for consistency
2022-07-09 12:16:44 -07:00
Justine Tunney
2189877856 Implement new JSON parser for redbean 2022-07-09 11:44:19 -07:00
Justine Tunney
727d9cbf56 Bump redbean to 2.0.9 2022-07-09 05:50:05 -07:00
Justine Tunney
c9e68b0ebc Make redbean serialization deterministic 2022-07-09 04:09:51 -07:00
Justine Tunney
182d7720cf Fix GitHub actions
Our build was flaking due to ETXTBSY errors running multiple redbean
instances in parallel. This is due to the StoreAsset() support which
seems to cause enough problems it's worth making a *breaking change*
turning it off by default for now. There's a new -* flag, to restore
redbean's old self-modifying behavior.
2022-07-09 01:18:55 -07:00
Justine Tunney
1c83670229 Write more redbean unit tests
- Fix DescribeSigset()
- Introduce new unix.rmrf() API
- Fix redbean sigaction() doc example code
- Fix unix.sigaction() w/ more than two args
- Improve redbean re module API (non-breaking)
- Enhance Lua with Python string multiplication
- Make third parameter of unix.socket() default to 0
2022-07-08 23:10:02 -07:00
Justine Tunney
fe5c475f83 Add Deflate() / Inflate() to redbean and fix bugs
The Compress() and Uncompress() APIs were a mistake. The functions
themselves work fine, but it's a design blemish and does superfluous
work. Since they were only introduced in the last few weeks, they're now
deprecated and references to them have been scrubbed from the website
and other documentation. Please use the new APIs since the old APIs will
be removed at some point in the future.

This change introduces automated Lua unit tests for the Redbean APIs.
There's a few functions that were broken which have now been fixed, e.g.
Underlong() and Decimate().
2022-07-08 09:47:04 -07:00
Paul Kulchenko
a18044c504
Add OnServerListen hook to configure listen() (#459) 2022-07-08 07:17:25 -07:00
Paul Kulchenko
11ac8f11a9
Reduce redbean logging for EBADF (#461)
This may happen on explicitly closed client connections, so reduce the
logging level similar to other errors.
2022-07-08 07:14:50 -07:00
Paul Kulchenko
059fe22ea3
Improve redbean fetch() (#460)
* Updated Fetch to return `nil,error` on errors
* Fix localhost connect when only IP address is specified
2022-07-08 07:13:51 -07:00
Justine Tunney
853b6c3864 Improve system calls
- Wrap clock_getres()
- Wrap sched_setscheduler()
- Make sleep() api conformant
- Polyfill sleep() using select()
- Improve clock_gettime() polyfill
- Make nanosleep() POSIX conformant
- Slightly improve some DNS functions
- Further strengthen pledge() sandboxing
- Improve rounding of timeval / timespec
- Allow layering of pledge() calls on Linux
- Polyfill sched_yield() using select() on XNU
- Delete more system constants we probably don't need
2022-07-08 06:42:03 -07:00
Paul Kulchenko
5df3e4e7a8
Redbean zip and unix.poll doc update (#457) 2022-06-30 17:41:51 -07:00
Justine Tunney
3c92adfd6e Strengthen the pledge() polyfill 2022-06-27 13:02:17 -07:00
Paul Kulchenko
32eec7df4c
Update FATALF to exit without stack trace (#452) 2022-06-26 21:25:02 -07:00
Paul Kulchenko
283729dfe0
Redbean report write lock failure (#450)
This change updates redbean to report a non-writable path
when using StoreAsset, instead of crashing the worker.
2022-06-26 21:16:13 -07:00
Paul Kulchenko
b4e9a77e74
Improve redbean ZeroBrane integration (#451) 2022-06-26 21:13:53 -07:00
Justine Tunney
f6dab99f44 Release redbean 2.0.8 2022-06-26 17:25:54 -07:00
Justine Tunney
fbc053e018 Make fixes and improvements
- Introduce __assert_disable global
- Improve strsignal() thread safety
- Make system call tracing thread safe
- Fix SO_RCVTIMEO / SO_SNDTIMEO on Windows
- Refactor DescribeFoo() functions into one place
- Fix fork() on Windows when TLS and MAP_STACK exist
- Round upwards in setsockopt(SO_RCVTIMEO) on Windows
- Disable futexes on OpenBSD which seem extremely broken
- Implement a better kludge for monotonic time on Windows
2022-06-25 21:09:09 -07:00
Paul Kulchenko
1bce1ca2e0
Polish redbean APIs and docs (#446)
* Remove undocumented ProgramSslCompression
* Rename GetLastModified to GetAssetLastModified
* Rename IsCompressed to IsAssetCompressed
* Reorganize currently deprecated functions
* Update sockopt documentation for clarity
2022-06-23 18:54:35 -07:00
Justine Tunney
a9d77362f9 Release redbean 2.0.7 2022-06-23 18:14:17 -07:00
Justine Tunney
4b9ee980a1 Release redbean 2.0.6 2022-06-23 17:37:55 -07:00
Justine Tunney
3d07f362d7 Release redbean 2.0.5 2022-06-23 16:09:20 -07:00
Justine Tunney
44da16255a Show crash reports on SIGSYS
This will help make it easier to troubleshoot ABI breakages with on
operating systems that, unlike Linux don't have ironclad guarantees
to not break userspace.
2022-06-23 13:01:01 -07:00
Justine Tunney
206f073181 Add stat mode macros to redbean unix api 2022-06-23 04:05:51 -07:00
Justine Tunney
17cbe73411 Add finger demo to redbean and fix regression
This change fixes a regression in unix.connect() caused by the recent
addition of UNIX domain sockets. The BSD finger command has been added
to third_party for fun and profit. A new demo has been added to redbean
showing how a protocol as simple as finger can be implemented.
2022-06-23 03:42:05 -07:00
Justine Tunney
fc097ac275 Add unix domain socket support to redbean 2022-06-22 03:04:25 -07:00
Justine Tunney
a988896048 Add atomics to chibicc
This change also fixes #434 and makes the chibicc assembler better.
2022-06-20 03:08:00 -07:00
Justine Tunney
5ddf43332e Release redbean 2.0.4 2022-06-19 20:17:58 -07:00
Justine Tunney
2c7fe0f361 Fix redbean regression from previous commit 2022-06-19 00:59:07 -07:00
Justine Tunney
8c895453e2 Fix build break due to deleted file 2022-06-18 01:55:32 -07:00
Justine Tunney
c1cfca8ae1 Make fixes and improvements
- Polyfill UTIME_OMIT on XNU
- Refactor Lua build code so it's better
- Add unix module to lua.com (Discord request)
- Add unix.utimensat() and unix.futimens() to redbean
- Avoid creating double slash path in linenoise (#428)
- Remove double slashes in NT paths automatically (#428)
- Make strerror() smarter about showing NT errors (#428)

Fixes #428
2022-06-18 01:46:12 -07:00
Justine Tunney
67b28b9af1 Update redbean documentation 2022-06-17 08:05:47 -07:00
Justine Tunney
52ed099fe6 Favor $HOME directory over /tmp
This should strike a better compromise that keeps people happy about the
security of APE when it extracts the loader. This way systems with users
who aren't trusted (e.g. CPanel) won't be at any risk of compromise when
there isn't an `ape` loader on the system `$PATH`.

This change also bumps redbean up to 2.0.3
2022-06-17 07:56:54 -07:00
Justine Tunney
5e04827949 Add fileio and sqlar SQLite extensions to redbean 2022-06-17 06:04:09 -07:00
Justine Tunney
bc90db0d4b redbean 2.0.2 2022-06-17 04:18:20 -07:00
Justine Tunney
34e39ad027 Enable sqlite zipfile module in redbean
This change also breaks out a bunch of extension files that the SQLite
authors inlined into a shell.c amalgamation.
2022-06-17 02:49:04 -07:00
Justine Tunney
2c7f865b12 Improve http caching in redbean 2022-06-17 02:44:15 -07:00
Justine Tunney
c06ffd458c Write some lock contention tests 2022-06-16 09:06:09 -07:00
Justine Tunney
e466dd0553 Add torture test for zipos file descriptors
This change hardens the code for opening /zip/ files using the system
call interface. Thread safety and signal safety has been improved for
file descriptors in general. We now document fixed addresses that are
needed for low level allocations.
2022-06-15 16:29:49 -07:00
Justine Tunney
579080cd4c Improve generation of redbean static headers
This change fixes an issue where Cache-Control could be outputted twice
when Lua code calls SetHeader() from OnHttpRequest().
2022-06-14 22:01:13 -07:00
Justine Tunney
42b34c26f8 Release redbean 2.0.1 2022-06-14 20:21:34 -07:00
Paul Kulchenko
8b13e1acc8
Fixed redbean compression functions (#421) 2022-06-14 17:50:25 -07:00
Paul Kulchenko
58c77a9b1d
Fix recent regression in redbean helpers (#419) 2022-06-14 17:22:51 -07:00
Justine Tunney
4ddfc47d6e Make some more fixups 2022-06-12 09:37:17 -07:00
Justine Tunney
c260345e06 Make locks more reliable
This change switches most of the core locks to be re-entrant, in order
to reduce the chance of deadlocking code that does, clever things with
asynchronous signal handlers. This change implements it it in pthreads
so we're one step closer to having a standardized threading primitives
2022-06-11 02:07:20 -07:00
Justine Tunney
5ea618f0af Make clock_gettime() faster on FreeBSD 2022-06-11 00:05:06 -07:00
Justine Tunney
5deda43766 Handle wildcard certificates in redbean 2022-06-10 21:59:14 -07:00
Justine Tunney
c6d8e516b2 Print warning when microbenchmarking w/ powersave
RDTSC on Linux has so much jitter when the CPU is in powersave mode
causing things like microbenchmarks to have a 1000% margin of error
2022-06-10 21:07:03 -07:00
Justine Tunney
f8520e10b2 Add ProgramMaxPayloadSize() API to Redbean
This way we don't have to use the flag and can configure it using
.init.lua instead.
2022-06-10 03:28:46 -07:00
Justine Tunney
adac64a52b Reduce makefile dependencies by 10%
The includes in libc/calls/calls.h have now been refactored so that
functions with struct parameters are declared in libc/calls/struct/
2022-06-08 20:01:28 -07:00
Justine Tunney
da6d610056 Use Lua auto buffers when possible 2022-05-29 19:04:30 -07:00