Commit graph

565 commits

Author SHA1 Message Date
Justine Tunney
c921dc78f0 Release pledge.com v1.3 2022-07-24 08:35:13 -07:00
Justine Tunney
8593580d0a Fix lsqlite3.lversion() and document local changes
Fixes #507
2022-07-24 05:13:28 -07:00
Justine Tunney
f968e2a726 Improve pledge() and unveil() further
- Fix getpriority()
- Add AT_MINSIGSTKSZ
- Fix bugs in BPF code
- Show more stuff in printargs.com
- Write manual test for pledge.com
- pledge() now generates tinier BPF code
- Have pledge("exec") only enable execve()
- Fix pledge.com chroot setuid functionality
- Improve pledge.com unveiling of ape loader
2022-07-24 03:10:16 -07:00
Paul Kulchenko
638e14bbf3
Improve Redbean shutdown (#506)
* Update redbean shutdown to call OnServerStop when all shutdown/logging is done

* Move closing file descriptors during daemonization earlier

This should fix using opened file descriptors, for example, SQLite DB
files and redbean itself when StoreAsset is used. Fixes #182.

* Move opening logs earlier to capture logs from Listen and .init.lua

* Move pidpath handling outside of daemonize, as it can be used independently
2022-07-23 18:58:31 -07:00
Justine Tunney
16fc83f9ce Explicitly disable Linux capabilities 2022-07-23 12:06:41 -07:00
Justine Tunney
ffedbfe14d Fix copy/paste error in redbean docs 2022-07-23 07:37:39 -07:00
Justine Tunney
3828c08aa0 Release redbean 2.0.15 2022-07-23 07:22:19 -07:00
Justine Tunney
31e4b0867b Pledge and unveil redbean's unit tests 2022-07-23 07:22:19 -07:00
Justine Tunney
48ce3ad7cc Do some work on redbean
- Rewrite Slurp() API to be like string.sub()
- Introduce a new Barf() API for creating files
- Update Redbean `-S` sandbox flag to do unveiling
2022-07-22 20:44:24 -07:00
Justine Tunney
742251dd92 Release redbean 2.0.14 2022-07-22 17:07:25 -07:00
Justine Tunney
76d2f68c91 Release pledge.com v1.1
This change fixes bugs, adds more system calls, and improves
compatibility with OpenBSD. Going forward, versions on the web will be
pinned to a permanent version. There were many other changes over the
last week which also improved this new release.
2022-07-22 13:44:00 -07:00
Paul Kulchenko
b5904947e9
Add GetResponseBody to redbean (#502)
* Add GetResponseBody to redbean to get access to generated response
* Update GetStatus to return status code set by redbean itself (outside
  of Lua code)
2022-07-22 11:22:39 -07:00
Justine Tunney
516b68606f Add pretty printing to redbean serializers 2022-07-22 10:10:33 -07:00
Justine Tunney
84caee23ba Make sorted serialization faster
Redbean Lua and JSON serialization now goes faster because we're now
inserting object entries into tree data structure rather than making
an array and sorting it at the end. For example, when serializing an
object with 10,000 entries this goes twice as fast. However it still
goes slower than saying EncodeJson(x, {sorted=false}).
2022-07-22 04:19:01 -07:00
Jared Miller
d4000bb8f7
Correct more typos (#500) 2022-07-21 20:53:30 -07:00
Justine Tunney
8b469389f6 Remove plenty of makefile misconfigurations 2022-07-21 09:20:59 -07:00
Justine Tunney
1837dc2e85 Make improvements
- Introduce path module to redbean
- Fix glitch with linenoise printing extra line on eof
- Introduce closefrom() and close_range() system calls
- Make file descriptor closing more secure in pledge.com
2022-07-21 03:36:42 -07:00
Jared Miller
7e2eae5c15
Remove trailing whitespace from all files (#497) 2022-07-20 20:31:16 -07:00
Justine Tunney
d3f3cb7ab4 Fix build 2022-07-20 14:00:38 -07:00
Justine Tunney
097634d75c Make pledge.com binary 4kb smaller 2022-07-20 13:59:28 -07:00
jared
ed205e98a1
WIP: Correct all typos (#498) 2022-07-20 14:01:15 -07:00
Justine Tunney
98254a7c1f Make pledge() and unveil() work amazingly
This change reconciles our pledge() implementation with the OpenBSD
kernel source code. We now a polyfill that's much closer to OpenBSD's
behavior. For example, it was discovered that "stdio" permits threads.
There were a bunch of Linux system calls that needed to be added, like
sched_yield(). The exec / execnative category division is now dropped.
We're instead using OpenBSD's "prot_exec" promise for launching APE
binaries and dynamic shared objects. We also now filter clone() flags.

The pledge.com command has been greatly improved. It now does unveiling
by default when Landlock is available. It's now smart enough to unveil a
superset of paths that OpenBSD automatically unveils with pledge(), such
as /etc/localtime. pledge.com also now checks if the executable being
launched is a dynamic shared object, in which case it unveils libraries.

These changes now make it possible to pledge curl on ubuntu 20.04 glibc:

    pledge.com -p 'stdio rpath prot_exec inet dns tty sendfd recvfd' \
        curl -s https://justine.lol/hello.txt

Here's what pledging curl on Alpine 3.16 with Musl Libc looks like:

    pledge.com -p 'stdio rpath prot_exec dns inet' \
        curl -s https://justine.lol/hello.txt

Here's what pledging curl.com w/ ape loader looks like:

    pledge.com -p 'stdio rpath prot_exec dns inet' \
        o//examples/curl.com https://justine.lol/hello.txt

The most secure sandbox, is curl.com converted to static ELF:

    o//tool/build/assimilate.com o//examples/curl.com
    pledge.com -p 'stdio rpath dns inet' \
        o//examples/curl.com https://justine.lol/hello.txt

A weird corner case needed to be handled when resolving symbolic links
during the unveiling process, that's arguably a Landlock bug. It's not
surprising since Musl and Glibc are also inconsistent here too.
2022-07-19 21:33:49 -07:00
Justine Tunney
69f4152f38 Always initialize thread local storage
We had previously not enabled TLS in MODE=tiny in order to keep the
smallest example programs (e.g. life.com) just 16kb in size. But it
was error prone doing that, so now we just always enable it because
this change uses hacks to ensure it won't increase life.com's size.

This change also fixes a bug on NetBSD, where signal handlers would
break thread local storage if SA_SIGINFO was being used. This looks
like it might be a bug in NetBSD, but it's got a simple workaround.
2022-07-19 00:21:46 -07:00
Justine Tunney
bf59defc0c Fix GitHub Actions build
This is an unusual failure that seems to happen intermittently across
the various build modes. It should not be possible for life.elf to be
exiting with status zero.
2022-07-18 20:45:18 -07:00
Paul Kulchenko
574eba8352
Add redbean OnLogLatency hook (#495) 2022-07-18 20:17:14 -07:00
Stephen Gregoratto
6598940d8a
Make unveil() improvements (#493)
- Merge the multiple masks to just one.
- Add documentation to for sys_unveil.
- Inline the chomp function in the unveil tool.
2022-07-18 08:26:40 -07:00
Justine Tunney
e81edf7b04 Improve pledge() and unveil()
The pledge.com command now supports the new [WIP] unveil() support. For
example, to strongly sandbox our command for listing directories.

    o//tool/build/assimilate.com o//examples/ls.com
    pledge.com -v /etc -p 'stdio rpath' o//examples/ls.com /etc

This file system sandboxing is going to be perfect for us, because APE
binaries are self-contained static executables that really don't use the
filesystem that much. On the other hand, with non-static executables,
sandboxing is going to be more difficult. For example, here's how to
sandbox the `ls` command on the latest Alpine:

    pledge.com -v rx:/lib -v /usr/lib -v /etc -p 'stdio rpath exec' ls /etc

This change fixes the `execpromises` API with pledge().

This change also adds unix.unveil() to redbean.

Fixes #494
2022-07-18 07:58:20 -07:00
Stephen Gregoratto
1c6b5c0acd
[WIP] Polyfill OpenBSD unveil for Linux (#490) 2022-07-18 02:12:42 -07:00
Justine Tunney
5b11033d4d Add redbean -I flag for launching browser
It's now possible with any redbean (including redbean-original) to
launch the system web browser without having to use the Lua API. For
example, you can create an args file:

    echo -I/ >.args
    zip redbean-original.com .args

That will white-label redbean so it launches a specific page when you
double-click on the executable.

See https://github.com/jart/cosmopolitan/discussions/472
2022-07-17 06:12:57 -07:00
Justine Tunney
4d25f8c3c9 Add tcp syn packet fingerprinting to redbean
This change also fixes bugs in enoprotoopt reporting with setsockopt and
getsockopt error returns.
2022-07-17 02:43:49 -07:00
Justine Tunney
866b21a151 Get redbean -X running in blinkenlights again
This change improves the loading of APE executables in Blinkenlights and
adds some system call wrappers that were previous missing.
2022-07-16 11:52:45 -07:00
Wiebe
4700984456
Update Redbean help.txt (#492)
Fix typos and add previously undocumented functions:

- GetHttpReason
- IsHiddenPath
- IsAcceptablePath
- IsReasonablePath
- ProgramTimeout
2022-07-16 11:00:28 -07:00
Justine Tunney
6c724c0f1a Update experiment with tty audio 2022-07-15 23:07:32 -07:00
Justine Tunney
aa34340f3d Add pipelining to cocmd 2022-07-15 20:47:20 -07:00
Justine Tunney
6c49e36537 Release redbean 2.0.13 2022-07-15 07:47:36 -07:00
Justine Tunney
7f966de489 Limit pledge.com default virtual mem to total ram 2022-07-15 06:53:01 -07:00
Justine Tunney
baf51a4a23 Add utf-8 validation to ljson 2022-07-15 06:20:07 -07:00
Justine Tunney
28b9d9f781 Fix C stack remaining check in MODE=tiny 2022-07-14 07:23:15 -07:00
Justine Tunney
1d744ea11b Improve upon the new pledge command 2022-07-14 04:39:45 -07:00
Justine Tunney
b707fca77a Make JSON parser perfectly conformant 2022-07-13 23:02:19 -07:00
Justine Tunney
60164a7266 Add assimilate.com command for APE binaries 2022-07-13 20:56:24 -07:00
Gautham
0cea6c560f
Make JSON parser nearly perfectly compliant (#483) 2022-07-13 07:38:23 -07:00
Justine Tunney
1d490fcb94 Add pledge.com for launching commands in a sandbox 2022-07-13 04:31:46 -07:00
Gautham
12d9f7ade6
Make ljson more strict (#482) 2022-07-13 02:39:19 -07:00
Justine Tunney
30cc2c8dc1 Release redbean 2.0.12 2022-07-13 00:05:31 -07:00
Justine Tunney
e3cd476a9b Improve Lua and JSON serialization 2022-07-12 23:35:11 -07:00
Justine Tunney
3027d67037 Import more Musl math 2022-07-12 15:55:12 -07:00
Justine Tunney
3f3e7e92d7 Audit every single JSON test 2022-07-12 12:40:38 -07:00
Justine Tunney
6ee18986e4 Further improve JSON serialization 2022-07-11 23:06:49 -07:00
Justine Tunney
4814b6bdf8 Use ARM's faster math functions on non-tiny builds 2022-07-11 18:34:10 -07:00
William Chargin
0272f638a5
printimage: keep aspect ratio when scaling to fit (#479)
When `printimage` is invoked without `-w` or `-h`, we now preserve the
aspect ratio of each input image when scaling it to fit in the window.
A new flag `-i` ignores the aspect ratio, recovering the old behavior
when neither `-w` nor `-h` is passed. When `-i` is passed alongside
exactly one of `-w` or `-h`, the other dimension is just taken from the
window size, ignoring aspect ratio.

We also unconditionally print a newline between images to prevent them
from overlapping.

wchargin-branch: printimage-fit
wchargin-source: 4c2cfcffe9ce1a3b30c0ff051e3c6a2c166ae1c7
2022-07-11 15:58:21 -07:00
Gautham
11a1c62d11
Reduce JSON depth limit to 128 (#477)
- also remove tpyo from net.mk
- STACK_FRAME_UNLIMITED is not needed
- also let parser allow 0e1, 0e+1 as floats
2022-07-11 10:23:44 -07:00
Gautham
aeee2f9106
Add depth limit for JSON (#476) 2022-07-11 08:17:58 -07:00
Justine Tunney
3f015b1e51 Make some minor fixups to bug reporting, etc. 2022-07-11 05:58:24 -07:00
Justine Tunney
68ca49bfdd Improve APE install scripts and add uninstaller
See #350 thanks @tkchia
2022-07-10 14:13:45 -07:00
Justine Tunney
331fdd1d29 Fix some broken builds and broken hearts 2022-07-10 09:08:38 -07:00
Gautham
c0b325bafa
Add json.org tests for DecodeJson (#473) 2022-07-10 08:35:51 -07:00
Justine Tunney
5f4f6b0e69 Make _Thread_local work across platforms
We now rewrite the binary image at runtime on Windows and XNU to change
mov %fs:0,%reg instructions to use %gs instead. There's also simpler
threading API introduced by this change and it's called _spawn() and
_join(), which has replaced most clone() usage.
2022-07-10 04:01:17 -07:00
Justine Tunney
e4d6e263d4 Rename ParseJson() to DecodeJson() for consistency 2022-07-09 17:34:41 -07:00
Justine Tunney
28f0104330 Bump redbean up to 2.0.10 2022-07-09 17:28:14 -07:00
Justine Tunney
ee82cee432 Further improve json / lua serialization 2022-07-09 16:27:26 -07:00
Paul Kulchenko
9e86980191
Redbean fix listen leak (#470)
* Fix release of resources allocated to skipped sockets
* Update log messages for consistency
2022-07-09 12:16:44 -07:00
Justine Tunney
2189877856 Implement new JSON parser for redbean 2022-07-09 11:44:19 -07:00
Justine Tunney
727d9cbf56 Bump redbean to 2.0.9 2022-07-09 05:50:05 -07:00
Justine Tunney
c9e68b0ebc Make redbean serialization deterministic 2022-07-09 04:09:51 -07:00
William Chargin
a9145e7f1e
printimage: keep aspect ratio with -w xor -h (#468) 2022-07-09 01:38:13 -07:00
Justine Tunney
182d7720cf Fix GitHub actions
Our build was flaking due to ETXTBSY errors running multiple redbean
instances in parallel. This is due to the StoreAsset() support which
seems to cause enough problems it's worth making a *breaking change*
turning it off by default for now. There's a new -* flag, to restore
redbean's old self-modifying behavior.
2022-07-09 01:18:55 -07:00
Justine Tunney
1c83670229 Write more redbean unit tests
- Fix DescribeSigset()
- Introduce new unix.rmrf() API
- Fix redbean sigaction() doc example code
- Fix unix.sigaction() w/ more than two args
- Improve redbean re module API (non-breaking)
- Enhance Lua with Python string multiplication
- Make third parameter of unix.socket() default to 0
2022-07-08 23:10:02 -07:00
Justine Tunney
fe5c475f83 Add Deflate() / Inflate() to redbean and fix bugs
The Compress() and Uncompress() APIs were a mistake. The functions
themselves work fine, but it's a design blemish and does superfluous
work. Since they were only introduced in the last few weeks, they're now
deprecated and references to them have been scrubbed from the website
and other documentation. Please use the new APIs since the old APIs will
be removed at some point in the future.

This change introduces automated Lua unit tests for the Redbean APIs.
There's a few functions that were broken which have now been fixed, e.g.
Underlong() and Decimate().
2022-07-08 09:47:04 -07:00
Paul Kulchenko
a18044c504
Add OnServerListen hook to configure listen() (#459) 2022-07-08 07:17:25 -07:00
Paul Kulchenko
11ac8f11a9
Reduce redbean logging for EBADF (#461)
This may happen on explicitly closed client connections, so reduce the
logging level similar to other errors.
2022-07-08 07:14:50 -07:00
Paul Kulchenko
059fe22ea3
Improve redbean fetch() (#460)
* Updated Fetch to return `nil,error` on errors
* Fix localhost connect when only IP address is specified
2022-07-08 07:13:51 -07:00
Justine Tunney
853b6c3864 Improve system calls
- Wrap clock_getres()
- Wrap sched_setscheduler()
- Make sleep() api conformant
- Polyfill sleep() using select()
- Improve clock_gettime() polyfill
- Make nanosleep() POSIX conformant
- Slightly improve some DNS functions
- Further strengthen pledge() sandboxing
- Improve rounding of timeval / timespec
- Allow layering of pledge() calls on Linux
- Polyfill sched_yield() using select() on XNU
- Delete more system constants we probably don't need
2022-07-08 06:42:03 -07:00
Paul Kulchenko
5df3e4e7a8
Redbean zip and unix.poll doc update (#457) 2022-06-30 17:41:51 -07:00
Justine Tunney
3c92adfd6e Strengthen the pledge() polyfill 2022-06-27 13:02:17 -07:00
Paul Kulchenko
32eec7df4c
Update FATALF to exit without stack trace (#452) 2022-06-26 21:25:02 -07:00
Paul Kulchenko
283729dfe0
Redbean report write lock failure (#450)
This change updates redbean to report a non-writable path
when using StoreAsset, instead of crashing the worker.
2022-06-26 21:16:13 -07:00
Paul Kulchenko
b4e9a77e74
Improve redbean ZeroBrane integration (#451) 2022-06-26 21:13:53 -07:00
Justine Tunney
f6dab99f44 Release redbean 2.0.8 2022-06-26 17:25:54 -07:00
Justine Tunney
61257d48d4 Make some quick fixes and cleanup 2022-06-26 02:58:36 -07:00
Justine Tunney
fbc053e018 Make fixes and improvements
- Introduce __assert_disable global
- Improve strsignal() thread safety
- Make system call tracing thread safe
- Fix SO_RCVTIMEO / SO_SNDTIMEO on Windows
- Refactor DescribeFoo() functions into one place
- Fix fork() on Windows when TLS and MAP_STACK exist
- Round upwards in setsockopt(SO_RCVTIMEO) on Windows
- Disable futexes on OpenBSD which seem extremely broken
- Implement a better kludge for monotonic time on Windows
2022-06-25 21:09:09 -07:00
Paul Kulchenko
1bce1ca2e0
Polish redbean APIs and docs (#446)
* Remove undocumented ProgramSslCompression
* Rename GetLastModified to GetAssetLastModified
* Rename IsCompressed to IsAssetCompressed
* Reorganize currently deprecated functions
* Update sockopt documentation for clarity
2022-06-23 18:54:35 -07:00
Justine Tunney
a9d77362f9 Release redbean 2.0.7 2022-06-23 18:14:17 -07:00
Justine Tunney
4b9ee980a1 Release redbean 2.0.6 2022-06-23 17:37:55 -07:00
Justine Tunney
e55ef556aa Double default build process quota
See #430
Fixes #438
2022-06-23 16:46:12 -07:00
Tom Gillespie
7f26779d6f
Make build process quota overridable (#430)
P had the same assignment as F. This is now fixed so that it is
possible to increase RLIMIT_NPROC (e.g. when trying to build
on system accounts with more than 1024 processes running).
2022-06-23 16:40:47 -07:00
Justine Tunney
3d07f362d7 Release redbean 2.0.5 2022-06-23 16:09:20 -07:00
Justine Tunney
44da16255a Show crash reports on SIGSYS
This will help make it easier to troubleshoot ABI breakages with on
operating systems that, unlike Linux don't have ironclad guarantees
to not break userspace.
2022-06-23 13:01:01 -07:00
Justine Tunney
206f073181 Add stat mode macros to redbean unix api 2022-06-23 04:05:51 -07:00
Justine Tunney
17cbe73411 Add finger demo to redbean and fix regression
This change fixes a regression in unix.connect() caused by the recent
addition of UNIX domain sockets. The BSD finger command has been added
to third_party for fun and profit. A new demo has been added to redbean
showing how a protocol as simple as finger can be implemented.
2022-06-23 03:42:05 -07:00
Justine Tunney
fc097ac275 Add unix domain socket support to redbean 2022-06-22 03:04:25 -07:00
Justine Tunney
56eff8544f Fix V=0 make mode
See #431
2022-06-21 01:32:31 -07:00
Justine Tunney
a988896048 Add atomics to chibicc
This change also fixes #434 and makes the chibicc assembler better.
2022-06-20 03:08:00 -07:00
Justine Tunney
5ddf43332e Release redbean 2.0.4 2022-06-19 20:17:58 -07:00
Justine Tunney
2c7fe0f361 Fix redbean regression from previous commit 2022-06-19 00:59:07 -07:00
Justine Tunney
8c895453e2 Fix build break due to deleted file 2022-06-18 01:55:32 -07:00
Justine Tunney
c1cfca8ae1 Make fixes and improvements
- Polyfill UTIME_OMIT on XNU
- Refactor Lua build code so it's better
- Add unix module to lua.com (Discord request)
- Add unix.utimensat() and unix.futimens() to redbean
- Avoid creating double slash path in linenoise (#428)
- Remove double slashes in NT paths automatically (#428)
- Make strerror() smarter about showing NT errors (#428)

Fixes #428
2022-06-18 01:46:12 -07:00
Justine Tunney
67b28b9af1 Update redbean documentation 2022-06-17 08:05:47 -07:00
Justine Tunney
52ed099fe6 Favor $HOME directory over /tmp
This should strike a better compromise that keeps people happy about the
security of APE when it extracts the loader. This way systems with users
who aren't trusted (e.g. CPanel) won't be at any risk of compromise when
there isn't an `ape` loader on the system `$PATH`.

This change also bumps redbean up to 2.0.3
2022-06-17 07:56:54 -07:00
Justine Tunney
5e04827949 Add fileio and sqlar SQLite extensions to redbean 2022-06-17 06:04:09 -07:00