Commit graph

178 commits

Author SHA1 Message Date
Paul Kulchenko
c3fb624647
Redbean fix text detect (#388)
This resets istext between requests, which may cause response to be
zipped when not needed (on non-text content).
2022-04-20 00:24:25 -07:00
Justine Tunney
5a132f9652 Add seccomp bpf sandboxing to redbean
It's now possible to pass the `-S` or `-SS` flags to sandbox redbean
worker proecsses after they've been forked. The first `-S` flag is
intended to be a permissive builtin policy that limits system calls to
only that which the various parts of redbean serving need. The second
`-SS` flag is intended to be more restrictive, preventing things like
the Lua extensions you download off the web from using the HTTP client
or sockets APIs. In upcoming changes you'll be able to implement your
own Berkeley Packet Filter sandbox programs and load them via Lua.
2022-04-18 08:54:42 -07:00
Justine Tunney
7166679620 Fix bugs and add security features to redbean
- Fix a regression with the previous change that broke redbean
- Add chroot(), resource limit, seccomp, and other stuff to redbean
- Write lots and lots of documentation
- Iron out more system call issues
2022-04-18 00:01:26 -07:00
Justine Tunney
ab38f0823d Make small fixes 2022-04-17 10:40:32 -07:00
Justine Tunney
80308079ec Disable redbean repl on windows 2022-04-16 23:52:03 -07:00
Justine Tunney
a6b02ce5a6 Add lua repl interface to redbean
You can now interact with the global web server state on the command
line, which the web server is running. This supports Emacs shortcuts
with history, readline parity, <tab> completions, plus hints. Enjoy!
2022-04-16 20:31:16 -07:00
Justine Tunney
dc0ea6640e Fix bugs with recent change
This change makes further effort towards improving our poll()
implementation on the New Technology. The stdin worker didn't work out
so well for Python so it's not being used for now. System call tracing
with the --strace flag should now be less noisy now on Windows unless
you modify the strace.internal.h defines to turn on some optional ones
that are most useful for debugging the system call wrappers.
2022-04-16 10:40:23 -07:00
Justine Tunney
933411ba99 Improve synchronization
- Fix bugs in kDos2Errno definition
- malloc() should now be thread safe
- Fix bug in rollup.com header generator
- Fix open(O_APPEND) on the New Technology
- Fix select() on the New Technology and test it
- Work towards refactoring i/o for thread safety
- Socket reads and writes on NT now poll for signals
- Work towards i/o completion ports on the New Technology
- Make read() and write() intermittently check for signals
- Blinkenlights keyboard i/o so much better on NT w/ poll()
- You can now poll() files and sockets at the same time on NT
- Fix bug in appendr() that manifests with dlmalloc footers off
2022-04-15 15:31:55 -07:00
Paul Kulchenko
233144b19d
Fix memory deallocation while yielding in redbean. (#384)
The yielded coroutine was removed from the stack too early,
leaving it not being anchored, which led to memory freed prematurely.
2022-04-14 18:13:53 -07:00
Justine Tunney
fb7e8ef1e6 Add more raw system calls to redbean
We now have execve, setitimer, sigaction, sigsuspend, and sigprocmask.
2022-04-13 14:43:42 -07:00
Justine Tunney
281a0f2730 Implement raw system call for redbean lua code
You can now call functions like fork() from Lua and it'll work across
all supported platforms, including Windows. This gives you a level of
control of the system that Lua traditionally hasn't been able to have
due to its focus on old portable stdio rather modern POSIX APIs. Demo
code has been added to redbean-demo.com to show how it works.

This change also modifies Lua so that integer literals with a leading
zero will be interpreted as octal. That should help avoid shooting in
the foot with POSIX APIs that frequently use octal mode bits.

This change fixes a bug in opendir(".") on New Technology.

Lastly, redbean will now serve crash reports to private network IPs.
This is consistent with other frameworks. However that isn't served
to public IPs unless the -E flag is passed to redbean at startup.
2022-04-13 08:53:24 -07:00
Justine Tunney
f684e348d4 Improve signals and memory protection
- Document sigaction()
- Simplify New Technology fork() code
- Testing and many bug fixes for mprotect()
- Distribute Intel Xed ILD in the amalgamation
- Turn Xed enums into defines to avoid DWARF bloat
- Improve polyfilling of SA_SIGINFO on BSDs and fix bugs
- setpgid(getpid(), getpid()) on Windows will ignore CTRL-C
- Work around issues relating to NT mappings being executable
- Permit automatic executable stack override via `ape_stack_pf`
2022-04-12 22:11:00 -07:00
Paul Kulchenko
668dc42bac
Reset Lua stack to drop unused returned values (#382)
Since some Lua code may return values that will be left on stack, they
need to be removed to avoid growing the stack unnecessarily.
2022-04-09 12:36:05 -07:00
Justine Tunney
c13142dad2 Fix redbean certificate free error
This fixes a regression from a change a few weeks ago in git commit
af645fcbec which caused certificates to
not be free()'d correctly if the certificates are chained. dlmalloc()
should have printed an error in most build modes. ASAN caught it too.
2022-04-06 11:05:34 -07:00
Justine Tunney
3b9e66ecba Add Linux ptrace() tutorial 2022-03-24 08:00:36 -07:00
Justine Tunney
23b72eb617 Add support for symbol table in .com files
This change fixes minor bugs and adds a feature, which lets us store the
ELF symbol table, inside the ZIP directory. We use the path /zip/.symtab
which can be safely removed using a zip editing tool, to make the binary
smaller after compilation. This supplements the existing method of using
a separate .com.dbg file, which is still supported. The intent is people
don't always know that it's a good idea to download the debug file. It's
not great having someone's first experience be a crash report, that only
has numbers rather than symbols. This will help fix that!
2022-03-23 06:34:46 -07:00
Justine Tunney
393ca4be40 Polyfill SIGCHLD on the New Technology
redbean will now cleanup child processes properly. New accounting
information is available too, such as page faults and memory usage. The
way it works is Cosmopolitan Libc samples the process collection on
entry into read() and poll() to see if SIGCHLD needs to be raised.

This change also fixes an issue with chibicc /tmp cleanup. There was
also a regression in MODE=dbg because STL needed ASAN runtime support.
2022-03-22 21:31:12 -07:00
Justine Tunney
5022f9e920 Trim down redbean a little bit
This makes redbean.c a little less long. It also reduces the size of
redbean-original.com from being 333K to 213K.
2022-03-21 19:32:30 -07:00
Paul Kulchenko
87029ac3f9
Redbean stream yield implementation (#370)
* Simplify handling of coroutine state
* Update redbean to allow yielding from Lua to support streaming
* Add stack checks for Lua resume calls in redbean
2022-03-21 18:07:30 -07:00
Justine Tunney
e5314dedde Fix performance bottlenecks with nt fork redbean 2022-03-21 08:25:33 -07:00
Justine Tunney
d57f87dc40 Fix some issues with zipos and redbean
- redbean.com -D /zip/dir/ now works, for pure fun
- possibly fixed bug with redbean serving empty files
- zipos stat() mode now indicates directories on windows

See #372
2022-03-21 07:36:36 -07:00
Nick Owens
f78f2fcac3
redbean: allow symlinks for -D path (#372)
Previously ProgramDirectory called isdirectory which returns false for
symlinks. instead just check directory with stat, and allow user to pass
`-D /tmp/foo/`, where /tmp/foo might be a symlink.
2022-03-21 05:43:49 -07:00
Justine Tunney
5e8ae2d5bc Restart CI for New Technology and UBSAN hunting
Continuous Integration (via runit and runitd) is now re-enabled on win7
and win10. The `make test` command, which runs the tests on all systems
is now the fastest and most stable it's been since the project started.

UBSAN is now enabled in MODE=dbg in addition to ASAN. Many instances of
undefined behavior have been removed. Mostly things like passing a NULL
argument to memcpy(), which works fine with Cosmopolitan Libc, but that
doesn't prevents the compiler from being unhappy. There was an issue w/
GNU make where static analysis claims a sprintf() call can overflow. We
also now have nicer looking crash reports on Windows since uname should
now be supported and msys64 addr2line works reliably.
2022-03-21 04:32:57 -07:00
Paul Kulchenko
d5ff2c3fb9
Make improvements to redbean (#373)
* Allow ProgramUniprocess to be set on Windows
* Add closing p tag to Maxmind example
* Enable using symlinked directories with redbean -A flag
* Rename gc in redbean for consistency
* Improve StoreFile by adding checks and skipping ./ in stored path
* Add freeing memory in redbean LuaEncodeUrl

Ref #366
2022-03-21 03:20:09 -07:00
Justine Tunney
0cb6b6ff4b Get Redbean fork() working on the New Technology
Now that we have understandable system call tracing on Windows, this
change rewrites many of the polyfill internals for that platform, to
help things get closer to tip top shape. Support for complex forking
scenarios had been in a regressed state for quite some time. Now, it
works! Subsequent changes should be able to address the performance.
2022-03-20 08:01:14 -07:00
Paul Kulchenko
efedef6e65
Fix Lua EncodeUrl without values (#368) 2022-03-19 18:00:15 -07:00
Justine Tunney
39688a73e4 Polish recent changes and make improvements
- Simulate SIGPIPE on Windows NT
- Fix commandv() regression on Windows NT
- Fix sigprocmask() strace bug on OpenBSD
- Add many more system calls to --strace logging
- Make errno state more pristine in redbean strace
2022-03-19 03:37:00 -07:00
Justine Tunney
14e192e5ba Introduce --strace flag for system call tracing
This is similar to the --ftrace (c function call trace) flag, except
it's less noisy since it only logs system calls to stderr. Having this
flag is valuable because (1) system call tracing tells us a lot about
the behavior of complex programs and (2) it's usually very hard to get
system call tracing on various operating systems, e.g. strace, ktrace,
dtruss, truss, nttrace, etc. Especially on Apple platforms where even
with the special boot trick, debuggers still aren't guaranteed to work.

    make -j8 o//examples
    o//examples/hello.com --strace

This is enabled by default in MODE=, MODE=opt, and MODE=dbg. In MODE=dbg
extra information will be printed.

    make -j8 MODE=dbg o/dbg/examples
    o/dbg/examples/hello.com --strace |& less

This change also changes:

- Rename IsText() → _istext()
- Rename IsUtf8() → _isutf8()
- Fix madvise() on Windows NT
- Fix empty string case of inet_ntop()
- vfork() wrapper now saves and restores errno
- Update xsigaction() to yoink syscall support
2022-03-18 18:07:28 -07:00
Justine Tunney
c371db6663 Add maxmind to redbean 2022-03-18 03:17:08 -07:00
Justine Tunney
af645fcbec Make exciting improvements
- Add Lua backtraces to redbean!
- Wipe serving keys after redbean forks
- Audit redbean to remove free via exit
- Log SSL client ciphersuite preferences
- Increase ASAN malloc() backtrace depth
- Make GetSslRoots() behave as a singleton
- Move leaks.c from LIBC_TESTLIB to LIBC_LOG
- Add undocumented %n to printf() for newlines
- Fix redbean memory leak reindexing inode change
- Fix redbean memory leak with Fetch() DNS object
- Restore original environ after __cxa_finalize()
- Make backtrace always work after __cxa_finalize()
- Introduce COUNTEXPR() diagnostic / benchmark tool
- Fix a few more instances of errno being clobbered
- Consolidate the ANSI color disabling internal APIs
2022-03-18 03:02:00 -07:00
Paul Kulchenko
2a938b3eaa
Use last X-Forwarded-For header (#367)
This header is non-standard but AWS seems to need this.
2022-03-14 17:21:15 -07:00
Paul Kulchenko
cfc557f7c7
Add storing folders in redbean from CLI (#366) 2022-03-14 17:19:31 -07:00
Paul Kulchenko
38112aeb20
Fix Redbean when file is read-only (#365) 2022-03-14 17:13:28 -07:00
Paul Kulchenko
22409b2b5e
Redbean SSL identification (#360)
* Let Fetch() be used earlier in initialization
* Have ssl log messages show cert name
* Introduce GetSslIdentity Lua API
2022-03-14 17:11:05 -07:00
Paul Kulchenko
abac6f729c
Add ProgramUniprocess to redbean (#364) 2022-03-07 18:15:44 -08:00
Paul Kulchenko
1e3c5e10ad
Update docs on chmod permissions (#336)
Closes #335
2022-03-07 18:13:49 -08:00
Paul Kulchenko
4abae20172
Redbean StoreAsset fix and lua cli (#326)
* Fix StoreAsset update for existing assets in redbean
* Add lua code execution and asset storage from redbean command line
2022-03-04 18:47:15 -08:00
Paul Kulchenko
9bab356c37
Fix double Content-Type after SetHeader (#327) 2022-03-04 18:44:39 -08:00
Paul Kulchenko
d938b89f4f
Fix redbean browser launch before server starts (#359) 2022-03-04 17:41:19 -08:00
Justine Tunney
d6a039821f Release redbean 1.5 2021-11-15 07:39:38 -08:00
Paul Kulchenko
fdb543cbb3
Fix incorrect reference in Lua encoding (#324) 2021-11-13 13:10:09 -08:00
Paul Kulchenko
1d6216a775
Add encode json and encode Lua functions to redbean (#322) 2021-11-13 12:49:29 -08:00
Justine Tunney
777d08a839 Fix mistake with IsHeaderRepeatable 2021-11-12 20:54:30 -08:00
Justine Tunney
98192c1079 Add undocumented IsHeaderRepeatable to redbean
This API is intended to help unblock framework code. It should
ideally be abstracted by redbean which is why it's undocumented.

See #97
2021-11-12 19:03:35 -08:00
Justine Tunney
245a9c206b Fix redbean build
Also ran clang-format on redbean.c

See #312 and #321
2021-11-12 16:34:18 -08:00
Paul Kulchenko
1bdc8faa65
Add redbean function for simple HMAC (#321) 2021-11-12 16:26:14 -08:00
Paul Kulchenko
8f05990d5a
Extend GetZipPaths to accept an optional prefix (#320) 2021-11-12 15:28:05 -08:00
Gautham
6f658f058b
Change noinline to dontinline (#312)
We defined `noinline` as an abbreviation for the longer version
`__attribute__((__noinline__))` which caused name clashes since
third party codebases often write it as `__attribute__((noinline))`.
2021-11-12 15:12:18 -08:00
Paul Kulchenko
ca611efc43
Redbean getstatus and more (#308)
* Add GetBody() Lua API to redbean.
   This improves consistency with RFC 7230 terminology and
   should be favored over the old GetPayload function.
* Add GetStatus() API to redbean.
   This is useful to get status after it's changed/set by Redbean,
   for example if 505 or 508 is set when ServeRedirect is called.
* Introduce GetAssetComment() API to redbean.
   This function should be favored over the old name GetComment().
* Introduce IsLoopbackClient() API to redbean
* Limit redbean reason to 128 chars when set instead of reporting an error
2021-11-12 15:00:41 -08:00
Paul Kulchenko
660ff56d40
Redbean fix lua panic (#300)
* Fix Lua panic in redbean when calling request/connection functions from .init.lua

The error calls were triggered appropriately, but used the global Lua
state instead of the current Lua state (within protected call), which
triggered Lua panic. This executes the error in the proper context.

* Rename global Lua state object to make its (mis-)usage easier to detect
2021-10-25 14:54:56 -07:00