Make improvements to redbean

The following Lua APIs have been added:

  - IsDaemon() → bool
  - ProgramPidPath(str)

The following Lua hooks have been added:

  - OnClientConnection(ip:int,port:int,serverip:int,serverport:int) → bool
  - OnProcessCreate(pid:int,ip:int,port:int,serverip:int,serverport:int)
  - OnProcessDestroy(pid:int)
  - OnServerStart()
  - OnServerStop()
  - OnWorkerStart()
  - OnWorkerStop()

redbean now does a better job at applying gzip on the fly from the local
filesystem, using a streaming chunked api with constant memory, which is
useful for doing things like serving a 4gb text file off NFS, and having
it start transmitting in milliseconds. redbean will also compute entropy
on the beginnings of files to determine if compression is profitable.

This change pays off technical debts relating to memory, such as relying
on exit() to free() allocations. That's now mostly fixed so it should be
easier now to spot memory leaks in malloc traces.

This change also fixes bugs and makes improvements to our SSL support.
Uniprocess mode failed handshakes are no longer an issue. Token Alpn is
offered so curl -v looks less weird. Hybrid SSL certificate loading is
now smarter about naming conflicts. Self-signed CA root anchors will no
longer be delivered to the client during the handshake.
This commit is contained in:
Justine Tunney 2021-07-10 15:02:03 -07:00
parent 98c674d915
commit 8c4cce043c
25 changed files with 22326 additions and 359 deletions

View file

@ -722,25 +722,17 @@ static const struct DisArg {
{"Yvqp", DisYvqp}, //
};
static int CompareString8(const char a[8], const char b[8]) {
uint64_t x, y;
x = READ64BE(a);
y = READ64BE(b);
return x > y ? 1 : x < y ? -1 : 0;
}
char *DisArg(struct Dis *d, char *p, const char *s) {
char key[8];
int c, m, l, r;
char k[8];
int m, l, r;
l = 0;
r = ARRAYLEN(kDisArgs) - 1;
strncpy(key, s, 8);
strncpy(k, s, 8);
while (l <= r) {
m = (l + r) >> 1;
c = CompareString8(kDisArgs[m].s, key);
if (c < 0) {
if (READ64BE(kDisArgs[m].s) < READ64BE(k)) {
l = m + 1;
} else if (c > 0) {
} else if (READ64BE(kDisArgs[m].s) > READ64BE(k)) {
r = m - 1;
} else {
return kDisArgs[m].f(d, d->xedd->op.rde, p);

View file

@ -192,6 +192,45 @@ SECURITY
-addext 'keyUsage = critical,cRLSign,keyCertSign'
sudo ./redbean.com -C ca.crt -K .ca.key -p 80 -p 443
Let's say you're migrating away from NGINX and you use Let's Encrypt.
In that case you'll likely want something like the following:
certbot certonly --nginx --key-type ecdsa \
--cert-name redbean-ecdsa -d redbean.dev -d www.redbean.dev
certbot certonly --nginx --key-type rsa \
--cert-name redbean-rsa -d redbean.dev -d www.redbean.dev
You can then program /var/www/html/.init.lua as such:
ProgramPrivateKey(Slurp('/etc/letsencrypt/live/redbean-ecdsa/privkey.pem'))
ProgramCertificate(Slurp('/etc/letsencrypt/live/redbean-ecdsa/fullchain.pem'))
ProgramPrivateKey(Slurp('/etc/letsencrypt/live/redbean-rsa/privkey.pem'))
ProgramCertificate(Slurp('/etc/letsencrypt/live/redbean-rsa/fullchain.pem'))
if IsDaemon() then
ProgramUid(33) # see `vipw` to get appropriate number
ProgramGid(33) # see `vigr` to get appropriate number
ProgramPort(80)
ProgramPort(443)
ProgramLogPath('/var/log/redbean.log')
ProgramPidPath('/var/run/redbean.pid')
end
function OnHttpRequest()
path = GetPath()
if path == '/favicon.ico' or
path == '/site.webmanifest' or
path == '/favicon-16x16.png' or
path == '/favicon-32x32.png' or
path == '/apple-touch-icon' then
SetLogLevel(kLogWarn)
end
Route()
SetHeader('Content-Language', 'en-US')
end
You'd then run redbean as follows:
redbean.com -dD /var/www/html
SSL verbosity is controlled as follows for troubleshooting:
-V log ssl errors
@ -295,13 +334,51 @@ GLOBALS
HOOKS
OnHttpRequest
OnHttpRequest()
If this function is defined in the global scope by your /.init.lua
then redbean will call it at the ealiest possible moment to
handover control for all messages (with the exception of OPTIONS
*). See functions like Route which asks redbean to do its default
thing from the handler.
OnClientConnection(ip:int,port:int,serverip:int,serverport:int) → bool
If this function is defined it'll be called from the main process
each time redbean accepts a new client connection. If it returns
true then redbean will close the connection without calling fork.
OnProcessCreate(pid:int,ip:int,port:int,serverip:int,serverport:int)
If this function is defined it'll be called from the main process
each time redbean forks a connection handler worker process. The
ip/port of the remote client is provided, along with the ip/port
of the listening interface that accepted the connection. This may
be used to create a server activity dashboard, in which case the
data provider handler should set SetHeader('Connection','Close').
This won't be called in unprocess mode.
OnProcessDestroy(pid:int)
If this function is defined it'll be called from the main process
each time redbean reaps a child connection process using wait4().
This won't be called in unprocess mode.
OnServerStart()
If this function is defined it'll be called from the main process
right before the main event loop starts.
OnServerStop()
If this function is defined it'll be called from the main process
after all the connection processes have been reaped and exit() is
ready to be called.
OnWorkerStart()
If this function is defined it'll be called from the child worker
process after it's been forked and before messages are handled.
This won't be called in unprocess mode.
OnWorkerStop()
If this function is defined it'll be called from the child worker
process once _exit() is ready to be called. This won't be called
in unprocess mode.
FUNCTIONS
Write(data:str)
@ -708,6 +785,9 @@ FUNCTIONS
ProgramPrivateKey(Slurp("/etc/letsencrypt/fullchain.pem")) for
local file system only.
IsDaemon() → bool
Returns true if -d flag was passed to redbean.
ProgramUid(int)
Same as the -U flag if called from .init.lua for setuid()
@ -736,6 +816,16 @@ FUNCTIONS
space then redbean will truncate the log file if has access to
change the log file after daemonizing.
ProgramPidPath(str)
Same as the -P flag if called from .init.lua for setting the pid
file path on the local file system. It's useful for reloading
daemonized redbean using `kill -HUP $(cat /var/run/redbean.pid)`
or terminating redbean with `kill $(cat /var/run/redbean.pid)`
which will gracefully terminate all clients. Sending the TERM
signal twice will cause a forceful shutdown, which might make
someone with a slow internet connection who's downloading big
files unhappy.
Slurp(filename:str) → str
Reads file data from local file system.
@ -792,21 +882,39 @@ FUNCTIONS
Underlong(str) → str
Canonicalizes overlong encodings.
bsf(x:int) → int
Crc32(initial:int,data:str) → int
Computes 32-bit CRC-32 used by zip/zlib/gzip/etc.
Crc32c(initial:int,data:str) → int
Computes 32-bit Castagnoli Cyclic Redundancy Check.
Md5(str) → str
Computes MD5 checksum, returning 16 bytes of binary.
Sha1(str) → str
Computes SHA1 checksum, returning 20 bytes of binary.
Sha224(str) → str
Computes SHA224 checksum, returning 28 bytes of binary.
Sha256(str) → str
Computes SHA256 checksum, returning 32 bytes of binary.
Sha384(str) → str
Computes SHA384 checksum, returning 48 bytes of binary.
Sha512(str) → str
Computes SHA512 checksum, returning 64 bytes of binary.
Bsf(x:int) → int
Returns position of first bit set. Passing 0 will raise an error.
Same as the Intel x86 instruction BSF.
bsr(x:int) → int
Bsr(x:int) → int
Returns binary logarithm of 𝑥. Passing 0 will raise an error. Same
as the Intel x86 instruction BSR.
crc32(initial:int,data:str) → int
Computes Phil Katz CRC-32 used by zip/zlib/gzip/etc.
crc32c(initial:int,data:str) → int
Computes 32-bit Castagnoli Cyclic Redundancy Check.
popcnt(x:int) → int
Popcnt(x:int) → int
Returns number of bits set in integer.
LSQLITE3 MODULE

File diff suppressed because it is too large Load diff

93
tool/viz/dumphexc.c Normal file
View file

@ -0,0 +1,93 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
vi: set net ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi
Copyright 2021 Justine Alexandra Roberts Tunney
Permission to use, copy, modify, and/or distribute this software for
any purpose with or without fee is hereby granted, provided that the
above copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE
AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL
DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR
PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE.
*/
#include "libc/fmt/fmt.h"
#include "libc/macros.internal.h"
#include "libc/stdio/stdio.h"
struct Append {
size_t i, n;
char *p;
};
int AppendFmt(struct Append *b, const char *fmt, ...) {
int n;
char *p;
va_list va, vb;
va_start(va, fmt);
va_copy(vb, va);
n = vsnprintf(b->p + b->i, b->n - b->i, fmt, va);
if (n >= b->n - b->i) {
do {
if (b->n) {
b->n += b->n >> 1; /* this is the important line */
} else {
b->n = 16;
}
} while (b->i + n + 1 > b->n);
b->p = realloc(b->p, b->n);
vsnprintf(b->p + b->i, b->n - b->i, fmt, vb);
}
va_end(vb);
va_end(va);
b->i += n;
return n;
}
char *DumpHexc(const char *data, size_t size, size_t *z) {
long o;
int b, i, n;
char A[128], *p;
struct Append buf;
if (size == -1) size = data ? strlen(data) : 0;
buf.i = 0;
buf.n = 256;
buf.p = calloc(1, 256);
AppendFmt(&buf, "\"\\\n");
for (b = o = 0; (n = MIN(16, size)); data += n, size -= n) {
p = A;
for (i = 0; i < n; ++i) {
*p++ = '\\';
*p++ = 'x';
*p++ = "0123456789abcdef"[(data[i] & 0xF0) >> 4];
*p++ = "0123456789abcdef"[(data[i] & 0x0F) >> 0];
}
if (o) AppendFmt(&buf, "\\\n");
AppendFmt(&buf, "%.*s", p - A, A);
o += n;
}
AppendFmt(&buf, "\"");
if (z) *z = buf.i;
return buf.p;
}
int main(int argc, char *argv[]) {
char *p;
size_t n, g;
char buf[512];
struct Append b = {0};
while ((g = fread(buf, 1, sizeof(buf), stdin))) {
AppendFmt(&b, "%.*s", g, buf);
}
if (!ferror(stdin)) {
p = DumpHexc(b.p, b.i, &n);
fwrite(p, 1, n, stdout);
}
printf("\n");
return ferror(stdin) || ferror(stdout) ? 1 : 0;
}