This change boosts SSL handshake performance from 2,627 to ~10,000 per
second which is the same level of performance as NGINX at establishing
secure connections. That's impressive if we consider that redbean is a
forking frontend application server. This was accomplished by:
1. Enabling either SSL session caching or SSL tickets. We choose to
use tickets since they reduce network round trips too and that's
a more important metric than wrk'ing localhost.
2. Fixing mbedtls_mpi_sub_abs() which is the most frequently called
function. It's called about 12,000 times during an SSL handshake
since it's the basis of most arithmetic operations like addition
and for some strange reason it was designed to make two needless
copies in addition to calling malloc and free. That's now fixed.
3. Improving TLS output buffering during the SSL handshake only, so
that only a single is write and read system call is needed until
blocking on the ping pong.
redbean will now do a better job wiping sensitive memory from a child
process as soon as it's not needed. The nice thing about fork is it's
much faster than reverse proxying so the goal is to use the different
address spaces along with setuid() to minimize the risk that a server
key will be compromised in the event that application code is hacked.
The following Lua APIs have been added:
- IsDaemon() → bool
- ProgramPidPath(str)
The following Lua hooks have been added:
- OnClientConnection(ip:int,port:int,serverip:int,serverport:int) → bool
- OnProcessCreate(pid:int,ip:int,port:int,serverip:int,serverport:int)
- OnProcessDestroy(pid:int)
- OnServerStart()
- OnServerStop()
- OnWorkerStart()
- OnWorkerStop()
redbean now does a better job at applying gzip on the fly from the local
filesystem, using a streaming chunked api with constant memory, which is
useful for doing things like serving a 4gb text file off NFS, and having
it start transmitting in milliseconds. redbean will also compute entropy
on the beginnings of files to determine if compression is profitable.
This change pays off technical debts relating to memory, such as relying
on exit() to free() allocations. That's now mostly fixed so it should be
easier now to spot memory leaks in malloc traces.
This change also fixes bugs and makes improvements to our SSL support.
Uniprocess mode failed handshakes are no longer an issue. Token Alpn is
offered so curl -v looks less weird. Hybrid SSL certificate loading is
now smarter about naming conflicts. Self-signed CA root anchors will no
longer be delivered to the client during the handshake.
The implementations of the getproto* functions follow from the getserv*
functions: same static name allocation, same type of internal function
that opens a file to search, aliases are not written to the struct, same
type of error handling/returns.
This changes also fixes a getaddrinfo AI_PASSIVE memory error. When
getaddrinfo is passed name = NULL and AI_PASSIVE in hints->ai_flags, it was
setting the s_addr value to INADDR_ANY but *not* returning the addrinfo
pointer via *res = ai. This caused a free(NULL) memory error when the caller
tried to free res, because the caller expects res to be a valid pointer to a
struct addrinfo.
Our non-standard API parseport() has been updated to use strtoimax.
strtoimax has an extra parameter endptr to store where the parsing was
terminated. endptr is used in parseport to check if the provided string
was valid.
- Fix Content-Type inference when file extension has number
- Remove shoddy Class A granular IP classiifcation
- Have setuid() and setgid() take effect w/o daemonization
- Make GetParams() return empty table instead of nil
- Change SetLogLevel(int) to only apply to one message
- Make SetLogLevel(int) good enough to be access_log off
- Introduce ProgramUid(int) which is same as -U INT
- Introduce ProgramGid(int) which is same as -G INT
- Introduce ProgramLogPath(str) which is same as -L PATH
- Introduce ProgramDirectory(str) which is same as -D PATH
- Introduce ProgramLogBodies(bool) which is same as -b
- Introduce ProgramLogMessages(bool) which is same as -m
You can now say the following in your redbean Lua code:
status,headers,payload = Fetch("https://foo.example")
The following Lua APIs have been introduced:
- Fetch(str) → str,{str:str},str
- GetHttpReason(int) → str
- GetHttpReason(int) → str
- ProgramSslFetchVerify(bool)
- ProgramSslClientVerify(bool)
The following flags have been introduced:
- `-j` enables client SSL verification
- `-k` disables Fetch() SSL verification
- `-t INT` may now be passed a negative value for keepalive
Lua exceptions now invoke Cosmopolitan's garbage collector when
unwinding the stack. So it's now safe to use _gc() w/ Lua 𝔱𝔥𝔯𝔬𝔴
See #97
Things are a little better. The LLD that comes with Linux seems to work.
Old versions like LLVM 8 haven't been supported since Cosmopolitan v0.2.
Running Clang on Windows with --target=x86_64-pc-linux-gnu doesn't seem
to work. It has something to do with the recently added .zip section in
the linker script. But even if that's removed, LLD on Windows thinks it
is building an EFI application for some reason. Linker scripts are such
a brittle house of cards, even for just ld.bfd alone..
We should just find a way to run our one true musl-cross-make linux gcc
toolchain under Blinkenlights on non-Linux because GCC and Clang are so
nondeterministic, inconsistent, and unreproducible when built for other
operating systems. We need an actually portable compiler/linker that'll
always behave the same way no matter what.
See #180
For every call of getservbyname/getservbyport the lookup of
/etc/services is done by opening the file and parsing each line
one-by-one. This is slow, but the implementation is simple.
This change also adds fixes for the gethostbyname function.
Under MODE=tiny or MODE=tinylinux we now go back to using my homebrew
version of DEFLATE decompression which is 10x smaller but 10x slower
than Chromium Zlib. In tiny mode we also disable compressed responses
howewver redbean will still serve precompressed responses. This change
also removes a few other dependencies like strftime() and getaddrinfo()
which means you can't say `-l localhost` in tiny mode, you have to say
something like `-l 127.0.0.1`.
m=tinylinux
make -j8 MODE=$m o/$m/tool/net/redbean-original.com
ls -hal o/$m/tool/net/redbean-original.com
This change reduces the above size from 191.4 to 150.9 kb.
gethostbyname, gethostbyaddr follow simple implementations: they
internally call getaddrinfo and getnameinfo respectively, and fill out
the minimum details. remaining functions are stubs.
- Use nullness checks when calling weakly linked functions.
- Avoid typedef for reasons described in Linux Kernel style guide.
- Avoid enum in in Windows headers. Earlier in Cosmo's history all one
hundred files in libc/nt/enum/ used to be enums and it resulted in
gigabytes of DWARF data almost as large as everything else in the
codebase combined.
- Bitfields aren't our friends. They have frequent ABI breakages,
inconsistent arithmetic across compilers, and different endianness
between cpus. Compiler authors also haven't invested much roi into
making bit fields go fast so they produce poor assembly.
- Use memccpy() instead of strncpy() or snprintf() for length-bounded
copying of C strings. strncpy() is a misunderstood function and
snprintf() is awesome but memccpy() deserves more love.
Your redbean can now interoperate with clients that require TLS crypto.
This is accomplished using a protocol polyglot that lets us distinguish
between HTTP and HTTPS regardless of the port number. Certificates will
be generated automatically, if none are supplied by the user. Footprint
increases by only a few hundred kb so redbean in MODY=tiny is now 1.0mb
- Add lseek() polyfills for ZIP executable
- Automatically polyfill /tmp/FOO paths on NT
- Fix readdir() / ftw() / nftw() bugs on Windows
- Introduce -B flag for slower SSL that's stronger
- Remove mbedtls features Cosmopolitan doesn't need
- Have base64 decoder support the uri-safe alternative
- Remove Truncated HMAC because it's forbidden by the IETF
- Add all the mbedtls test suites and make them go 3x faster
- Support opendir() / readdir() / closedir() on ZIP executable
- Use Everest for ECDHE-ECDSA because it's so good it's so good
- Add tinier implementation of sha1 since it's not worth the rom
- Add chi-square monte-carlo mean correlation tests for getrandom()
- Source entropy on Windows from the proper interface everyone uses
We're continuing to outperform NGINX and other servers on raw message
throughput. Using SSL means that instead of 1,000,000 qps you can get
around 300,000 qps. However redbean isn't as fast as NGINX yet at SSL
handshakes, since redbean can do 2,627 per second and NGINX does 4.3k
Right now, the SSL UX story works best if you give your redbean a key
signing key since that can be easily generated by openssl using a one
liner then redbean will do all the things that are impossibly hard to
do like signing ecdsa and rsa certificates that'll work in chrome. We
should integrate the let's encrypt acme protocol in the future.
Live Demo: https://redbean.justine.lol/
Root Cert: https://redbean.justine.lol/redbean1.crt
This change configures Mbed TLS to support the fewest number of things
possible required to run an HTTPS server that caters to the sweet spot
of being legacy enough to support the vast majority of user agents but
modern enough that Chrome and Firefox remain happy. That should entail
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_RSA_WITH_AES_128_CBC_SHA
Even though other suites still get included so what usually happens in
practice is ECDHE-RSA-AES256-GCM-SHA384 under TLS 1.2 will be selected
and the binary footprint is reasonable, and should cost us about 200kb
- SIOCGIFCONFIG: reads and enumerate all the network interfaces
- SIOCGIFADDR: reads network address for a given interface
- SIOCGIFFLAGS: reads network flags for a given interface
- SIOCGIFNETMASK: reads network netmask for a given interface
- SIOCGIFBRDADDR: reads network broadcast address for a given interface
- SIOCGIFDSTADDR: reads peer destination address for a given
interface (not supported for Windows)
This change defines Linux ABI structs for the above interfaces and adds
polyfills to ensure they behave consistently on XNU and Windows.
* removed unnecessary files (like amiga/osdep.h)
* makefile has 4 targets: zip, zipnote, zipcloak, zipsplit
* added clang-format off at the start of all source files
* added necessary headers
The getnameinfo implementation requires an address -> name lookup on the
hosts file (ie struct HostsTxt) and the previous implementation used
flags to check whether HostsTxt was sorted according to address or name,
and then re-sorted it if necessary. Now getnameinfo lookup does not
require sorting, it does a simple linear lookup, and so the related code
was simplified
See #172 for discussion.
This change gets redbean SQLite working in write mode on Windows.
Warnings have been added to the appropriate and responsible places.
Hacking proprietary PC systems into production-worthy servers isn't
terribly high on the list of priorities. Consider BSD or Linux when
building online systems that service requests from multiple people.
Fixes#193
