mirrors/cosmopolitan
1
0
Fork 0
mirror of https://github.com/jart/cosmopolitan.git synced 2025-07-04 10:18:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Make fixes and improvements

Browse source 
- Invent iso8601us() for faster timestamps
- Improve --strace descriptions of sigset_t
- Rebuild the Landlock Make bootstrap binary
- Introduce MODE=sysv for non-Windows builds
- Permit OFD fcntl() locks under pledge(flock)
- redbean can now protect your kernel from ddos
- Have vfork() fallback to sys_fork() not fork()
- Change kmalloc() to not die when out of memory
- Improve documentation for some termios functions
- Rewrite putenv() and friends to conform to POSIX
- Fix linenoise + strace verbosity issue on Windows
- Fix regressions in our ability to show backtraces
- Change redbean SetHeader() to no-op if value is nil
- Improve fcntl() so SQLite locks work in non-WAL mode
- Remove some unnecessary work during fork() on Windows
- Create redbean-based SSL reverse proxy for IPv4 TurfWar
- Fix ape/apeinstall.sh warning when using non-bash shells
- Add ProgramTrustedIp(), and IsTrustedIp() APIs to redbean
- Support $PWD, $UID, $GID, and $EUID in command interpreter
- Introduce experimental JTqFpD APE prefix for non-Windows builds
- Invent blackhole daemon for firewalling IP addresses via UNIX named socket
- Add ProgramTokenBucket(), AcquireToken(), and CountTokens() APIs to redbean
This commit is contained in:
Justine Tunney 2022-10-17 11:02:04 -07:00
parent 648bf6555c
commit f7ff77d865
No known key found for this signature in database
GPG key ID: BE714B4575D6E328
209 changed files with 3818 additions and 998 deletions
Download patch file Download diff file Expand all files Collapse all files

154
tool/net/help.txt
View file

@ -1878,6 +1878,160 @@ FUNCTIONS
string of unspecified format describing the error. Calls to this
function may be wrapped in assert() if an exception is desired.
IsTrustedIp(ip:int)
└─→ bool
Returns true if IP address is trustworthy.
If the ProgramTrustedIp() function has NOT been called then redbean
will consider the networks 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12,
and 192.168.0.0/16 to be trustworthy too. If ProgramTrustedIp() HAS
been called at some point earlier in your redbean's lifecycle, then
it'll trust the IPs and network subnets you specify instead.
The network interface addresses used by the host machine are always
considered trustworthy, e.g. 127.0.0.1. This may change soon, if we
decide to export a GetHostIps() API which queries your NIC devices.
ProgramTrustedIp(ip:int[, cidr:int])
Trusts an IP address or network.
This function may be used to configure the IsTrustedIp() function
which is how redbean determines if a client is allowed to send us
headers like X-Forwarded-For (cf GetRemoteAddr vs. GetClientAddr)
without them being ignored. Trusted IPs is also how redbean turns
off token bucket rate limiting selectively, so be careful. Here's
an example of how you could trust all of Cloudflare's IPs:
ProgramTrustedIp(ParseIp("103.21.244.0"), 22);
ProgramTrustedIp(ParseIp("103.22.200.0"), 22);
ProgramTrustedIp(ParseIp("103.31.4.0"), 22);
ProgramTrustedIp(ParseIp("104.16.0.0"), 13);
ProgramTrustedIp(ParseIp("104.24.0.0"), 14);
ProgramTrustedIp(ParseIp("108.162.192.0"), 18);
ProgramTrustedIp(ParseIp("131.0.72.0"), 22);
ProgramTrustedIp(ParseIp("141.101.64.0"), 18);
ProgramTrustedIp(ParseIp("162.158.0.0"), 15);
ProgramTrustedIp(ParseIp("172.64.0.0"), 13);
ProgramTrustedIp(ParseIp("173.245.48.0"), 20);
ProgramTrustedIp(ParseIp("188.114.96.0"), 20);
ProgramTrustedIp(ParseIp("190.93.240.0"), 20);
ProgramTrustedIp(ParseIp("197.234.240.0"), 22);
ProgramTrustedIp(ParseIp("198.41.128.0"), 17);
Although you might want consider trusting redbean's open source
freedom embracing solution to DDOS protection instead!
ProgramTokenBucket([replenish:num, cidr:int, reject:int, ignore:int, ban:int])
Enables DDOS protection.
Imagine you have 2**32 buckets, one for each IP address. Each bucket
can hold about 127 tokens. Every second a background worker puts one
token in each bucket. When a TCP client socket is opened, it takes a
token from its bucket, and then proceeds. If the bucket holds only a
third of its original tokens, then redbean sends them a 429 warning.
If the client ignores this warning and keeps sending requests, until
there's no tokens left, then the banhammer finally comes down.
This model of network rate limiting generously lets people "burst" a
tiny bit. For example someone might get a strong craving for content
and smash the reload button in Chrome 64 times in a fow seconds. But
since the client only get 1 new token per second, they'd better cool
their heels for a few minutes after doing that. This amount of burst
can be altered by choosing the `reject` / `ignore` / `ban` threshold
arguments. For example, if the `reject` parameter is set to 126 then
no bursting is allowed, which probably isn't a good idea.
redbean is programmed to acquire a token immediately after accept()
is called from the main server process, which is well before fork()
or read() or any Lua code happens. redbean then takes action, based
on the token count, which can be accept / reject / ignore / ban. If
redbean determines a ban is warrented, then 4-byte datagram is sent
to the unix domain socket `/var/run/blackhole.sock` which should be
operated using the blackholed program we distribute separately.
The trick redbean uses on Linux for example is insert rules in your
raw prerouting table. redbean is very fast at the application layer
so the biggest issue we've encountered in production is are kernels
themselves, and programming the raw prerouting table dynamically is
how we solved that.
`replenish` is the number of times per second a token should be
added to each bucket. The default value is 1 which means one token
is granted per second to all buckets. The minimum value is 1/3600
which means once per hour. The maximum value for this setting is
1e6, which means once every microsecond.
`cidr` is the specificity of judgement. Since creating 2^32 buckets
would need 4GB of RAM, redbean defaults this value to 24 which means
filtering applies to class c network blocks (i.e. x.x.x.*), and your
token buckets only take up 2^24 bytes of RAM (16MB). This can be set
to any number on the inclusive interval [8,32], where having a lower
number means you use less ram/cpu, but splash damage applies more to
your clients; whereas higher numbers means more ram/cpu usage, while
ensuring rate limiting only applies to specific compromised actors.
`reject` is the token count or treshold at which redbean should send
429 Too Many Request warnings to the client. Permitted values can be
anywhere between -1 and 126 inclusively. The default value is 30 and
-1 means disable to disable (assuming AcquireToken() will be used).
`ignore` is the token count or treshold, at which redbean should try
simply ignoring clients and close the connection without logging any
kind of warning, and without sending any response. The default value
for this setting is `MIN(reject / 2, 15)`. This must be less than or
equal to the `reject` setting. Allowed values are [-1,126] where you
can use -1 as a means of disabling `ignore`.
`ban` is the token count at which redbean should report IP addresses
to the blackhole daemon via a unix-domain socket datagram so they'll
get banned in the kernel routing tables. redbean's default value for
this setting is `MIN(ignore / 10, 1)`. Permitted values are [-1,126]
where -1 may be used as a means of disabling the `ban` feature.
This function throws an exception if the constraints described above
are not the case. Warnings are logged should redbean fail to connect
to the blackhole daemon, assuming it hasn't been disabled. It's safe
to use load balancing tools when banning is enabled, since you can't
accidentally ban your own network interface addresses, loopback ips,
or ProgramTrustedIp() addresses where these rate limits don't apply.
It's assumed will be called from the .init.lua global scope although
it could be used in interpreter mode, or from a forked child process
in which case the only processes that'll have ability to use it will
be that same process, and any descendent processes. This function is
only able to be called once.
This feature is not available in unsecure mode.
AcquireToken([ip:uint32])
└─→ int8
Atomically acquires token.
This routine atomically acquires a single token for an `ip` address.
The return value is the token count before the subtraction happened.
No action is taken based on the count, since the caller will decide.
`ip` should be an IPv4 address and this defaults to GetClientAddr(),
although other interpretations of its meaning are possible.
Your token buckets are stored in shared memory so this can be called
from multiple forked processes. which operate on the same values.
CountTokens([ip:uint32])
└─→ int8
Counts number of tokens in bucket.
This function is the same as AcquireToken() except no subtraction is
performed, i.e. no token is taken.
`ip` should be an IPv4 address and this defaults to GetClientAddr(),
although other interpretations of its meaning are possible.
────────────────────────────────────────────────────────────────────────────────
CONSTANTS