Commit graph

135 commits

Author SHA1 Message Date
Gavin Hayes
1ef955c33b
Improve zipos path handling (#579)
This change adds opendir, readdir, and stat support for the /zip/ root,
as well as directories not explicitly encoded in the zip file.
2022-09-03 22:50:23 -07:00
Justine Tunney
5a632cf72d Add lchown, lchmod, statvfs, fstatvfs 2022-08-22 20:50:16 -07:00
Justine Tunney
99a92048b4 Get awk to build and make it hackable 2022-08-21 13:39:20 -07:00
Justine Tunney
73845be1f0 Restore zip.com and .symtab files
This change restores the .symtab symbol table files in our flagship
programs (e.g. redbean.com, python.com) needed to show backtraces. This
also rolls back earlier changes to zip.com w.r.t. temp directories since
the right way to do it turned out to be the -b DIR flag.

This change also improves the performance of zip.com. It turned out
mmap() wasn't being used, because zip.com was assuming a 4096-byte
granularity, but cosmo requires 65536. There was also a chance to speed
up stdio scanning using the unlocked functions.
2022-08-21 00:17:20 -07:00
Justine Tunney
83d41e4588 Clean up some code 2022-08-20 12:32:51 -07:00
Justine Tunney
d76dfadc7a Reduce GNU Make latency 17% for cosmo
fgets() is now 4x faster which makes Make 2% faster. Landlock Make now
has a builtin $(uniq ...) function that uses critbit trees rather than
functional programming. Since uniq is the most important function this
optimization makes our cold start latency 15% faster.
2022-08-19 15:29:24 -07:00
Gavin Hayes
897e33ccc4
Fix stdio fmt of "%.0e" and "%.0g" (#544)
* Fix %.0e by always rounding even if precision is 0
* Fix %.0g by treating it the same as %.1g
* Fix %g tests to match glibc, add more tests
2022-08-18 15:53:07 -07:00
Justine Tunney
c2211c9e63 Polyfill statfs() and fstatfs() on Windows 2022-08-17 19:01:51 -07:00
Justine Tunney
6c0bbfac4a Fix some bugs 2022-08-14 13:28:07 -07:00
Justine Tunney
0ea0d33a77 Reduce build graph by another eight percent 2022-08-13 13:11:56 -07:00
Justine Tunney
367d06d9e4 Fold LIBC_UNICODE into LIBC_STR 2022-08-13 08:42:32 -07:00
Justine Tunney
17aea99bb3 Fold LIBC_ALG into LIBC_MEM 2022-08-13 08:32:34 -07:00
Justine Tunney
8a0a2c0c36 Fold LIBC_RAND into LIBC_STDIO/TINYMATH/INTRIN 2022-08-11 12:32:00 -07:00
Justine Tunney
05b8f82371 Fold LIBC_BITS into LIBC_INTRIN 2022-08-11 12:13:18 -07:00
Justine Tunney
10fd8bdb70 Unbloat the build
This change resurrects ae5d06dc53
2022-08-11 00:15:29 -07:00
Justine Tunney
7b993d561c Use private keyword on makefile target variables 2022-08-10 18:36:11 -07:00
Justine Tunney
7e1c78b4f3 Cherry-pick bug fixes and Landlock Make 1.0.2 2022-08-10 12:56:45 -07:00
Justine Tunney
c1d99676c4 Revert "Unbloat build config"
This reverts commit ae5d06dc53.
2022-08-10 12:44:56 -07:00
Justine Tunney
ae5d06dc53 Unbloat build config
- 10.5% reduction of o//depend dependency graph
- 8.8% reduction in latency of make command
- Fix issue with temporary file cleanup

There's a new -w option in compile.com that turns off the recent
Landlock output path workaround for "good commands" which do not
unlink() the output file like GNU tooling does.

Our new GNU Make unveil sandboxing appears to have zero overhead
in the grand scheme of things. Full builds are pretty fast since
the only thing that's actually slowed us down is probably libcxx

    make -j16 MODE=rel
    RL: took 85,732,063µs wall time
    RL: ballooned to 323,612kb in size
    RL: needed 828,560,521µs cpu (11% kernel)
    RL: caused 39,080,670 page faults (99% memcpy)
    RL: 350,073 context switches (72% consensual)
    RL: performed 0 reads and 11,494,960 write i/o operations

pledge() and unveil() no longer consider ENOSYS to be an error.
These functions have also been added to Python's cosmo module.

This change also removes some WIN32 APIs and System Five magnums
which we're not using and it's doubtful anyone else would be too
2022-08-10 04:43:09 -07:00
Justine Tunney
13c1c45075 Make some last minute improvements to make.com 2022-08-07 05:59:53 -07:00
Justine Tunney
6a5717a48f Make more libc improvements
- Make memmem() faster
- Make readdir() thread safe
- Remove 64kb limit from mkdeps.com
- Add old crypt() function from Musl
- Improve new fix-third-party.py tool
- Improve libc/isystem/ headers and fix bugs
2022-08-06 17:18:40 -07:00
Justine Tunney
a8cf0f7e89 Add more missing libc functionality 2022-08-06 10:50:51 -07:00
Justine Tunney
5a2bb07b36 Add ctermid and L_ctermid 2022-07-22 04:39:04 -07:00
Justine Tunney
84caee23ba Make sorted serialization faster
Redbean Lua and JSON serialization now goes faster because we're now
inserting object entries into tree data structure rather than making
an array and sorting it at the end. For example, when serializing an
object with 10,000 entries this goes twice as fast. However it still
goes slower than saying EncodeJson(x, {sorted=false}).
2022-07-22 04:19:01 -07:00
Jared Miller
d4000bb8f7
Correct more typos (#500) 2022-07-21 20:53:30 -07:00
Justine Tunney
3f015b1e51 Make some minor fixups to bug reporting, etc. 2022-07-11 05:58:24 -07:00
Justine Tunney
ee82cee432 Further improve json / lua serialization 2022-07-09 16:27:26 -07:00
Justine Tunney
c9e68b0ebc Make redbean serialization deterministic 2022-07-09 04:09:51 -07:00
Justine Tunney
1c83670229 Write more redbean unit tests
- Fix DescribeSigset()
- Introduce new unix.rmrf() API
- Fix redbean sigaction() doc example code
- Fix unix.sigaction() w/ more than two args
- Improve redbean re module API (non-breaking)
- Enhance Lua with Python string multiplication
- Make third parameter of unix.socket() default to 0
2022-07-08 23:10:02 -07:00
Justine Tunney
853b6c3864 Improve system calls
- Wrap clock_getres()
- Wrap sched_setscheduler()
- Make sleep() api conformant
- Polyfill sleep() using select()
- Improve clock_gettime() polyfill
- Make nanosleep() POSIX conformant
- Slightly improve some DNS functions
- Further strengthen pledge() sandboxing
- Improve rounding of timeval / timespec
- Allow layering of pledge() calls on Linux
- Polyfill sched_yield() using select() on XNU
- Delete more system constants we probably don't need
2022-07-08 06:42:03 -07:00
Justine Tunney
f6dab99f44 Release redbean 2.0.8 2022-06-26 17:25:54 -07:00
Justine Tunney
61257d48d4 Make some quick fixes and cleanup 2022-06-26 02:58:36 -07:00
Justine Tunney
d5312b60f7 Make improvements to locking
This change makes pthread_mutex_lock() as fast as _spinlock() by
default. Thread instability issues on NetBSD have been resolved.
Improvements made to gdtoa thread code. Crash reporting will now
synchronize between threads in a slightly better way.
2022-06-19 01:30:12 -07:00
Justine Tunney
1c387727fe Add long double printf formatting support
You can now easily print numbers that have 80 bits of precision. This
change also imports many of the test vectors from the gdtoa codebase.
2022-06-14 02:18:03 -07:00
Justine Tunney
a3865ecc3c Make more fixes and improvements
- Fix Makefile flaking due to ZIPOBJ_FLAGS generation
- Make printf() floating point and gdtoa thread safe
- Polish up the runit / runitd programs some more
- Prune some more makefile dependencies
2022-06-13 11:02:13 -07:00
Justine Tunney
8b72490431 Make mutex calling code 10x tinier
Calls to lock/unlock functions are now NOPs by default. The first time
clone() is called, they get turned into CALL instructions. Doing this
caused funcctions like fputc() to shrink from 85 bytes to 45+4 bytes.
Since the ANSI solution of `(__threaded && lock())` inlines os much
superfluous binary content into functions all over the place.
2022-06-12 20:17:12 -07:00
Justine Tunney
8cdec62f5b Apply even more fixups
- Finish cleaning up the stdio unlocked APIs
- Make __cxa_finalize() properly thread safe
- Don't log locks if threads aren't being used
- Add some more mutex guards to places using _mmi
- Specific lock names now appear in the --ftrace logs
- Fix mkdeps.com generating invalid Makefiles sometimes
- Simplify and fix bugs in the test runner infrastructure
- Fix issue where sometimes some functions wouldn't be logged
2022-06-12 11:57:00 -07:00
Justine Tunney
c260345e06 Make locks more reliable
This change switches most of the core locks to be re-entrant, in order
to reduce the chance of deadlocking code that does, clever things with
asynchronous signal handlers. This change implements it it in pthreads
so we're one step closer to having a standardized threading primitives
2022-06-11 02:07:20 -07:00
Justine Tunney
adac64a52b Reduce makefile dependencies by 10%
The includes in libc/calls/calls.h have now been refactored so that
functions with struct parameters are declared in libc/calls/struct/
2022-06-08 20:01:28 -07:00
Justine Tunney
425ff5dff0 Refactor some code
- Write tests for cthreads
- Fix bugs in pe2.com tool
- Fix ASAN issue with GetDosEnviron()
- Consolidate the cthread header files
- Some code size optimizations for MODE=
- Attempted to squash a tls linker warning
- Attempted to get futexes working on FreeBSD
2022-05-28 14:09:21 -07:00
Justine Tunney
de5de19004 Make improvements
- Document redbean's argon2 module
- Fix regressions in cthreads library
- Make testlib work better with threads
- Give the cthreads library lots of love
- Remove some of the stdio assembly code
- Implement getloadavg() across platforms
- Code size optimizations for errnos, etc.
- Only check for signals in main thread on Windows
- Make errnos for dup2 / dup3 consistent with posix

This change also fixes a bug in the argon2 module, where the NUL
terminator was being included in the hash encoded ascii string. This
shouldn't require any database migrations to folks who found this module
and productionized it, since the argon2 library treats it as a c string.
2022-05-28 00:28:09 -07:00
Justine Tunney
99e67c348b Reduce Makefile dependencies by 4% 2022-05-23 15:07:01 -07:00
Justine Tunney
312ed5c67c Fix some issues and do some code cleanup 2022-05-23 10:15:53 -07:00
Justine Tunney
1f229e4efc Use re-entrant locks on stdio 2022-05-22 08:28:33 -07:00
Justine Tunney
ec2cb88058 Make fixes and improvements
- Document more compiler flags
- Expose new __print_maps() api
- Better overflow checking in mmap()
- Improve the shell example somewhat
- Fix minor runtime bugs regarding stacks
- Make kill() on fork()+execve()'d children work
- Support CLONE_CHILD_CLEARTID for proper joining
- Fix recent possible deadlock regression with --ftrace
2022-05-19 16:57:49 -07:00
Justine Tunney
6e52cba37a Fix stdio regression
This change fixes a nasty regression caused by
80b211e314 which deadlocked.

This change also causes MbedTLS to prefer the ChaCha ciphersuite on
older CPUs that don't have AES hardware instructions.
2022-05-19 00:51:15 -07:00
Justine Tunney
ce71677156 Improve threading support further 2022-05-17 04:14:28 -07:00
Justine Tunney
80b211e314 Add raw memory visualization tool to redbean
This change introduces a `-W /dev/pts/1` flag to redbean. What it does
is use the mincore() system call to create a dual-screen terminal
display that lets you troubleshoot the virtual address space. This is
useful since page faults are an important thing to consider when using a
forking web server. Now we have a colorful visualization of which pages
are going to fault and which ones are resident in memory.

The memory monitor, if enabled, spawns as a thread that just outputs
ANSI codes to the second terminal in a loop. In order to make this
happen using the new clone() polyfill, stdio is now thread safe.

This change also introduces some new demo pages to redbean. It also
polishes the demos we already have, to look a bit nicer and more
presentable for the upcoming release, with better explanations too.
2022-05-14 04:33:58 -07:00
Justine Tunney
578cb21591 Add syscalls to Blinkenlights and fix bugs 2022-05-13 13:31:21 -07:00
Justine Tunney
47b3274665 Make improvements
- Add rusage to redbean Lua API
- Add more redbean documentation
- Add pledge() to redbean Lua API
- Polyfill OpenBSD pledge() for Linux
- Increase PATH_MAX limit to 1024 characters
- Untrack sibling processes after fork() on Windows
2022-04-28 09:57:07 -07:00