Commit graph

611 commits

Author SHA1 Message Date
Justine Tunney
be7c5e1071 Fix debug mode build 2022-04-16 12:25:53 -07:00
Justine Tunney
dc0ea6640e Fix bugs with recent change
This change makes further effort towards improving our poll()
implementation on the New Technology. The stdin worker didn't work out
so well for Python so it's not being used for now. System call tracing
with the --strace flag should now be less noisy now on Windows unless
you modify the strace.internal.h defines to turn on some optional ones
that are most useful for debugging the system call wrappers.
2022-04-16 10:40:23 -07:00
Justine Tunney
933411ba99 Improve synchronization
- Fix bugs in kDos2Errno definition
- malloc() should now be thread safe
- Fix bug in rollup.com header generator
- Fix open(O_APPEND) on the New Technology
- Fix select() on the New Technology and test it
- Work towards refactoring i/o for thread safety
- Socket reads and writes on NT now poll for signals
- Work towards i/o completion ports on the New Technology
- Make read() and write() intermittently check for signals
- Blinkenlights keyboard i/o so much better on NT w/ poll()
- You can now poll() files and sockets at the same time on NT
- Fix bug in appendr() that manifests with dlmalloc footers off
2022-04-15 15:31:55 -07:00
Paul Kulchenko
233144b19d
Fix memory deallocation while yielding in redbean. (#384)
The yielded coroutine was removed from the stack too early,
leaving it not being anchored, which led to memory freed prematurely.
2022-04-14 18:13:53 -07:00
Justine Tunney
fb7e8ef1e6 Add more raw system calls to redbean
We now have execve, setitimer, sigaction, sigsuspend, and sigprocmask.
2022-04-13 14:43:42 -07:00
Justine Tunney
281a0f2730 Implement raw system call for redbean lua code
You can now call functions like fork() from Lua and it'll work across
all supported platforms, including Windows. This gives you a level of
control of the system that Lua traditionally hasn't been able to have
due to its focus on old portable stdio rather modern POSIX APIs. Demo
code has been added to redbean-demo.com to show how it works.

This change also modifies Lua so that integer literals with a leading
zero will be interpreted as octal. That should help avoid shooting in
the foot with POSIX APIs that frequently use octal mode bits.

This change fixes a bug in opendir(".") on New Technology.

Lastly, redbean will now serve crash reports to private network IPs.
This is consistent with other frameworks. However that isn't served
to public IPs unless the -E flag is passed to redbean at startup.
2022-04-13 08:53:24 -07:00
Justine Tunney
f684e348d4 Improve signals and memory protection
- Document sigaction()
- Simplify New Technology fork() code
- Testing and many bug fixes for mprotect()
- Distribute Intel Xed ILD in the amalgamation
- Turn Xed enums into defines to avoid DWARF bloat
- Improve polyfilling of SA_SIGINFO on BSDs and fix bugs
- setpgid(getpid(), getpid()) on Windows will ignore CTRL-C
- Work around issues relating to NT mappings being executable
- Permit automatic executable stack override via `ape_stack_pf`
2022-04-12 22:11:00 -07:00
Justine Tunney
c95c9d9508 Trim down MODE= linkage slightly 2022-04-12 08:05:22 -07:00
Justine Tunney
046c7ebd4a Improve locks and signals
- Introduce fast spinlock API
- Double rand64() perf w/ spinlock
- Improve raise() on New Technology
- Support gettid() across platforms
- Implement SA_NODEFER on New Technology
- Move the lock intrinsics into LIBC_INTRIN
- Make SIGTRAP recoverable on New Technology
- Block SIGCHLD in wait4() on New Technology
- Add threading prototypes for XNU and FreeBSD
- Rewrite abort() fixing its minor bugs on XNU/NT
- Shave down a lot of the content in libc/bits/bits.h
- Let signal handlers modify CPU registers on New Technology
2022-04-12 05:20:17 -07:00
Justine Tunney
f68f1789bd Add XNU magnums for preadv() and writev()
Apple introduced support sometime between 2018 and 2020.
2022-04-12 01:10:26 -07:00
Justine Tunney
e500e8fb48 Delete some junk constants
Some of these are from userspace APIs and therefore don't need to be in
consts.sh. Others are poorly supported, rarely used, niche hardware not
in scope for support any time soon. Ideally, we should only have listed
the stuff that's supported really well. This is a conservative deletion
2022-04-11 23:49:53 -07:00
Justine Tunney
a157940ba6 Delete most undocumented New Technology APIs
This change removes LIBC_KERNELBASE which is legacy code from the
initial import which was generated off a script that resolved the
delegated references, on Windows 10. All the important stuff here
should have long since been filed under kernel32.dll for windows7

Many FooA functions that were never assigned an arity are removed
because we almost never use the ASCII versions of WIN32 functions
therefore it's not worth having them slowing down the build. Some
other functions that overlap uncomfortably with libc are gone too

If something you need was removed, file an issue we'll restore it
2022-04-11 23:41:12 -07:00
Paul Kulchenko
183b3ed6a2
Update lsqlite3 to add update hook support for redbean (#383) 2022-04-09 12:36:44 -07:00
Paul Kulchenko
668dc42bac
Reset Lua stack to drop unused returned values (#382)
Since some Lua code may return values that will be left on stack, they
need to be removed to avoid growing the stack unnecessarily.
2022-04-09 12:36:05 -07:00
Justine Tunney
d31bebdd2d Add LISP interpreter 2022-04-07 20:30:22 -07:00
Justine Tunney
4f98ad1054 Improve system call support on NT
- Improve i/o perf on New Technology
- Code cleanup on read() for New Technology
- Fix bad bug with dup() of socket on New Technology
- Clean up some more strace errors on New Technology
2022-04-07 20:30:04 -07:00
Justine Tunney
29bf8b1a30 Make improvements
- Make rand64() thread safe
- Introduce lemur64 lcg prng
- Improve strace on New Technology
- Improve msync() on New Technology
2022-04-07 00:15:35 -07:00
Justine Tunney
43ba3009b2 Make another minor readme change 2022-04-06 13:07:51 -07:00
Justine Tunney
722d3df711 Delete old commented out Clang tests
See #343
2022-04-06 13:02:15 -07:00
Justine Tunney
837ff6cb49 Update readme to recommend nightlies 2022-04-06 12:55:14 -07:00
Justine Tunney
42fbcff149 Fix some more kinks in strace.com 2022-04-06 12:32:43 -07:00
Justine Tunney
c13142dad2 Fix redbean certificate free error
This fixes a regression from a change a few weeks ago in git commit
af645fcbec which caused certificates to
not be free()'d correctly if the certificates are chained. dlmalloc()
should have printed an error in most build modes. ASAN caught it too.
2022-04-06 11:05:34 -07:00
Justine Tunney
f59b5dbd7c Get strace.com working beautifully 2022-04-06 09:43:11 -07:00
Justine Tunney
82f2e758c7 Change the default command prompts 2022-04-06 06:56:11 -07:00
Justine Tunney
ce9bdbb0bf Delete some more unneeded Make code 2022-04-06 06:55:08 -07:00
Justine Tunney
58d74ede4c Flatten GNU Make directory structure 2022-04-06 00:22:54 -07:00
Justine Tunney
04d39d47f1 Fix basename() and dirname() 2022-04-06 00:13:44 -07:00
Justine Tunney
9b11206ae3 Improve alloca() safety even more 2022-04-01 22:50:15 -07:00
Justine Tunney
38cb6e71ca Improve alloca() memory safety
Now that all the bugs have been wormed out of the ASAN memory module we
can successfully check for underruns on large stack allocations.
2022-04-01 22:44:43 -07:00
Justine Tunney
1ff1854107 Fix rusage memory reporting on NT
This issue was caused by c23b6ecc3 a few days ago, where we started
polling that information for the first time.
2022-03-31 17:07:30 -07:00
Sami Samhuri
27bc49034c
Fix a couple typos (#361) 2022-03-29 22:54:09 -07:00
Sami Samhuri
1ccd269cdd
Fix comment pointing at vfork.S (#362)
Looks like this comment was slightly out of date due to some file renaming.
2022-03-29 22:53:36 -07:00
Paul Kulchenko
2fe386ff8c
Fix Redbean EncodeUrl params w/o values (#376) 2022-03-27 22:36:43 -07:00
Justine Tunney
072e1d2910 Make signal handling work well across platforms
- Fix sigsuspend() on XNU
- Fix strsignal() on non-Linux
- Add unit tests for strsignal()
- Add unit tests for setitimer()
- Add unit tests for sigsuspend()
- Rewrite setitimer() for New Technology
- Rewrite nanosleep() for New Technology
- Polyfill SIGALRM on the New Technology
- select(0,0,0,0) on NT now calls pause()
- Remove some NTDLL calls that aren't needed
- Polyfill SA_NOCLDWAIT on the New Technology
- Polyfill SA_RESETHAND on the New Technology
- Polyfill sigprocmask() on the New Technology
- Polyfill SIGCHLD+SIG_IGN on the New Technology
- Polyfill SA_RESTART masking on the New Technology
- Deliver console signals from main thread on New Technology
- Document SA_RESTART behavior w/ @sarestartable / @norestart
- System call trace in MODE=dbg now prints inherited FDs and signal mask
2022-03-25 07:28:57 -07:00
Justine Tunney
3b9e66ecba Add Linux ptrace() tutorial 2022-03-24 08:00:36 -07:00
Justine Tunney
98909b1391 Fix more things
- Update a couple unicode data files
- Disable strace during logger calls
- SQLite now uses pread() / pwrite()
- pread() past EOF on NT now returns 0
- Make the NT mmap() and fork() code elegant
- Give NT a big performance boost with memory
- Add many more mmap() tests to prove it works
2022-03-24 00:12:29 -07:00
Justine Tunney
b90fa996b4 Get life.com in MODE=tiny back down to 12kb 2022-03-23 08:09:01 -07:00
Justine Tunney
23b72eb617 Add support for symbol table in .com files
This change fixes minor bugs and adds a feature, which lets us store the
ELF symbol table, inside the ZIP directory. We use the path /zip/.symtab
which can be safely removed using a zip editing tool, to make the binary
smaller after compilation. This supplements the existing method of using
a separate .com.dbg file, which is still supported. The intent is people
don't always know that it's a good idea to download the debug file. It's
not great having someone's first experience be a crash report, that only
has numbers rather than symbols. This will help fix that!
2022-03-23 06:34:46 -07:00
Justine Tunney
393ca4be40 Polyfill SIGCHLD on the New Technology
redbean will now cleanup child processes properly. New accounting
information is available too, such as page faults and memory usage. The
way it works is Cosmopolitan Libc samples the process collection on
entry into read() and poll() to see if SIGCHLD needs to be raised.

This change also fixes an issue with chibicc /tmp cleanup. There was
also a regression in MODE=dbg because STL needed ASAN runtime support.
2022-03-22 21:31:12 -07:00
Justine Tunney
c23b6ecc31 Fix some win32 definitions
You can now use psapi.dll and pdh.dll. Some TODOs for Windows have been
cleared out. We might have a working load average for the platform that
should help GNU Make work well.
2022-03-22 19:54:36 -07:00
Justine Tunney
e2e0b042c1 Delete redundant libcxx files 2022-03-22 13:05:00 -07:00
Justine Tunney
868af3f950 Import C++ Standard Template Library
You can now use the hardest fastest and most dangerous language there is
with Cosmopolitan. So far about 75% of LLVM libcxx has been added. A few
breaking changes needed to be made to help this go smoothly.

- Rename nothrow to dontthrow
- Rename nodiscard to dontdiscard
- Add some libm functions, e.g. lgamma, nan, etc.
- Change intmax_t from int128 to int64 like everything else
- Introduce %jjd formatting directive for int128_t
- Introduce strtoi128(), strtou128(), etc.
- Rename bsrmax() to bsr128()

Some of the templates that should be working currently are std::vector,
std::string, std::map, std::set, std::deque, etc.
2022-03-22 06:41:54 -07:00
Justine Tunney
5022f9e920 Trim down redbean a little bit
This makes redbean.c a little less long. It also reduces the size of
redbean-original.com from being 333K to 213K.
2022-03-21 19:32:30 -07:00
Paul Kulchenko
87029ac3f9
Redbean stream yield implementation (#370)
* Simplify handling of coroutine state
* Update redbean to allow yielding from Lua to support streaming
* Add stack checks for Lua resume calls in redbean
2022-03-21 18:07:30 -07:00
Justine Tunney
e5314dedde Fix performance bottlenecks with nt fork redbean 2022-03-21 08:25:33 -07:00
Justine Tunney
d57f87dc40 Fix some issues with zipos and redbean
- redbean.com -D /zip/dir/ now works, for pure fun
- possibly fixed bug with redbean serving empty files
- zipos stat() mode now indicates directories on windows

See #372
2022-03-21 07:36:36 -07:00
Justine Tunney
4881ae7527 Update Argon2 for style
- Make sure notice licenses are embedded
- Remove copyright and docs from headers
2022-03-21 07:27:03 -07:00
Nick Owens
f78f2fcac3
redbean: allow symlinks for -D path (#372)
Previously ProgramDirectory called isdirectory which returns false for
symlinks. instead just check directory with stat, and allow user to pass
`-D /tmp/foo/`, where /tmp/foo might be a symlink.
2022-03-21 05:43:49 -07:00
Justine Tunney
5e8ae2d5bc Restart CI for New Technology and UBSAN hunting
Continuous Integration (via runit and runitd) is now re-enabled on win7
and win10. The `make test` command, which runs the tests on all systems
is now the fastest and most stable it's been since the project started.

UBSAN is now enabled in MODE=dbg in addition to ASAN. Many instances of
undefined behavior have been removed. Mostly things like passing a NULL
argument to memcpy(), which works fine with Cosmopolitan Libc, but that
doesn't prevents the compiler from being unhappy. There was an issue w/
GNU make where static analysis claims a sprintf() call can overflow. We
also now have nicer looking crash reports on Windows since uname should
now be supported and msys64 addr2line works reliably.
2022-03-21 04:32:57 -07:00
Paul Kulchenko
d5ff2c3fb9
Make improvements to redbean (#373)
* Allow ProgramUniprocess to be set on Windows
* Add closing p tag to Maxmind example
* Enable using symlinked directories with redbean -A flag
* Rename gc in redbean for consistency
* Improve StoreFile by adding checks and skipping ./ in stored path
* Add freeing memory in redbean LuaEncodeUrl

Ref #366
2022-03-21 03:20:09 -07:00