Commit graph

416 commits

Author SHA1 Message Date
Justine Tunney
950a1b310b
Embed cocmd.com interpreter for system() / open()
This change lets you use system() in an easier and portable way. The
problem with the call in the past has always been that bourne and
cmd.com on Windows have less than nothing in common, so pretty much the
only command system() could be used for across platforms was maybe echo.
cmd.exe is also a security liability due to its escaping rules.

Since cocmd.com implements 85% of what we need from bourne, in a really
tiny way, it makes perfect sense to be embedded in these functionss. We
get a huge performance boost too.

Fixes #644
2022-10-02 15:29:57 -07:00
Justine Tunney
daca5499b9
Add some tests for execve() 2022-10-02 14:58:14 -07:00
Justine Tunney
3f3cb0650b
Have execve() escape double quotes in cmd.exe's preferred style
This makes it possible for us to use system() and popen() with paths
that redirect to filenames that contain spaces, e.g.

    system("echo.com hello >\"hello there.txt\"")

It's difficult to solve this problem, because WIN32 only allows passing
one single argument when launching programs and each program is allowed
to tokenize that however it wants. Most software follows the convention
of cmd.exe which is poorly documented and positively byzantine.

In the future we're going to solve this by not using cmd.exe at all and
instead embedding the cocmd.com interpreter into the system() function.
In the meantime, our documentation has been updated to help recalibrate
any expectation the user might hold regarding the security of using the
Windows command interpreter.

Fixes #644
2022-10-02 09:15:46 -07:00
Justine Tunney
acd8900071
Add fexecve() and map O_EXEC to O_PATH on Linux 2022-10-02 09:15:46 -07:00
Justine Tunney
27b5deefb1
Introduce testlib_extract() helper 2022-10-02 09:15:46 -07:00
Justine Tunney
5005f2e446
Rewrite brk() and sbrk()
This change reduces the .bss memory requirement for all executables by
O(64kb). The brk system calls are now fully tested and figured out and
might be useful for tiny programs that only target System Five.
2022-10-01 23:11:56 -07:00
Gavin Hayes
87708c5d6e
Change accept type to struct sockaddr * (#630) 2022-09-20 07:49:16 -07:00
Justine Tunney
6e582d245b
Improve some timespec functions 2022-09-19 16:13:37 -07:00
Justine Tunney
2395a9eced
Work around copy_file_range() GitHub Actions issue 2022-09-19 16:12:56 -07:00
Justine Tunney
1ea01fc905
Fix pledge() thread killing semantics
- tkill(tid, sig) should be allowed by stdio
- tgkill(getpid(), tid, sig) should be allowed by stdio

Fixes #628
2022-09-19 15:34:11 -07:00
Justine Tunney
c7a8cd21e9
Improve system call wrappers
This change improves copy_file_range(), sendfile(), splice(), openpty(),
closefrom(), close_range(), fadvise() and posix_fadvise() in addition to
writing tests that confirm things like errno and seeking behavior across
platforms. We now less aggressively polyfill behavior with some of these
functions when the platform support isn't available. Please see:

https://justine.lol/cosmopolitan/functions.html
2022-09-19 15:06:25 -07:00
Justine Tunney
be29b709b7
Fix pledge.com interop with Go DNS
Based on a pure analysis of the Go source code, it wants:

- SOL_SOCKET + SO_BROADCAST
- IPPROTO_IPV6 + IPV6_V6ONLY

Fixes #627
2022-09-18 03:56:52 -07:00
Justine Tunney
40991ec992
Fix getgroups / setgroups tests across platforms
See #619
2022-09-18 03:18:38 -07:00
Gavin Hayes
4c40c500b8
Add getgroups and setgroups (#619) 2022-09-18 02:48:53 -07:00
Justine Tunney
994e1f4386
Improve pthread_join()
Since we're now on Windows 8, we can have clone() work as advertised on
Windows, where it sends a futex wake to the child tid. It's also likely
we no longer need to work around thread flakes on OpenBSD, in _wait0().
2022-09-16 14:02:06 -07:00
Justine Tunney
134ffee519
Change support vector to Windows 8+
Doing this makes binaries tinier, since we don't need to have all the
extra code for supporting a 32-bit address space. It also benefits us
because we're able to use WIN32 futexes, which makes locking simpler.

b69f3d2488 is what officially ended our
Windows 7 support. This change is merely a formalization. You can use
old versions of Cosmo now and forevermore if you need Windows 7 since
our repository is hermetic and vendors all its dependencies.

Won't fix #617
2022-09-15 03:55:05 -07:00
Justine Tunney
3f49889841
Make important improvements
- Fix preadv() and pwritev() for old distros
- Introduce _npassert() and _unassert() macros
- Prove that file locks work properly on Windows
- Support fcntl(F_DUPFD_CLOEXEC) on more systems
2022-09-14 22:39:08 -07:00
Justine Tunney
1ad2f530f9
Add script.com and whois.com courtesy of FreeBSD 2022-09-13 20:26:13 -07:00
Justine Tunney
654ceaba7d
Clean up threading code some more 2022-09-13 20:17:34 -07:00
Justine Tunney
aab4ee4072
Add sys_ prefix to unwrapped system calls
This change also implements getlogin() and getlogin_r().
2022-09-13 11:20:35 -07:00
Justine Tunney
8f5678882d
Make dos errno multimapping linkage tinier
640 bytes for old kDos2Errno table
182 bytes for new kDos2Errno under hello2.com (MODE=fastbuild)
122 bytes for new kDos2Errno under hello2.com (MODE=tiny)
2022-09-13 06:25:50 -07:00
Justine Tunney
06f9a5b627
Get repository to build with GCC 11
See #594
2022-09-13 04:14:55 -07:00
Justine Tunney
22b63d0b98
Add *NSYNC to libc/isystem 2022-09-13 01:47:12 -07:00
Justine Tunney
6f7d0cb1c3
Pay off more technical debt
This makes breaking changes to add underscores to many non-standard
function names provided by the c library. MODE=tiny is now tinier and
we now use smaller locks that are better for tiny apps in this mode.
Some headers have been renamed to be in the same folder as the build
package, so it'll be easier to know which build dependency is needed.
Certain old misguided interfaces have been removed. Intel intrinsics
headers are now listed in libc/isystem (but not in the amalgamation)
to help further improve open source compatibility. Header complexity
has also been reduced. Lastly, more shell scripts are now available.
2022-09-12 23:36:56 -07:00
Justine Tunney
b69f3d2488
Optimize memory layout 2022-09-12 04:26:52 -07:00
Justine Tunney
b5cb71ab84
Use *NSYNC for POSIX threads locking APIs
Condition variables, barriers, and r/w locks now work very well.
2022-09-11 11:04:50 -07:00
Justine Tunney
cdb2284f0d
Remove stdio lock macros from amalgamation 2022-09-10 12:03:36 -07:00
Justine Tunney
333768440c
Clean up the TLS code 2022-09-10 11:49:13 -07:00
Justine Tunney
cfcf5918bc
Rewrite recursive mutex code 2022-09-10 09:18:52 -07:00
Justine Tunney
155b378a39
Tidy up the threading implementation
The organization of the source files is now much more rational.
Old experiments that didn't work out are now deleted. Naming of
things like files is now more intuitive.
2022-09-10 02:56:25 -07:00
Justine Tunney
e9272f03fb
Make some minor touchups for nightlies 2022-09-09 19:12:40 -07:00
Justine Tunney
3265324e00
Don't relocate file descriptor memory
This change fixes #496 where ASAN spotted a race condition that could
happen in multithreaded programs, with more than OPEN_MAX descriptors
when using ZipOS or Windows NT, which require tracking open file info
and this change fixes that table so it never relocates, thus allowing
us to continue to enjoy the benefits of avoiding locks while reading.
2022-09-09 16:54:28 -07:00
Justine Tunney
c3208eb9d5
Add more POSIX threads APIs 2022-09-09 11:30:33 -07:00
Justine Tunney
e97f1a99cf
Fix pthread stacks with larger guard size 2022-09-09 06:41:22 -07:00
Justine Tunney
2d17ab016c
Perform more low-level code cleanup 2022-09-09 04:07:08 -07:00
Justine Tunney
9f963dc597
Clean up some of the threading code 2022-09-08 12:31:56 -07:00
Justine Tunney
0547eabcd6
Polyfill IPv6 on non-Linux 2022-09-08 06:06:22 -07:00
Justine Tunney
b73e35c6fa
Improve open source compatibility
This change tunes the default stack size for the outside world to 8mb
while at the same time, reducing Cosmopolitan's default stack size to
64kb. You can override the stack size using STATIC_STACK_SIZE(). Your
build scripts should point to o//ape/public/ape.lds

This change also fixes the definition of SOMAXCONN and removes AF_RDS
since it's not polyfilled and Python 3.11 complained.
2022-09-08 03:19:35 -07:00
Justine Tunney
0e2b1bfeed
Make garbage collection thread safe
- You can now use _gc(malloc()) in multithreaded programs
- This change fixes a bug where fork() on NT disabled TLS
- Fixed TLS code morphing on XNU/NT, for R8-R15 registers
2022-09-08 02:33:01 -07:00
Justine Tunney
571c2c3c69
Fix race condition in makedirs() 2022-09-07 23:12:44 -07:00
Gavin Hayes
b7c07d548c
Add FP_ILOGB constants (#599) 2022-09-07 21:37:53 -07:00
Justine Tunney
6c323383e5
Make some fixups to POSIX threads 2022-09-07 21:13:50 -07:00
Justine Tunney
de511bc71a
Add phtread_setname_np() and pthread_getname_np() 2022-09-07 19:04:05 -07:00
Gavin Hayes
a849a63771
Implement sigpending for sysv and nt (#597) 2022-09-07 05:38:12 -07:00
Justine Tunney
4339d9f15e
Add pthread attributes and other libc functions 2022-09-07 05:28:32 -07:00
Justine Tunney
8bd9ad8342
Simplify redbean serialization code 2022-09-06 20:07:29 -07:00
Gavin Hayes
598640864a
Fix inability to add some signals to mask on NT (#596) 2022-09-06 18:35:26 -07:00
Justine Tunney
d861d2787b
Import OpenBSD sorting algorithms
OpenBSD's qsort() function is more secure than the ones used by
FreeBSD, NetBSD and MacOS. The best part is it goes faster too!
This change also imports the OpenBSD mergesort() and heapsort()
2022-09-06 17:52:26 -07:00
Justine Tunney
11ec60d5fc
Use a better sorting algorithm
This change changes qsort() to use the same code as NetBSD and MacOS
because it goes 6x faster than Musl's SmoothSort function. Smoothsort
can still be used if you need something that's provenly linearithmic.
This change also improves GNU Make performance on whole by 7 percent!

netbsd nearly   l:    70,196c    22,673ns   m:    68,428c    22,102ns
musl nearly     l:    53,844c    17,391ns   m:    58,726c    18,968ns
unixv6 nearly   l:    65,885c    21,280ns   m:    63,082c    20,375ns

netbsd reverse  l:   120,290c    38,853ns   m:   122,619c    39,605ns
musl reverse    l:   801,826c   258,985ns   m:   794,689c   256,680ns
unixv6 reverse  l:    58,977c    19,049ns   m:    59,764c    19,303ns

netbsd random   l:   146,745c    47,398ns   m:   145,782c    47,087ns
musl random     l:   855,804c   276,420ns   m:   850,912c   274,840ns
unixv6 random   l:   214,325c    69,226ns   m:   213,906c    69,090ns

netbsd 2n       l:    77,299c    24,967ns   m:    76,773c    24,797ns
musl 2n         l:   818,012c   264,213ns   m:   818,282c   264,301ns
unixv6 2n       l: 3,967,009c 1,281,322ns   m: 3,941,792c 1,273,177ns

https://justine.lol/dox/sort.pdf
2022-09-06 11:06:10 -07:00
Justine Tunney
55c6297e13
Make more compatibility improvements 2022-09-06 07:04:13 -07:00