Commit graph

743 commits

Author SHA1 Message Date
tkchia
bc8532688b
Enable CPU exception handling w/ IDT & TSS (#640) 2022-09-29 14:43:08 -07:00
Justine Tunney
b46f763ee7
Permit multiple certs with same common name
This is needed to support Cloudflare interop.
2022-09-27 18:17:04 -07:00
Gavin Hayes
87708c5d6e
Change accept type to struct sockaddr * (#630) 2022-09-20 07:49:16 -07:00
Justine Tunney
6cc9e08f1b
Introduce cosmocc and cosmoc++ toolchain commands 2022-09-20 04:34:03 -07:00
Justine Tunney
bfb85fe6d0
Remove a bunch of stuff that shouldn't be in docs 2022-09-20 02:57:46 -07:00
Justine Tunney
6e582d245b
Improve some timespec functions 2022-09-19 16:13:37 -07:00
Justine Tunney
1ea01fc905
Fix pledge() thread killing semantics
- tkill(tid, sig) should be allowed by stdio
- tgkill(getpid(), tid, sig) should be allowed by stdio

Fixes #628
2022-09-19 15:34:11 -07:00
Justine Tunney
c7a8cd21e9
Improve system call wrappers
This change improves copy_file_range(), sendfile(), splice(), openpty(),
closefrom(), close_range(), fadvise() and posix_fadvise() in addition to
writing tests that confirm things like errno and seeking behavior across
platforms. We now less aggressively polyfill behavior with some of these
functions when the platform support isn't available. Please see:

https://justine.lol/cosmopolitan/functions.html
2022-09-19 15:06:25 -07:00
Justine Tunney
224c12f54d
Make zlib go faster 2022-09-18 06:30:45 -07:00
Justine Tunney
be29b709b7
Fix pledge.com interop with Go DNS
Based on a pure analysis of the Go source code, it wants:

- SOL_SOCKET + SO_BROADCAST
- IPPROTO_IPV6 + IPV6_V6ONLY

Fixes #627
2022-09-18 03:56:52 -07:00
Justine Tunney
40991ec992
Fix getgroups / setgroups tests across platforms
See #619
2022-09-18 03:18:38 -07:00
tkchia
c937fbb222
[metal] Export struct offsets from C code (#625)
Rather than computing them by hand.
2022-09-18 02:54:55 -07:00
tkchia
cdf9e8c8a3
Bare metal VGA: fix mapping of ECMA-48 colors to VGA colors (#622)
See https://github.com/jart/cosmopolitan/pull/613#issuecomment-1247490693
2022-09-18 02:50:00 -07:00
Gavin Hayes
4c40c500b8
Add getgroups and setgroups (#619) 2022-09-18 02:48:53 -07:00
Justine Tunney
775944a2d0
Fix important bugs in redbean
This change upgrades to the latest Chromium Zlib, fixes bugs in redbean,
and introduces better support for reverse proxies like Cloudflare. This
change improves the security of redbean and it's recommended that users
upgrade to the release that'll follow. This change also updates the docs
to clarify how to use the security tools redbean provides e.g. pledge(),
unveil(), and the MODE=asan builds which improve memory safety.
2022-09-17 02:18:06 -07:00
Justine Tunney
994e1f4386
Improve pthread_join()
Since we're now on Windows 8, we can have clone() work as advertised on
Windows, where it sends a futex wake to the child tid. It's also likely
we no longer need to work around thread flakes on OpenBSD, in _wait0().
2022-09-16 14:02:06 -07:00
tkchia
3733b43a8f
Bare metal: fix corner case in __normalize_e820( ) (#624)
See https://github.com/jart/cosmopolitan/pull/609#issuecomment-1247083243
2022-09-15 21:09:29 -07:00
Justine Tunney
134ffee519
Change support vector to Windows 8+
Doing this makes binaries tinier, since we don't need to have all the
extra code for supporting a 32-bit address space. It also benefits us
because we're able to use WIN32 futexes, which makes locking simpler.

b69f3d2488 is what officially ended our
Windows 7 support. This change is merely a formalization. You can use
old versions of Cosmo now and forevermore if you need Windows 7 since
our repository is hermetic and vendors all its dependencies.

Won't fix #617
2022-09-15 03:55:05 -07:00
Justine Tunney
3f49889841
Make important improvements
- Fix preadv() and pwritev() for old distros
- Introduce _npassert() and _unassert() macros
- Prove that file locks work properly on Windows
- Support fcntl(F_DUPFD_CLOEXEC) on more systems
2022-09-14 22:39:08 -07:00
Justine Tunney
1ad2f530f9
Add script.com and whois.com courtesy of FreeBSD 2022-09-13 20:26:13 -07:00
Justine Tunney
654ceaba7d
Clean up threading code some more 2022-09-13 20:17:34 -07:00
tkchia
6a3330d7c9
VGA tty initialization should not run outside of bare metal mode (#614) 2022-09-13 15:27:19 -07:00
tkchia
945699694e
Bare metal VGA: clarify code comments (#616) 2022-09-13 15:26:33 -07:00
Justine Tunney
aab4ee4072
Add sys_ prefix to unwrapped system calls
This change also implements getlogin() and getlogin_r().
2022-09-13 11:20:35 -07:00
Justine Tunney
8f5678882d
Make dos errno multimapping linkage tinier
640 bytes for old kDos2Errno table
182 bytes for new kDos2Errno under hello2.com (MODE=fastbuild)
122 bytes for new kDos2Errno under hello2.com (MODE=tiny)
2022-09-13 06:25:50 -07:00
Justine Tunney
9fa6725067
Fix bug with sendfile() on XNU 2022-09-13 04:32:29 -07:00
Justine Tunney
06f9a5b627
Get repository to build with GCC 11
See #594
2022-09-13 04:14:55 -07:00
Justine Tunney
446a1f7c5a
Fix the build 2022-09-13 02:28:07 -07:00
tkchia
df08b541af
Bare metal VGA: implement "status report" escape codes (#613)
* Bare metal VGA: implement "status report" escape codes
* Minor fix to pseudoteletypewriter code

Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-09-13 02:14:10 -07:00
tkchia
e0fabd1d49
[WIP] Get bare metal working outside of an emulator (#609)
You can now run bare metal on bare metal!

* Fix handling of int 0x15 eax = 0xe820 memory map
* Fix some issues in initial page table creation
* hello4.com now works outside emulators
* Ensure area for identity page tables are zeroed first
* Simplify logic for creating page table entries, this partly
  reverts 577c0f6226
* Add degenerate MBR partition entry, to ease testing

Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-09-13 02:01:49 -07:00
Justine Tunney
22b63d0b98
Add *NSYNC to libc/isystem 2022-09-13 01:47:12 -07:00
Justine Tunney
6f7d0cb1c3
Pay off more technical debt
This makes breaking changes to add underscores to many non-standard
function names provided by the c library. MODE=tiny is now tinier and
we now use smaller locks that are better for tiny apps in this mode.
Some headers have been renamed to be in the same folder as the build
package, so it'll be easier to know which build dependency is needed.
Certain old misguided interfaces have been removed. Intel intrinsics
headers are now listed in libc/isystem (but not in the amalgamation)
to help further improve open source compatibility. Header complexity
has also been reduced. Lastly, more shell scripts are now available.
2022-09-12 23:36:56 -07:00
Justine Tunney
b69f3d2488
Optimize memory layout 2022-09-12 04:26:52 -07:00
Justine Tunney
9b89a7c51d
Add *NSYNC mu_test 2022-09-11 18:56:29 -07:00
Justine Tunney
b5cb71ab84
Use *NSYNC for POSIX threads locking APIs
Condition variables, barriers, and r/w locks now work very well.
2022-09-11 11:04:50 -07:00
Justine Tunney
3de35e196c
Fix the build 2022-09-10 19:57:50 -07:00
Justine Tunney
5dff63a31c
Import nsync 2022-09-10 16:12:31 -07:00
Justine Tunney
cdb2284f0d
Remove stdio lock macros from amalgamation 2022-09-10 12:03:36 -07:00
Justine Tunney
333768440c
Clean up the TLS code 2022-09-10 11:49:13 -07:00
Justine Tunney
cfcf5918bc
Rewrite recursive mutex code 2022-09-10 09:18:52 -07:00
tkchia
bae7367774
Bare metal VGA: minor improvements (#608) 2022-09-10 04:28:28 -07:00
Justine Tunney
155b378a39
Tidy up the threading implementation
The organization of the source files is now much more rational.
Old experiments that didn't work out are now deleted. Naming of
things like files is now more intuitive.
2022-09-10 02:56:25 -07:00
Justine Tunney
e9272f03fb
Make some minor touchups for nightlies 2022-09-09 19:12:40 -07:00
Justine Tunney
3265324e00
Don't relocate file descriptor memory
This change fixes #496 where ASAN spotted a race condition that could
happen in multithreaded programs, with more than OPEN_MAX descriptors
when using ZipOS or Windows NT, which require tracking open file info
and this change fixes that table so it never relocates, thus allowing
us to continue to enjoy the benefits of avoiding locks while reading.
2022-09-09 16:54:28 -07:00
Justine Tunney
c3208eb9d5
Add more POSIX threads APIs 2022-09-09 11:30:33 -07:00
Justine Tunney
1729a8259c
Introduce MODE=fastbuild for 28% speedup 2022-09-09 08:59:59 -07:00
Gautham
094ccbb4bd
Add more foo64 stubs for Rust (#607) 2022-09-09 06:50:45 -07:00
Justine Tunney
e97f1a99cf
Fix pthread stacks with larger guard size 2022-09-09 06:41:22 -07:00
tkchia
1db76c288e
Bare metal VGA: enable bright background colors, disable blinking (#603)
Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-09-09 04:35:16 -07:00
Justine Tunney
2d17ab016c
Perform more low-level code cleanup 2022-09-09 04:07:08 -07:00
Gautham
c32e2d4486
Set stackaddr if mmap is called in pthread_create (#606) 2022-09-08 21:07:53 -07:00
Justine Tunney
9f963dc597
Clean up some of the threading code 2022-09-08 12:31:56 -07:00
Justine Tunney
0547eabcd6
Polyfill IPv6 on non-Linux 2022-09-08 06:06:22 -07:00
Justine Tunney
b73e35c6fa
Improve open source compatibility
This change tunes the default stack size for the outside world to 8mb
while at the same time, reducing Cosmopolitan's default stack size to
64kb. You can override the stack size using STATIC_STACK_SIZE(). Your
build scripts should point to o//ape/public/ape.lds

This change also fixes the definition of SOMAXCONN and removes AF_RDS
since it's not polyfilled and Python 3.11 complained.
2022-09-08 03:19:35 -07:00
Justine Tunney
0e2b1bfeed
Make garbage collection thread safe
- You can now use _gc(malloc()) in multithreaded programs
- This change fixes a bug where fork() on NT disabled TLS
- Fixed TLS code morphing on XNU/NT, for R8-R15 registers
2022-09-08 02:33:01 -07:00
Justine Tunney
571c2c3c69
Fix race condition in makedirs() 2022-09-07 23:12:44 -07:00
Gavin Hayes
b7c07d548c
Add FP_ILOGB constants (#599) 2022-09-07 21:37:53 -07:00
Justine Tunney
6c323383e5
Make some fixups to POSIX threads 2022-09-07 21:13:50 -07:00
Justine Tunney
de511bc71a
Add phtread_setname_np() and pthread_getname_np() 2022-09-07 19:04:05 -07:00
Justine Tunney
114176c304
Add more foo64() functions for Rust 2022-09-07 05:48:57 -07:00
Gavin Hayes
a849a63771
Implement sigpending for sysv and nt (#597) 2022-09-07 05:38:12 -07:00
Justine Tunney
4339d9f15e
Add pthread attributes and other libc functions 2022-09-07 05:28:32 -07:00
Justine Tunney
8bd9ad8342
Simplify redbean serialization code 2022-09-06 20:07:29 -07:00
tkchia
3fdb1c14f1
Add VGA support on bare metal (#588)
If your main module has this declaration:

    STATIC_YOINK("vga_console");

Then a VGA driver will be linked into your executable which
displays your stdio characters on the PC display, whereas
before we could only use the serial port. Your display is an
ANSI terminal and it's still a work in progress.
2022-09-06 18:41:08 -07:00
Gavin Hayes
598640864a
Fix inability to add some signals to mask on NT (#596) 2022-09-06 18:35:26 -07:00
Justine Tunney
d861d2787b
Import OpenBSD sorting algorithms
OpenBSD's qsort() function is more secure than the ones used by
FreeBSD, NetBSD and MacOS. The best part is it goes faster too!
This change also imports the OpenBSD mergesort() and heapsort()
2022-09-06 17:52:26 -07:00
Justine Tunney
dbf12c30b0
Make more compatibility improvements 2022-09-06 12:36:34 -07:00
Justine Tunney
11ec60d5fc
Use a better sorting algorithm
This change changes qsort() to use the same code as NetBSD and MacOS
because it goes 6x faster than Musl's SmoothSort function. Smoothsort
can still be used if you need something that's provenly linearithmic.
This change also improves GNU Make performance on whole by 7 percent!

netbsd nearly   l:    70,196c    22,673ns   m:    68,428c    22,102ns
musl nearly     l:    53,844c    17,391ns   m:    58,726c    18,968ns
unixv6 nearly   l:    65,885c    21,280ns   m:    63,082c    20,375ns

netbsd reverse  l:   120,290c    38,853ns   m:   122,619c    39,605ns
musl reverse    l:   801,826c   258,985ns   m:   794,689c   256,680ns
unixv6 reverse  l:    58,977c    19,049ns   m:    59,764c    19,303ns

netbsd random   l:   146,745c    47,398ns   m:   145,782c    47,087ns
musl random     l:   855,804c   276,420ns   m:   850,912c   274,840ns
unixv6 random   l:   214,325c    69,226ns   m:   213,906c    69,090ns

netbsd 2n       l:    77,299c    24,967ns   m:    76,773c    24,797ns
musl 2n         l:   818,012c   264,213ns   m:   818,282c   264,301ns
unixv6 2n       l: 3,967,009c 1,281,322ns   m: 3,941,792c 1,273,177ns

https://justine.lol/dox/sort.pdf
2022-09-06 11:06:10 -07:00
Justine Tunney
55c6297e13
Make more compatibility improvements 2022-09-06 07:04:13 -07:00
Justine Tunney
12d9e1e128
Improve quality of our ANSI C clock() function
It now works most excellently across all supported operating
sytsems (earlier it didn't work on NT and XNU). Demo code is
available in examples/clock.c and this change also adds some
of the newer ANSI C time functions like timespec_get(), plus
timespec_getres() which hasn't even come out yet as it's C23
2022-09-05 23:03:49 -07:00
Justine Tunney
7ff0ea8c13
Make pthread mutexes more scalable
pthread_mutex_lock() now uses a better algorithm which goes much faster
in multithreaded environments that have lock contention. This comes at
the cost of adding some fixed-cost overhead to mutex invocations. That
doesn't matter for Cosmopolitan because our core libraries all encode
locking operations as NOP instructions when in single-threaded mode.
Overhead only applies starting the moment you first call clone().
2022-09-05 15:57:51 -07:00
Gavin Hayes
7de2f229a7
Discard ignored signals on New Technology (#592) 2022-09-05 09:17:56 -07:00
Justine Tunney
8cd4248f7f
Make SIG_DFL / SIG_IGN C++ compatible 2022-09-05 08:49:26 -07:00
Justine Tunney
d721ff8938
Remove testonly keyword 2022-09-05 08:41:43 -07:00
Justine Tunney
9be364d40a
Implement POSIX threads API 2022-09-05 08:27:15 -07:00
Justine Tunney
af24f21556
Unsandbox assembly generation 2022-09-05 08:27:15 -07:00
Justine Tunney
6a04bc3318
Release redbean 2.0.18 2022-09-04 06:57:59 -07:00
Justine Tunney
8dd4ec68d0
Add more missing C / C++ headers 2022-09-04 04:53:52 -07:00
Gavin Hayes
9c017c98d3
Add sysconf to unistd.h (#575) 2022-09-04 00:10:17 -07:00
Gavin Hayes
74879bf067
Add pollfd to poll.h (#576) 2022-09-04 00:09:50 -07:00
Gavin Hayes
1ef955c33b
Improve zipos path handling (#579)
This change adds opendir, readdir, and stat support for the /zip/ root,
as well as directories not explicitly encoded in the zip file.
2022-09-03 22:50:23 -07:00
Gavin Hayes
494d74271b
Fix inet_ntop(AF_INET6) buffer length (#582) 2022-09-03 22:46:45 -07:00
Justine Tunney
a4522fba8d Fix build 2022-09-03 21:10:37 -07:00
Justine Tunney
0c70e8963d Add notpossible keyword
This is the same as `unreachable` except it always traps violations,
even if we're not running in MODE=dbg. This is useful for impossible
conditions relating to system calls. It avoids terrifying bugs where
control falls through to an unrelated function.
2022-09-03 20:35:31 -07:00
Justine Tunney
b66bd064d8 Improve quality of uname/gethostname/getdomainname 2022-09-03 20:20:40 -07:00
Justine Tunney
c5c4dfcd21 Improve quality of raise(), abort(), and tkill()
This change fixes a nasty bug where SIG_IGN and SIG_DFL weren't working
as advertised on BSDs. This change also fixes the tkill() definition on
MacOS so it maps to __pthread_kill().
2022-09-03 20:17:54 -07:00
Justine Tunney
c5659b93f8 Create unit test framework for subprocesses 2022-09-03 18:02:01 -07:00
Gavin Hayes
263711965f
Change sigaction_f to match sysv signature (#585) 2022-09-02 05:08:35 -07:00
Gavin Hayes
d213a48190
Implement inet_pton for AF_INET6 (#583) 2022-08-31 15:14:42 -07:00
Gavin Hayes
adaaeb7dd0
Add itimerval to sys/time.h (#572) 2022-08-26 09:44:41 -07:00
Justine Tunney
5a632cf72d Add lchown, lchmod, statvfs, fstatvfs 2022-08-22 20:50:16 -07:00
Justine Tunney
58d22c7c3d Handle missing stdio handles on Windows 2022-08-22 17:03:11 -07:00
Justine Tunney
2227cefa5d Run fix-third-party.py and man2txt on sed 2022-08-21 22:26:48 -07:00
Justine Tunney
57b39d8793 Revert recent fixupobj.com change 2022-08-21 17:33:54 -07:00
Justine Tunney
bf757c4646 Fix _Thread_local with fork() on Windows 2022-08-21 15:51:44 -07:00
Justine Tunney
99a92048b4 Get awk to build and make it hackable 2022-08-21 13:39:20 -07:00
Justine Tunney
73845be1f0 Restore zip.com and .symtab files
This change restores the .symtab symbol table files in our flagship
programs (e.g. redbean.com, python.com) needed to show backtraces. This
also rolls back earlier changes to zip.com w.r.t. temp directories since
the right way to do it turned out to be the -b DIR flag.

This change also improves the performance of zip.com. It turned out
mmap() wasn't being used, because zip.com was assuming a 4096-byte
granularity, but cosmo requires 65536. There was also a chance to speed
up stdio scanning using the unlocked functions.
2022-08-21 00:17:20 -07:00
Justine Tunney
35203c0551 Do some string library work 2022-08-20 22:17:14 -07:00
Justine Tunney
83d41e4588 Clean up some code 2022-08-20 12:32:51 -07:00
Justine Tunney
d76dfadc7a Reduce GNU Make latency 17% for cosmo
fgets() is now 4x faster which makes Make 2% faster. Landlock Make now
has a builtin $(uniq ...) function that uses critbit trees rather than
functional programming. Since uniq is the most important function this
optimization makes our cold start latency 15% faster.
2022-08-19 15:29:24 -07:00