Commit graph

1685 commits

Author SHA1 Message Date
Justine Tunney
cf1559c448
Remove __threaded variable 2024-07-28 23:43:30 -07:00
Justine Tunney
01b09bc817
Support printf %n directive 2024-07-28 22:27:06 -07:00
Justine Tunney
18964e5d76
Fix remove() directory on Windows 2024-07-28 17:31:21 -07:00
Justine Tunney
e18fe1e112
Freshen build/bootstrap/cocmd
See https://news.ycombinator.com/item?id=41055121
2024-07-27 23:22:11 -07:00
Justine Tunney
8621034d42
Release Cosmopolitan v3.6.2 2024-07-27 20:20:54 -07:00
Justine Tunney
f147d3dde9
Fix some static analysis issues 2024-07-27 09:16:54 -07:00
Justine Tunney
cdfcee51ca
Properly serialize fork() operations
This change solves an issue where many threads attempting to spawn forks
at once would cause fork() performance to degrade with the thread count.
Things got real nasty on NetBSD, which slowed down the whole test fleet,
because there's no vfork() and we're forced to use fork() in our server.

   threads      count task
         1       1062 fork+exit+wait
         2        668 fork+exit+wait
         4         66 fork+exit+wait
         8         19 fork+exit+wait
        16         22 fork+exit+wait
        32         16 fork+exit+wait

Things are now much less bad on NetBSD, but not great, since it does not
have futexes; we rely on its semaphore file descriptors to do conditions

   threads      count task
         1       1085 fork+exit+wait
         2        842 fork+exit+wait
         4        532 fork+exit+wait
         8        400 fork+exit+wait
        16        276 fork+exit+wait
        32         66 fork+exit+wait

With OpenBSD which also lacks vfork(), things were just as bad as NetBSD

   threads      count task
         1        584 fork+exit+wait
         2        687 fork+exit+wait
         4        206 fork+exit+wait
         8         24 fork+exit+wait
        16         33 fork+exit+wait
        32         26 fork+exit+wait

But since OpenBSD has futexes fork() works terrifically thanks to *NSYNC

   threads      count task
         1        525 fork+exit+wait
         2        580 fork+exit+wait
         4        451 fork+exit+wait
         8        479 fork+exit+wait
        16        408 fork+exit+wait
        32        373 fork+exit+wait

This issue would most likely only manifest itself, when pthread_atfork()
callers manage to slip a spin lock into the outermost position of fork's
list of locks. Since fork() is very slow, a spin lock can be devastating

Needless to say vfork() rules and anyone who says differently is kidding
themselves. Look at what a FreeBSD 14.1 virtual machine with equal specs
can do over the course of three hundred milliseconds.

   threads      count task
         1       2559 vfork+exit+wait
         2       5389 vfork+exit+wait
         4      34933 vfork+exit+wait
         8      43273 vfork+exit+wait
        16      49648 vfork+exit+wait
        32      40247 vfork+exit+wait

So it's a shame that so few OSes support vfork(). It creates an unsavory
situation, where someone wanting to build a server that spawns processes
would be better served to not use threads and favor a multiprocess model
2024-07-27 08:23:44 -07:00
Justine Tunney
18a620cc1a
Make some improvements of little consequence 2024-07-27 08:20:18 -07:00
Justine Tunney
690d3df66e
Expand the virtual address space on Windows 2024-07-27 08:19:05 -07:00
Justine Tunney
642e9cb91a
Introduce cosmocc flags -mdbg -mtiny -moptlinux
The cosmocc.zip toolchain will now include four builds of the libcosmo.a
runtime libraries. You can pass the -mdbg flag if you want to debug your
cosmopolitan runtime. You can pass the -moptlinux flag if you don't want
windows code lurking in your binary. See tool/cosmocc/README.md for more
details on how these flags may be used and their important implications.
2024-07-26 05:10:25 -07:00
Justine Tunney
59692b0882
Make spinlocks faster (take two)
This change is green on x86 and arm test fleet.
2024-07-26 00:45:24 -07:00
Justine Tunney
02e1cbcd00
Revert "Make spin locks go faster"
This reverts commit c8e25d811c.
2024-07-25 22:24:32 -07:00
Justine Tunney
0679cfeb41
Fix build 2024-07-25 22:12:08 -07:00
Justine Tunney
c8e25d811c
Make spin locks go faster 2024-07-25 17:37:11 -07:00
Justine Tunney
7d88343973
Release Cosmopolitan v3.6.1 2024-07-25 13:34:02 -07:00
Justine Tunney
2c4b88753b
Add special errno handling to libcxx 2024-07-25 01:23:02 -07:00
Justine Tunney
0f486a13c8
Fix fdlibm license 2024-07-24 20:42:08 -07:00
Justine Tunney
1020dd41cc
bzero() should be defined without special defines 2024-07-24 16:15:30 -07:00
Justine Tunney
d3a13e8d70
Improve lock hierarchy
- NetBSD no longer needs a spin lock to create semaphores
- Windows fork() now locks process manager in correct order
2024-07-24 16:05:48 -07:00
Justine Tunney
7ba9a73840
Remove more _Atomic keywords from public headers
It's been thirteen years and C++ still hasn't implemented this wonderful
simple builtin keyword. In C++23 a solution was provided for making this
work in C++ which is libcxx's stdatomic.h. Including that header schleps
in literally 253 unique header files!! Many of the header files it needs
are libc header files like pthread.h where we need to have the _Atomic()
keyword, but since <atomic> depends on pthreads we can't have it include
the <stdatomic.h> header that defines _Atomic for C++ users, and instead
we simply make the type non-atomic, hoping and praying only C code shall
use those internal data structures. This just shows how STL clowns can't
be trusted to define the innermost primitives of a language. They should
instead be focusing on being the best at algorithms and data structures.
2024-07-24 13:56:03 -07:00
Justine Tunney
5dd7ddb9ea
Remove bad defines from early days of project
These definitions were causing issues with building LLVM. It is possible
they also caused crashes we've seen with our MacOS ARM64 OpenMP support.
2024-07-24 12:11:21 -07:00
Justine Tunney
f25fbbaaeb
Use libcxx abi v1 2024-07-24 09:49:48 -07:00
Justine Tunney
fbc4b03d4c
Restore support for AMD K8 2024-07-24 08:59:29 -07:00
Justine Tunney
e398f3887c
Make more improvements to threads and mappings
- NetBSD should now have faster synchronization
- POSIX barriers may now be shared across processes
- An edge case with memory map tracking has been fixed
- Grand Central Dispatch is no longer used on MacOS ARM64
- POSIX mutexes in normal mode now use futexes across processes
2024-07-24 01:19:54 -07:00
Justine Tunney
5660ec4741
Release Cosmopolitan v3.6.0
This release is an atomic upgrade to GCC 14.1.0 with C23 and C++23
2024-07-23 03:28:19 -07:00
Justine Tunney
62ace3623a
Release Cosmopolitan v3.5.9 2024-07-22 21:02:40 -07:00
Justine Tunney
6e809ee49b
Add unit test for process shared conditions 2024-07-22 18:48:54 -07:00
Justine Tunney
61c36c1dd6
Allow pthread_condattr_setpshared() to set shared 2024-07-22 18:41:45 -07:00
Justine Tunney
0a9a6f86bb
Support process shared condition variables 2024-07-22 16:35:29 -07:00
Justine Tunney
3de6632be6
Graduate some clock_gettime() constants to #define
- CLOCK_THREAD_CPUTIME_ID
- CLOCK_PROCESS_CPUTIME_ID

Cosmo now supports the above constants universally across supported OSes
therefore it's now safe to let programs detect their presence w/ #ifdefs
2024-07-22 07:14:35 -07:00
Justine Tunney
62a97c919f
Fix typos in APE specification
Fixes #1244
2024-07-22 01:41:44 -07:00
Justine Tunney
5d2d9e9640
Add back missing TlsAlloc() call
Cosmopolitan Libc once called this important function although somewhere
along the way, possibly in a refactoring, it got removed and __tls_alloc
has always been zero ever since.
2024-07-21 20:45:27 -07:00
Justine Tunney
e08a4cd99e
Release Cosmopolitan v3.5.8 2024-07-21 17:01:33 -07:00
Justine Tunney
7ebaff34c6
Fix ctype.h and wctype.h 2024-07-21 15:54:17 -07:00
Justine Tunney
30afd6ddbb
Improve multithreading 2024-07-21 14:40:45 -07:00
Justine Tunney
d3167126aa
Fix regression with last commit 2024-07-20 16:43:48 -07:00
Justine Tunney
29ce25c767
Start writing formal specification for APE 2024-07-20 10:04:22 -07:00
Justine Tunney
7996bf67b5
Release Cosmopolitan v3.5.7 2024-07-20 03:48:57 -07:00
Justine Tunney
626a5d02ee
Add missing lock statement 2024-07-20 03:47:22 -07:00
Justine Tunney
527aaa41eb
Prevent MODE=tiny ShowCrashReports() looping 2024-07-20 03:34:37 -07:00
Justine Tunney
3374cbba73
Release Cosmopolitan v3.5.6 2024-07-20 02:43:10 -07:00
Justine Tunney
2018cac11f
Use better memory strategy on Windows
Rather than using the the rollo global to pick addresses, we select them
randomly now using a conservative vaspace.
2024-07-20 02:20:03 -07:00
Justine Tunney
6a5d4ed65b
Fix bug with disabling sigaltstack() 2024-07-20 01:00:16 -07:00
Justine Tunney
493ffc9b7f
Release Cosmopolitan v3.5.5 2024-07-19 22:33:17 -07:00
Justine Tunney
101fb3d9b3
Make some new Windows 10 memory APIs available 2024-07-19 22:26:49 -07:00
Justine Tunney
86d884cce2
Get rid of .internal.h convention in LIBC_INTRIN 2024-07-19 19:38:00 -07:00
Justine Tunney
0ed916ad5c
Fix a bug in example code 2024-07-19 19:11:28 -07:00
Justine Tunney
1029dcc597
Reduce default stack size from 256kb to 81kb
This is the same as Musl Libc. Please note it only applies to threads.
2024-07-19 14:18:06 -07:00
Ikko Eltociear Ashimine
c697133a2d
Fix typo in accept4-sysv.c (#1235) 2024-07-19 05:46:29 -07:00
Justine Tunney
1ff037df3c
Add some documentation 2024-07-19 04:46:26 -07:00
Justine Tunney
567d8fe32d
Create variables for page size 2024-07-18 21:16:53 -07:00
Justine Tunney
23dfb79d33
Fix minor suboptimalities in memory manager 2024-07-18 19:19:51 -07:00
Justine Tunney
76cea6c687
Squeeze more performance out of memory manager 2024-07-08 03:08:42 -07:00
Justine Tunney
3f2a1b696e
Fix greenbean example
The memory leak detector was crashing. When using gc() you shouldn't use
the CheckForMemoryLeaks() function from inside the same function, due to
how it runs the atexit handlers.
2024-07-07 17:52:33 -07:00
Justine Tunney
f7780de24b
Make realloc() go 100x faster on Linux/NetBSD
Cosmopolitan now supports mremap(), which is only supported on Linux and
NetBSD. First, it allows memory mappings to be relocated without copying
them; this can dramatically speed up data structures like std::vector if
the array size grows larger than 256kb. The mremap() system call is also
10x faster than munmap() when shrinking large memory mappings.

There's now two functions, getpagesize() and getgransize() which help to
write portable code that uses mmap(MAP_FIXED). Alternative sysconf() may
be called with our new _SC_GRANSIZE. The madvise() system call now has a
better wrapper with improved documentation.
2024-07-07 12:40:30 -07:00
Justine Tunney
6be030cd7c
Fix MODE=tinylinux build 2024-07-06 01:51:08 -07:00
Justine Tunney
8c645fa1ee
Make mmap() scalable
It's now possible to create thousands of thousands of sparse independent
memory mappings, without any slowdown. The memory manager is better with
tracking memory protection now, particularly on Windows in a precise way
that can be restored during fork(). You now have the highest quality mem
manager possible. It's even better than some OSes like XNU, where mmap()
is implemented as an O(n) operation which means sadly things aren't much
improved over there. With this change the llamafile HTTP server endpoint
at /tokenize with a prompt of 50 tokens is now able to handle 2.6m r/sec
2024-07-05 23:26:00 -07:00
Justine Tunney
3756870635
Implement new red-black tree 2024-07-05 12:56:03 -07:00
Justine Tunney
fc65422660
Remove __mmap() and __munmap() 2024-07-05 12:55:46 -07:00
Justine Tunney
01587de761
Simplify memory manager 2024-07-05 05:47:15 -07:00
Justine Tunney
5a9a08d1cf
Fix regression in elf2pe program 2024-07-04 04:02:20 -07:00
Justine Tunney
bd6d9ff99a
Get deathstar demo working again on metal 2024-07-04 03:44:17 -07:00
Justine Tunney
15ea0524b3
Reduce code size of mandatory runtime
This change reduces o/tiny/examples/life from 44kb to 24kb in size since
it avoids linking mmap() when unnecessary. This is important, to helping
cosmo not completely lose touch with its roots.
2024-07-04 02:50:20 -07:00
Justine Tunney
70f77aad33
Release Cosmopolitan v3.5.4 2024-07-01 07:17:57 -07:00
Justine Tunney
61370983e1
Complete the Windows TLS fix made in e437bed00 2024-07-01 07:17:57 -07:00
Justine Tunney
239f8ce76e
Release Cosmopolitan v3.5.3 2024-07-01 02:07:56 -07:00
Justine Tunney
e437bed006
Fix crash caused when Windows needs a lot of TLS 2024-06-30 20:53:43 -07:00
Justine Tunney
76957983cf
Make POSIX threads improvements
- Ensure SIGTHR isn't blocked in newly created threads
- Use TIB rather than thread_local for thread atexits
- Make POSIX thread keys atomic within thread
- Don't bother logging prctl() to --strace
- Log thread destructor names to --strace
2024-06-30 15:38:59 -07:00
Justine Tunney
387310c659
Fix issue with ctl::vector constructor 2024-06-30 02:26:38 -07:00
Justine Tunney
4cb5e21ba8
Introduce pthread_decimate_np() api
This is useful with CheckForMemoryLeaks().
2024-06-30 02:26:06 -07:00
Justine Tunney
1bf2d8e308
Further improve mmap() locking story
The way to use double linked lists, is to remove all the things you want
to work on, insert them into a new list on the stack. Then once you have
all the work items, you release the lock, do your work, and then lock it
again, to add the shelled out items back to a global freelist.
2024-06-29 17:12:43 -07:00
Justine Tunney
98e684622b
Add iostream to CTL 2024-06-29 15:45:09 -07:00
Justine Tunney
617ddfee93
Release Cosmopolitan v3.5.2 2024-06-29 10:58:47 -07:00
Justine Tunney
464858dbb4
Fix bugs with new memory manager
This fixes a regression in mmap(MAP_FIXED) on Windows caused by a recent
revision. This change also fixes ZipOS so it no longer needs a MAP_FIXED
mapping to open files from the PKZIP store. The memory mapping mutex was
implemented incorrectly earlier which meant that ftrace and strace could
cause cause crashes. This lock and other recursive mutexes are rewritten
so that it should be provable that recursive mutexes in cosmopolitan are
asynchronous signal safe.
2024-06-29 10:53:57 -07:00
Justine Tunney
a16eb76f5e
Fix build break 2024-06-29 04:34:27 -07:00
Justine Tunney
021c53ba32
Add more CTL content 2024-06-28 19:09:54 -07:00
Justine Tunney
572ac7d100
Release Cosmopolitan v3.5.1 2024-06-24 06:54:15 -07:00
Justine Tunney
d461c6f47d
Do more quality assurance work 2024-06-24 06:53:49 -07:00
Justine Tunney
67b19ae733
Release Cosmopolitan v3.5.0 2024-06-23 22:45:14 -07:00
Justine Tunney
c4c812c154
Introduce ctl::set and ctl::map
We now have a C++ red-black tree implementation that implements standard
template library compatible APIs while compiling 10x faster than libcxx.
It's not as beautiful as the red-black tree implementation in Plinko but
this will get the job done and the test proves it upholds all invariants

This change also restores CheckForMemoryLeaks() support and fixes a real
actual bug I discovered with Doug Lea's dlmalloc_inspect_all() function.
2024-06-23 22:27:11 -07:00
Justine Tunney
f2c8ddbbe3
Fix --strace use-after-free in pthread_join() 2024-06-22 06:05:52 -07:00
Justine Tunney
d1d4388201
Delete ASAN
It hasn't been helpful enough to be justify the maintenance burden. What
actually does help is mprotect(), kprintf(), --ftrace and --strace which
can always be counted upon to work correctly. We aren't losing much with
this change. Support for ASAN on AARCH64 was never implemented. Applying
ASAN to the core libc runtimes was disabled many months ago. If there is
some way to have an ASAN runtime for user programs that is less invasive
we can potentially consider reintroducing support. But now is premature.
2024-06-22 05:45:49 -07:00
Justine Tunney
6ffed14b9c
Rewrite memory manager
Actually Portable Executable now supports Android. Cosmo's old mmap code
required a 47 bit address space. The new implementation is very agnostic
and supports both smaller address spaces (e.g. embedded) and even modern
56-bit PML5T paging for x86 which finally came true on Zen4 Threadripper

Cosmopolitan no longer requires UNIX systems to observe the Windows 64kb
granularity; i.e. sysconf(_SC_PAGE_SIZE) will now report the host native
page size. This fixes a longstanding POSIX conformance issue, concerning
file mappings that overlap the end of file. Other aspects of conformance
have been improved too, such as the subtleties of address assignment and
and the various subtleties surrounding MAP_FIXED and MAP_FIXED_NOREPLACE

On Windows, mappings larger than 100 megabytes won't be broken down into
thousands of independent 64kb mappings. Support for MAP_STACK is removed
by this change; please use NewCosmoStack() instead.

Stack overflow avoidance is now being implemented using the POSIX thread
APIs. Please use GetStackBottom() and GetStackAddr(), instead of the old
error-prone GetStackAddr() and HaveStackMemory() APIs which are removed.
2024-06-22 05:45:11 -07:00
Steven Dee (Jōshin)
9a5a13854d
CTL: utility.h, use ctl::swap in string (#1227)
* Add ctl utility.h

Implements forward, move, swap, and declval. This commit also adds a def
for nullptr_t to cxx.inc. We need it now because the CTL headers stopped
including anything from libc++, so we no longer get their basic types.

* Use ctl::swap in string

The STL spec says that swap is located in the string_view header anyawy.
Performance-wise this is a noop, but it’s slightly cleaner.
2024-06-19 01:00:59 -04:00
Steven Dee (Jōshin)
a795017416
Fix c.inc _Atomic define for C++ (#1231)
c.inc (AFAICT erroneously) defined _Atomic(t) as `volatile t *`, when it
should have just said `volatile t`, when __STDC_VERSION__ was too small.
This happens when we’re compiling C++, but in C++11, _Atomic is a define
supplied by the STL rather than a keyword supplied by the compiler. Wait
though, it gets better: in C++11, _Atomic hooks you into the morass that
is stdatomic.h, and ultimately refers everything back to std::atomic<T>.

The gory, horrifying details are in libcxx's __atomic/cxx_atomic_impl.h.
The tldr is that for our purposes it’s fine to just say volatile and use
the normal libc/intrin/atomic.h functions.
2024-06-17 21:12:02 -07:00
Jōshin
89fc95fefd
Rerun clang-format on the repo (#1217)
🚨 clang-format changes output per version!

This is with version 19.0.0. The modifications seem to be fixing the old
version’s errors - mainly involving omitted whitespace around binary ops
and inserted whitespace between goto labels and colons (if followed by a
curly brace.)

Also fixes a few mistakes made by e.g. someone (ahem) forgetting to pass
his ctl/string.h modifications through it.

We should add this to .git-blame-ignore-revs once we have its final hash
on master.
2024-06-15 16:34:48 -04:00
Justine Tunney
cc2c1893c5
Fix some nits 2024-06-05 04:05:49 -07:00
Justine Tunney
3093f0e467
Release Cosmopolitan v3.4.0 2024-06-05 03:07:03 -07:00
Justine Tunney
3609f65de3
Make malloc() go 200x faster
If pthread_create() is linked into the binary, then the cosmo runtime
will create an independent dlmalloc arena for each core. Whenever the
malloc() function is used it will index `g_heaps[sched_getcpu() / 2]`
to find the arena with the greatest hyperthread / numa locality. This
may be configured via an environment variable. For example if you say
`export COSMOPOLITAN_HEAP_COUNT=1` then you can restore the old ways.
Your process may be configured to have anywhere between 1 - 128 heaps

We need this revision because it makes multithreaded C++ applications
faster. For example, an HTTP server I'm working on that makes extreme
use of the STL went from 16k to 2000k requests per second, after this
change was made. To understand why, try out the malloc_test benchmark
which calls malloc() + realloc() in a loop across many threads, which
sees a a 250x improvement in process clock time and 200x on wall time

The tradeoff is this adds ~25ns of latency to individual malloc calls
compared to MODE=tiny, once the cosmo runtime has transitioned into a
fully multi-threaded state. If you don't need malloc() to be scalable
then cosmo provides many options for you. For starters the heap count
variable above can be set to put the process back in single heap mode
plus you can go even faster still, if you include tinymalloc.inc like
many of the programs in tool/build/.. are already doing since that'll
shave tens of kb off your binary footprint too. Theres also MODE=tiny
which is configured to use just 1 plain old dlmalloc arena by default

Another tradeoff is we need more memory now (except in MODE=tiny), to
track the provenance of memory allocation. This is so allocations can
be freely shared across threads, and because OSes can reschedule code
to different CPUs at any time.
2024-06-05 02:02:14 -07:00
Justine Tunney
9906f299bb
Refactor and improve CTL and other code 2024-06-04 05:45:48 -07:00
Justine Tunney
b003888696
Make __demangle() heap 10% more compact 2024-06-02 16:18:55 -07:00
Justine Tunney
2ca491dc56
Write more __demangle() tests 2024-06-02 07:37:15 -07:00
Justine Tunney
9aa353d88b
Document __demangle() and fix a const func ptr bug 2024-06-02 04:15:48 -07:00
Justine Tunney
c67faf61df
Delete some unintentional code 2024-06-01 20:36:58 -07:00
Justine Tunney
165c6b37e2
Add C++ demangling to privileged runtime
Cosmo will now print C++ symbols correctly in --ftrace logs and
backtraces. Doing this required reducing the memory requirement
of the __demangle() function by 3x. This was accomplished using
16-bit indices and 16-bit malloc granularity. That puts a limit
on the longest symbol we can successfully decode, which I think
would be around 6553 characters long, given a 65536-byte buffer
2024-06-01 20:10:58 -07:00
Jōshin
f032b5570b
Run clang-format (#1197) 2024-06-01 16:30:43 -04:00
Justine Tunney
ea081b262c
Add some noexcept annotations 2024-06-01 03:19:53 -07:00
Justine Tunney
9b6718ac99
Improve backtraces
We're now able to rewind the instruction pointer in x86 backtraces. This
helps ensure addr2line cannot print information about unrelated adjacent
code. I've restored -fno-schedule-insns2 in most cases because it really
does cause unpredictable breakage for backtraces.
2024-05-30 15:23:11 -07:00
Justine Tunney
cd672e251f
Improve crash signal reporting on Windows
This change fixes a bug where exiting a crash signal handler on Windows
after adding the signal to uc_sigmask, but not correcting the CPU state
would cause the signal handler to loop infinitely, causing process hang

Another issue is that very tiny programs, that don't link posix signals
would not have their SIGILL / SIGSEGV / etc. status reported to Cosmo's
bash shell when terminating on crash. That's fixed by a tiny handler in
WinMain() that knows how to map WIN32 crash codes to the POSIX flavors.
2024-05-30 14:04:10 -07:00
Justine Tunney
e4d25d68e4
Drop support for Windows 8
Microsoft caused some very gentle breakages for Cosmopolitan. They
removed the version information from the PEB which caused uname to
report WINDOWS 0.0.0. We should have called GetVersionExW but that
doesn't really exist anymore either. Windows policy is now to give
whatever version we used in ape/ape.S. Windows8 has been EOL since
2023-01-10 so lets avoid our modern executables being relegated to
legacy infrastructure. Requiring Windows 10+ going forward lets us
remove runtime compatibility bloat from the codebase. Further note
Cosmopolitan maintains a Windows Vista branch on GitHub, so anyone
preferring the older versions, can still have a future with Cosmo.

Another neat thing this fixes is UTF-8 support in the console. The
changes Microsoft made broke the if statement that enabled UTF8 in
terminals. This explains why bug reports had broken arrows. In the
future this should be less of an issue, since the PEB code is gone
which means we more strictly conform to only Microsoft's WIN32 API
2024-05-29 19:37:47 -07:00
Justine Tunney
f31a98d50a
Fix bug with realpath() on Windows 2024-05-29 18:47:01 -07:00
Justine Tunney
2816df59b2
Increase tinymalloc granularity 2024-05-29 18:26:01 -07:00
Justine Tunney
4c77acdfcf
Add LoadZipArgs() to <cosmo.h> 2024-05-29 10:12:20 -07:00
Justine Tunney
b74b974cfd
Introduce #include <tinygetopt.h>
The normal getopt() function is bloated because it links printf(). This
change exports the original authentic bsd getopt function, that cosmo's
always used internally so cosmocc users don't need to include internals
2024-05-29 10:11:17 -07:00
Justine Tunney
07cef612c3
Make dlmalloc 2.4x faster for multithreading
This change adds a TLS freelist for small dynamic memory allocations.
Cosmopolitan's TIB is now 512 bytes in size. Single-threaded malloc()
performance isn't impacted by this, until pthread_create() is called.
Single-threaded programs may also want to consider using:

    #include "libc/mem/tinymalloc.inc"

Which will shave 30k off the executable size and sometimes go faster.
2024-05-28 11:18:34 -07:00
Justine Tunney
deaef81463
Favor siginfo_t over struct siginfo 2024-05-28 02:34:17 -07:00
Justine Tunney
8e68384e15
Upgrade to 2022-era LLVM LIBCXX 2024-05-27 02:12:27 -07:00
Justine Tunney
2f4ca71f26
Release Cosmopolitan v3.3.10 2024-05-26 22:13:45 -07:00
Justine Tunney
086d7006da
Improve crash handler on XNU
This avoids an issue where a crash signal could cause the MacOS process
to freeze and consume all CPU rather than dying as it rightfully should
2024-05-26 18:42:09 -07:00
Gavin Hayes
0a51241f7a
ntspawn: fix initializing NtStartupInfoEx (#1190) 2024-05-26 20:54:09 -04:00
Justine Tunney
c68f6599e5
Fix definition of getpeername on FreeBSD
We were using the COMPAT magic number, which was recently removed.
2024-05-26 17:03:22 -07:00
Justine Tunney
af3f62a71a
Ensure io requests are always capped at 0x7ffff000
This gives us the Linux behavior across platforms.

Fixes #1189
2024-05-26 16:53:13 -07:00
Justine Tunney
6cf9b9e0fc
Release Cosmopolitan v3.3.9 2024-05-26 15:28:03 -07:00
Justine Tunney
1d4b452839
Refactor some code 2024-05-26 06:03:50 -07:00
Justine Tunney
edb03b89d8
Make stdin unbuffered when appropriate 2024-05-25 07:57:13 -07:00
Justine Tunney
7724664b13
Release Cosmopolitan v3.3.8 2024-05-25 05:59:20 -07:00
Justine Tunney
1df4296208
Fix stdio for character device regression
Caused by ed93fc3dd7
2024-05-25 05:58:09 -07:00
Justine Tunney
ce9aeb2aed
Release Cosmopolitan v3.3.7 2024-05-24 19:37:21 -07:00
Justine Tunney
ed93fc3dd7
Fix fread() with 2gb+ sizes 2024-05-24 19:28:23 -07:00
Justine Tunney
5f61d273e4
Add hwap constants to sys/auxv.h 2024-05-24 11:44:44 -07:00
Justine Tunney
bf3531de81
Make crash reports reliable in multithreaded case 2024-05-24 11:44:44 -07:00
Justine Tunney
f029375d39
Introduce MAP_HUGETLB 2024-05-24 11:44:44 -07:00
Justine Tunney
9b87dd2b87
Refactor some code 2024-05-24 11:44:44 -07:00
Jōshin
787b04f752
Run all BLAKE2B256 test vectors (#1185) 2024-05-24 10:59:23 -07:00
Justine Tunney
0b59f01b43
Put confstr() in unistd.h
Fixes #1184
2024-05-21 15:35:06 -07:00
Justine Tunney
cf70a44756
Support shebang on Windows
Fixes #1010
2024-05-20 22:11:42 -07:00
Jōshin
4292348707
Import libbsd readpassphrase (#1182)
Included from:

https://gitlab.freedesktop.org/libbsd/libbsd

At commit 04a24db27ad1572f766bad772cdd9c146e6d9cf0.
2024-05-20 03:37:30 -04:00
Jōshin
47183551d6
Fix sleb64 (#1179)
Fixes #920
2024-05-20 00:33:17 -07:00
Gavin Hayes
624119ea38
Fix NT accept/connect not initializing with SO_UPDATE_*_CONTEXT (#1164) 2024-05-17 02:45:30 -07:00
Justine Tunney
6659981457
Release Cosmopolitan v3.3.6 2024-05-08 04:20:22 -07:00
Justine Tunney
ae2a7ac844
Fix thread-local storage bugs on aarch64
This change fixes an issue where .tbss memory might not be initialized.
2024-05-08 04:20:22 -07:00
Justine Tunney
793393a341
Polyfill SA_RESETHAND on MacOS ARM64
This change solves the XNU crash loop mystery. Apple's documentation
claims to support this feature, but they only define the constant in
their header files. The kernel acknowledges thi SA_RESETHAND bit, by
clearing it from the sa_flags state, returns zero, and does nothing.
2024-05-08 04:20:22 -07:00
Justine Tunney
952b9009e8
Avoid crash looping on AARCH64 2024-05-08 04:20:22 -07:00
Justine Tunney
df68a6362b
Release Cosmopolitan v3.3.5 2024-05-07 18:18:10 -07:00
Justine Tunney
19c81863a3
Improve crash backtrace reliability
We're now able to pretty print a C++ backtrace upon crashing in pretty
much any runtime execution scenario. The default pledge sandbox policy
on Linux is now to return EPERM. If you call pledge and have debugging
functions linked (e.g. GetSymbolTable) then the symbol table shall get
loaded before any security policy is put in place. This change updates
build/bootstrap/fixupobj too and fixes some other sneaky build errors.
2024-05-07 18:10:28 -07:00
Jōshin
7d31fc311a
Loaders rewrite argv[0] for old binaries (#1170)
For this to work, a loader has to be able to tell the difference between
an ‘old’ and a ‘new’ binary. This is achieved via a repurposing of ELF’s
e_flags field. We previously tried to use the padding in e_ident for it,
but binutils was resetting it to zero in e.g. strip.

This introduces one new ELF flag for cosmopolitan binaries. It is called
`EF_APE_MODERN`. We choose 0x101ca75, "lol cat 5".

It should now be safe to install the ape loader binfmt registration with
the `P` flag.
2024-05-07 20:42:18 -04:00
Justine Tunney
57c0b065c8
Make old C++ demangler asynchronous signal safe
It's now possible to safely print C++ backtraces from signal handlers.
This symbol demangler doesn't need malloc, tls, or even static memory.
Additionally, this change makes it 2x faster and adds test cases. It's
almost as performant and accurate as the libcxxabi implementation now.
2024-05-07 03:41:33 -07:00
Justine Tunney
a6ecbb747d
Introduce libc/mem/tinymalloc.inc
This allocator shaves ~20kb off single-threaded tool programs and is
slightly faster than proper malloc for simple non-demanding programs
2024-05-07 03:19:49 -07:00
Justine Tunney
5fd7b07fac
Improve AVX512 feature detection 2024-05-07 03:19:49 -07:00
Justine Tunney
06d916b449
Add VirtualAlloc2 WIN32 API 2024-05-04 23:26:40 -07:00
Justine Tunney
f9fc7eb49f
Fix MODE=dbg build errors 2024-05-04 23:20:12 -07:00
Justine Tunney
b0df6c1fce
Implement proper time zone support
Cosmopolitan now supports 104 time zones. They're embedded inside any
binary that links the localtime() function. Doing so adds about 100kb
to the binary size. This change also gets time zones working properly
on Windows for the first time. It's not needed to have /etc/localtime
exist on Windows, since we can get this information from WIN32. We're
also now updated to the latest version of Paul Eggert's TZ library.
2024-05-04 23:06:37 -07:00
Justine Tunney
d5ebb1fa5b
Add MapViewOfFile3 WIN32 API 2024-05-04 12:25:41 -07:00
Justine Tunney
130fd66f9e
Release Cosmopolitan v3.3.4 2024-05-03 09:25:02 -07:00
Gavin Hayes
deff138e7e
recvfrom: don't convert address if addrsize is 0 (#1153) 2024-05-03 08:03:57 -07:00
Gavin Hayes
b6e40a3a58
Add /dev/(u)random on NT (#1163) 2024-05-03 07:59:51 -07:00
Cadence Ember
8f6bc9dabc
Let signals interrupt fgets unless SA_RESTART set (#1152) 2024-05-03 07:49:41 -07:00
Justine Tunney
181cd4cbe8
Add sysctlbyname() for MacOS 2024-05-02 23:21:43 -07:00
Justine Tunney
5c6877b02b
Introduce support for trapping math
The feenableexcept() and fedisableexcept() APIs are now provided which
let you detect when NaNs appear the moment it happens from anywhere in
your program. Tests have also been added for the mission critical math
functions expf() and erff(), whose perfect operation has been assured.
See examples/trapping.c to see how to use this powerful functionality.
2024-04-30 13:38:43 -07:00
Justine Tunney
403bc25412
Make MXCSR editable by signal handlers on Windows
It's now possible to recover from floating point exception traps.
2024-04-30 13:38:43 -07:00
Justine Tunney
2b00ee8723
Fix errno.h documentation typos
Closes #1156
2024-04-28 22:57:21 -07:00
Justine Tunney
ff2b6fc0b0
Add munlock() and mlock2() to pledge 2024-04-28 16:53:23 -07:00
Justine Tunney
0eef971494
Add much of C11 threads.h API 2024-04-28 07:04:08 -07:00
Gavin Hayes
2bfd6b37c1
Various paginate improvements (#1148)
* start on improving __paginate

* make __paginate more robust

* add __paginate_file

* cleanup __paginate unlinking
2024-04-26 23:12:25 -04:00
Gavin Hayes
69db501c68
Fix fork locking on win32 (#1141)
* Fix fork locking on win32

- __enable_threads / set __threaded in __proc_setup as threads are required for
  win32 subprocess management
- move mmi/fds locking out of pthread_atfork.c into fork.c so it's done anytime
  __threaded is set instead of being dependent of pthreads
- explicitly yoink _pthread_onfork_prepare, _pthread_onfork_parent, and
  _pthread_onfork_child in pthread_create.c so they are linked in in-case they
  are separated from _pthread_atfork

Big Thanks to @dfyz for help with locating the issue, testing, and devising a fix!

* fix child processes not being able to open files, initialize all necessary locks on fork
2024-04-25 23:01:27 -04:00
Jōshin
6e6fc38935
Apply clang-format update to repo (#1154)
Commit bc6c183 introduced a bunch of discrepancies between what files
look like in the repo and what clang-format says they should look like.
However, there were already a few discrepancies prior to that. Most of
these discrepancies seemed to be unintentional, but a few of them were
load-bearing (e.g., a #include that violated header ordering needing
something to have been #defined by a 'later' #include.)

I opted to take what I hope is a relatively smooth-brained approach: I
reverted the .clang-format change, ran clang-format on the whole repo,
reapplied the .clang-format change, reran clang-format again, and then
reverted the commit that contained the first run. Thus the full effect
of this PR should only be to apply the changed formatting rules to the
repo, and from skimming the results, this seems to be the case.

My work can be checked by applying the short, manual commits, and then
rerunning the command listed in the autogenerated commits (those whose
messages I have prefixed auto:) and seeing if your results agree.

It might be that the other diffs should be fixed at some point but I'm
leaving that aside for now.

fd '\.c(c|pp)?$' --print0| xargs -0 clang-format -i
2024-04-25 10:38:00 -07:00
Jōshin
cec4c04aa1
Remove .com logic from GetProgramExecutableName (#1134)
The `com` parameter to `TryPath` was always 1, so there was no reason to
have it. This patch changes the logic to be as though `com` was 0, which
provides a possible answer to the TODO question -- the answer is no.

If we never care about appending `.com`, then `CopyWithCwd` doesn't need
to return anything beyond a boolean success value.
2024-04-23 22:07:19 -04:00
Justine Tunney
223eb358c4
Import compiler warning fix from Musl 2024-04-23 09:29:28 -07:00
Szabolcs Nagy
ea3cc4ea0b
math: fix fma(x,y,0) when x*y rounds to -0
if x!=0, y!=0, z==0 then

  fma(x,y,z) == x*y

in all rounding modes, while adding z can ruin the sign of 0 if x*y
rounds to -0.
2024-04-23 09:22:33 -07:00
Jōshin
3e16e59f72
Move store after __asan_unpoison in __zipos_alloc (#1135)
Previously, the atomic store looked like it was happening while the
struct's memory was still poisoned. I was unable to observe any issues
with this, but this change seems to make the code more obviously correct
(at the cost of a redundant atomic store to zeroed space in case the map
needed to be extended.)
2024-04-10 23:04:03 -04:00
Justine Tunney
98d06b23cc
Release Cosmopolitan v3.3.3 2024-03-30 19:24:04 -07:00
Justine Tunney
9ff65cff63
Fix ZipOS extraction of ZIP64 assets 2024-03-30 19:00:40 -07:00
Justine Tunney
bef2cb05ff
Fix AVX512_BF16 2024-03-26 00:00:46 -07:00
mierenhoop
43885a76e4
Fix binary formatting for integers 2 and 3 (#1123) 2024-03-25 21:11:09 -07:00
Justine Tunney
045632a743
Make fixupobj work on Windows
This fixes a regression with recent cosmocc toolchain releases that
causes fixupobj to fail with "ifunc ftruncate #1 failed" on Windows
2024-03-24 03:14:25 -07:00
Justine Tunney
40b7da8422
Speed up fmaf() on x86 2024-03-22 19:26:56 -07:00
Justine Tunney
bb92347158
Add deprecated stubs for _bsf / _bsr 2024-03-05 09:45:51 -08:00
Justine Tunney
8bfd56b59e
Rename _bsr/_bsf to bsr/bsf
Now that these functions are behind _COSMO_SOURCE there's no reason for
having the ugly underscore anymore. To use these functions, you need to
pass -mcosmo to cosmocc.
2024-03-04 17:33:26 -08:00
Justine Tunney
a6baba1b07
Stop using .com extension in monorepo
The WIN32 CreateProcess() function does not require an .exe or .com
suffix in order to spawn an executable. Now that we have Cosmo bash
we're no longer so dependent on the cmd.exe prompt.
2024-03-03 03:12:19 -08:00
Justine Tunney
c8383f25b4
Release Cosmopolitan v3.3.2 2024-02-27 09:15:59 -08:00
Justine Tunney
64a9e6fe56
Fix compiler runtime for _Float16 type 2024-02-27 09:06:23 -08:00
Justine Tunney
0ef36489c8
Walk back most uses of __STRICT_ANSI__ 2024-02-27 04:09:49 -08:00
Justine Tunney
7c7bf4bb90
Release Cosmopolitan v3.3.1 2024-02-27 00:31:36 -08:00
Justine Tunney
592f6ebc20
Make quality improvements
- Write some more unit tests
- memcpy() on ARM is now faster
- Address the Musl complex math FIXME comments
- Some libm funcs like pow() now support setting errno
- Import the latest and greatest math functions from ARM
- Use more accurate atan2f() and log1pf() implementations
- atoi() and atol() will no longer saturate or clobber errno
2024-02-25 19:08:43 -08:00
Justine Tunney
38bceaeb8f
Give up on making clock_nanosleep() precise
Multiple projects I care about make the assumption that this isn't a
system call that sleeps for a particular number of nanonseconds, but
rather a function that parks processes on kernel scheduler quantums.
Anyone who wants the old behavior should use cosmo_clock_nanosleep()
2024-02-24 10:17:47 -08:00
Justine Tunney
e72a88ea70
Make fixups for libcrypt 2024-02-23 07:39:44 -08:00
Ivan Komarov
9b4701b3bc
Don't include <crypt.h> from <stdlib.h> (#1112) 2024-02-23 07:18:35 -08:00
Ivan Komarov
f7ff515961
*scanf() fixes to make TeX work (#1109)
* Fix reading the same symbol twice when using `{f,}scanf()`

PR #924 appears to use `unget()` subtly incorrectly when parsing
floating point numbers. The rest of the code only uses `unget()`
immediately followed by `goto Done;` to return back the symbol that
can't possibly belong to the directive we're processing.

With floating-point, however, the ungot characters could very well
be valid for the *next* directive, so we will essentially read them
twice. It can't be seen in `sscanf()` tests because `unget()` is a
no-op there, but the test I added for `fscanf()` fails like this:

        ...
        EXPECT_EQ(0xDEAD, i1)
                need 57005 (or 0xdead) =
                 got 908973 (or 0x000ddead)
        ...
        EXPECT_EQ(0xBEEF, i2)
                need 48879 (or 0xbeef) =
                 got 769775 (or 0x000bbeef)

This means we read 0xDDEAD instead of 0xDEAD and 0xBBEEF instead of
0xBEEF. I checked that both musl and glibc read 0xDEAD/0xBEEF, as
expected.

Fix the failing test by removing the unneeded `unget()` calls.

* Don't read invalid floating-point numbers in `*scanf()`

Currently, we just ignore any errors from `strtod()`. They can
happen either because no valid float can be parsed at all, or
because the state machine recognizes only a prefix of a valid
floating-point number.

Fix this by making sure `strtod()` parses everything we recognized,
provided it's non-empty. This requires to pop the last character
off the FP buffer, which is supposed to be parsed by the next
`*scanf()` directive.

* Make `%c` parsing in `*scanf()` respect the C standard

Currently, `%c`-style directives always succeed even if there
are actually fewer characters in the input than requested.

Before the fix, the added test fails like this:

        ...
        EXPECT_EQ(2, sscanf("ab", "%c %c %c", &c2, &c3, &c4))
                need 2 (or 0x02 or '\2' or ENOENT) =
                 got 3 (or 0x03 or '\3' or ESRCH)
        ...
        EXPECT_EQ(0, sscanf("abcd", "%5c", s2))
                need 0 (or 0x0 or '\0') =
                 got 1 (or 0x01 or '\1' or EPERM)

musl and glibc pass this test.
2024-02-23 07:15:30 -08:00
Justine Tunney
29eac8e2a2
Fix regression in logger functions
The recent release broke things so that the logger wasn't logging by
default. This ensures that `__log_file` initializes in correct order
2024-02-22 14:08:20 -08:00
Justine Tunney
e9a756e78b
Make --ftrace more crash proof with foreign code
Sometimes we need to interact with code that wasn't compiled using
`-fno-omit-frame-pointer`. For example, if a function pointer gets
passed and called by a foreign function, linked by cosmo_dlopen().
Function call tracing will now detect backtrace pointer corruption
and simply reduce the indentation level back to zero, as a result.
2024-02-22 11:06:31 -08:00
Justine Tunney
77a92f517b
Introduce getcpu() system call from glibc 2024-02-21 18:17:20 -08:00
Justine Tunney
3eb405e0e2
Resurrect <windows.h> as <windowsesque.h> 2024-02-21 16:41:11 -08:00
Justine Tunney
68b9479f0c
Make cosmo.h work a little better 2024-02-21 16:40:09 -08:00
Justine Tunney
957c61cbbf
Release Cosmopolitan v3.3
This change upgrades to GCC 12.3 and GNU binutils 2.42. The GNU linker
appears to have changed things so that only a single de-duplicated str
table is present in the binary, and it gets placed wherever the linker
wants, regardless of what the linker script says. To cope with that we
need to stop using .ident to embed licenses. As such, this change does
significant work to revamp how third party licenses are defined in the
codebase, using `.section .notice,"aR",@progbits`.

This new GCC 12.3 toolchain has support for GNU indirect functions. It
lets us support __target_clones__ for the first time. This is used for
optimizing the performance of libc string functions such as strlen and
friends so far on x86, by ensuring AVX systems favor a second codepath
that uses VEX encoding. It shaves some latency off certain operations.
It's a useful feature to have for scientific computing for the reasons
explained by the test/libcxx/openmp_test.cc example which compiles for
fifteen different microarchitectures. Thanks to the upgrades, it's now
also possible to use newer instruction sets, such as AVX512FP16, VNNI.

Cosmo now uses the %gs register on x86 by default for TLS. Doing it is
helpful for any program that links `cosmo_dlopen()`. Such programs had
to recompile their binaries at startup to change the TLS instructions.
That's not great, since it means every page in the executable needs to
be faulted. The work of rewriting TLS-related x86 opcodes, is moved to
fixupobj.com instead. This is great news for MacOS x86 users, since we
previously needed to morph the binary every time for that platform but
now that's no longer necessary. The only platforms where we need fixup
of TLS x86 opcodes at runtime are now Windows, OpenBSD, and NetBSD. On
Windows we morph TLS to point deeper into the TIB, based on a TlsAlloc
assignment, and on OpenBSD/NetBSD we morph %gs back into %fs since the
kernels do not allow us to specify a value for the %gs register.

OpenBSD users are now required to use APE Loader to run Cosmo binaries
and assimilation is no longer possible. OpenBSD kernel needs to change
to allow programs to specify a value for the %gs register, or it needs
to stop marking executable pages loaded by the kernel as mimmutable().

This release fixes __constructor__, .ctor, .init_array, and lastly the
.preinit_array so they behave the exact same way as glibc.

We no longer use hex constants to define math.h symbols like M_PI.
2024-02-20 13:27:59 -08:00
Justine Tunney
2ab9e9f7fd
Make improvements
- Introduce portable sched_getcpu() api
- Support GCC's __target_clones__ feature
- Make fma() go faster on x86 in default mode
- Remove some asan checks from core libraries
- WinMain() now ensures $HOME and $USER are defined
2024-02-12 10:23:00 -08:00
Ivan Komarov
d5225a693b
libc headers: make <complex.h> work, add struct ipv6_mreq (#1100)
1. `libc/isystem/complex.h` (included when you do `#include <complex.h>`)
   defines `_COMPLEX_H`, and then proceeds to include `libc/complex.h`,
   which contains the actual complex-related declarations. However, they
   are *also* guarded by `_COMPLEX_H` and hence effectively ignored.
   Fix this by changing `_COMPLEX_H` to `COSMOPOLITAN_LIBC_COMPLEX_H_`,
   which is consistent with what the other headers (such as `math.h`) do.
2. Cosmopolitan could only support IPv4 multicast requests for sockets,
   since a declaration for `struct ipv6_mreq` was missing. Add support
   for IPv6, too, by adding the missing declaration.
2024-02-05 17:22:56 -05:00
Justine Tunney
616717fa82
Fine tune OpenMP some more 2024-01-30 06:30:24 -08:00
Justine Tunney
369aebfc48
Make improvements
- Let OpenMP be usable via cosmocc
- Let libunwind be usable via cosmocc
- Make X86_HAVE(AVXVNNI) work correctly
- Avoid using MAP_GROWSDOWN on qemu-aarch64
- Introduce in6addr_any and in6addr_loopback
- Have thread stacks use MAP_GROWSDOWN by default
- Ask OpenMP to not use filesystem to manage threads
- Make NI_MAXHOST and NI_MAXSERV available w/o _GNU_SOURCE
2024-01-29 16:31:58 -08:00
Justine Tunney
5f8e9f14c1
Add OpenMP support 2024-01-28 22:39:02 -08:00
Justine Tunney
c1e18e7903
Restore MODE=dbg support
We recently broke MODE=dbg support when we added C++ exception support.
This change adds the missing UBSAN interfaces, needed to get it working
again. Some of the ASAN checking in the SJLJ guts needed to be disabled
since I doubt anyone's combined the two features until now.
2024-01-26 23:07:18 -08:00
Justine Tunney
f27808c4d2
Remove feature for embedding blink in ape scripts
Embedding Blink builds in Cosmo executables was a failed experiment. It
turned out to be easier than expected to let the mono repo have support
for multiple architectures. Blink still works great; it's supported and
recommended; just please use it as a separate program. For example, you
can use Blink to run Cosmo binaries on architectures like i486 / s390x.
2024-01-26 22:30:56 -08:00
Trung Nguyen
33418f6742
libc: Fix __cxa_thread_atexit prototype (#1088)
The first parameter should be a function pointer.

Otherwise, this would conflict with the definition provided by
`libcxxabi`.
2024-01-22 10:23:28 -08:00
Justine Tunney
8ab3a545c6
Increase build memory quota
If you install qemu-user from apt then glibc links a lot of address
space bloat that causes pthread_create() to ENOMEM (a.k.a. EAGAIN).
Boosting the virtual memory quota from 512m to 2048m will hopefully
future proof the build for the future, as Linux distros get fatter.
Please note this only applies to MODE=aarch64 on x86_64 builds when
you're using QEMU from Debian/Ubuntu rather than installing the one
cosmo provides in third_party/qemu/qemu-aarch64.gz. This change may
also be useful to people who are using the host compiler toolchain.
2024-01-22 10:02:30 -08:00
Justine Tunney
8ebe2e9020
Fix assertion error in pthread_create() 2024-01-22 10:02:30 -08:00
Justine Tunney
39b0a9c03e
Fix scanf() %n off by one at eof
Fixes #1094
2024-01-20 15:06:16 -08:00
Justine Tunney
1226eb7a5e
Fix gcc warning 2024-01-18 16:48:02 -08:00
Trung Nguyen
6db1200a7e
libc: Fix cxxabi.h header (#1087)
- `__cxa_*` runtime functions are expected to be in the `abi` namespace,
which is currently an alias for `__cxxabiv1`.
- Rely on the header provided by `libcxxabi` for functions that we do
not implement ourselves anymore.
2024-01-15 11:50:38 -08:00
Jōshin
6715b670b1
Skip the CAS on new_pos < 0 (#1080) 2024-01-12 21:08:28 -08:00
Justine Tunney
5d80e8dbf6
Add more Intel microarchitectures 2024-01-09 01:54:26 -08:00
Justine Tunney
eeb20775d2
Add dontthrow attribute to most libc functions
This will help C++ code that uses exceptions to be tinier. For example,
this change shaves away 1000 lines of assembly code from LLVM's libcxx,
which is 0.7% of all assembly instructions in the entire library.
2024-01-09 01:26:03 -08:00
Justine Tunney
cb19e172da
Release Cosmopolitan v3.2.4 2024-01-08 19:37:59 -08:00
Justine Tunney
6ab01716ce
Fix aarch64 setjmp abi
We now store values in jmp_buf where the compiler wants them to be. This
fixes code that calls __builtin_setjmp() and __builtin_longjmp() such as
libunwind. All libcxxabi tests are now passing on ARM64.

See #1076
2024-01-08 13:26:28 -08:00
Justine Tunney
81ce2e4cbc
Call thread finalizers on exit()
While we don't call POSIX thread key destructors from exit(), we do need
to call these, since C++ uses it for TLS object destructors.

See #1076
2024-01-08 11:45:02 -08:00
Justine Tunney
a4b455185b
Bring back gc() function
Renaming gc() to _gc() was a mistake since the better thing to do is put
it behind the _COSMO_SOURCE macro. We need this change because I haven't
wanted to use my amazing garbage collector ever since we renamed it. You
now need to define _COSMO_SOURCE yourself when using amalgamation header
and cosmocc users need to pass the -mcosmo flag to get the gc() function

Some other issues relating to cancelation have been fixed along the way.
We're also now putting cosmocc in a folder named `.cosmocc` so it can be
more safely excluded by grep --exclude-dir=.cosmocc --exclude-dir=o etc.
2024-01-08 10:26:28 -08:00
Trung Nguyen
8b33204f37
Add LLVM libcxxabi (#1063)
* third_party: Add libcxxabi

Added libcxxabi from LLVM 17.0.6
The library implements the Itanium C++ exception handling ABI.

* third_party/libcxxabi: Enable __cxa_thread_atexit

Enable `__cxa_thread_atexit` from libcxxabi.
`__cxa_thread_atexit_impl` is still implemented by the cosmo libc.
The original `__cxa_thread_atexit` has been removed.

* third_party/libcxx: Build with exceptions

Build libcxx with exceptions enabled.

- Removed `_LIBCPP_NO_EXCEPTIONS` from `__config`.
- Switched the exception implementation to `libcxxabi`. These two files
are taken from the same `libcxx` version as mentioned in `README.cosmo`.
- Removed `new_handler_fallback` in favor of `libcxxabi` implementation.
- Enable `-fexceptions` and `-frtti` for `libcxx`.
- Removed `THIRD_PARTY_LIBCXX` dependency from `libcxxabi` and
`libunwind`. These libraries do not use any runtime `libcxx` functions,
just headers.

* libc: Remove remaining redundant cxa functions

- `__cxa_pure_virtual` in `libcxxabi` is also a stub similar to the
existing one.
- `__cxa_guard_*` from `libcxxabi` is used instead of the ones from
Android.

Now there should be no more duplicate implementations.
`__cxa_thread_atexit_impl`, `__cxa_atexit`, and related supporting
functions, are still left to other libraries as in `libcxxabi`.

`libcxxabi` is also now added to `cosmopolitan.a` to make up for the
removed functions.

Affected in-tree libraries (`third_party/double-conversion`) have been
updated.
2024-01-08 08:45:10 -08:00
Trung Nguyen
94bab1618d
Add fixes to libunwind (#1069) 2024-01-08 08:31:13 -08:00
Justine Tunney
4cd02c29ed
Release Cosmopolitan v3.2.3 2024-01-07 12:39:09 -08:00
Justine Tunney
5ae2554c10
Have cosmo_dlopen() request -z execstack on Linux
The AMD HIP SDK for Linux ships prebuilt DSOs with an RWX PT_GNU_STACK
since old versions of GCC made it nearly impossible to build artifacts
where that wasn't the case, however modern glibc systems will flat out
refuse to link RWX DSOs from an execuatble that uses PT_GNU_STACK = RW
2024-01-07 11:37:18 -08:00
Justine Tunney
c4205f8305
Remove lingering dlopen thunk 2024-01-07 02:14:41 -08:00
Justine Tunney
b3fb6cff43
Add /dev/fd support to Windows
GNU bash needs this functionality, otherwise it can't do <(cmd...).
2024-01-06 18:20:21 -08:00
Justine Tunney
5e7137097d
Make breaking change to cosmo_dlsym()
The cosmo_dlsym() function now returns the raw function address. You
need to call cosmo_dltramp() on the result, to make it safe to call.
This change is important, because cosmo_dltramp() magic can't always
work; for some tricky functions, you need to translate ABIs by hand.
2024-01-06 15:45:26 -08:00
Justine Tunney
f224a55d57
Fix MODE=dbg build
Fixes #1067
2024-01-06 15:22:19 -08:00
Jōshin
636bc4007b
Enable argv[0] tests in more places (#1061)
Now we do them for assimilated binaries (except on OpenBSD or XNU
non-Silicon), for XnuSilicon, and for binaries with the preserve-
argv[0] auxv flag set. We check whether to pass the argv[0] value
at the test site rather than the Child site. We move a lot of the
test initialization into Child in the non-child case, in order to
get at the pre-init value of `__program_executable_name`. Finally,
we print out info about what we are skipping.
2024-01-06 11:42:03 -08:00
Justine Tunney
91de6f1f5d
Release Cosmopolitan 3.2.2 2024-01-05 21:00:22 -08:00
Justine Tunney
fad1279c61
Make cosmo_dlopen() safer and faster
If cosmo_dlopen() is linked on AMD64 then the runtime will switch to
using %gs for thread-local storage. This eliminates the need for the
imported symbol trampoline. It's now safer to pass function pointers
back and forth with imported libraries. Your program gets recompiled
at runtime to make it happen and the overhead is a few milliseconds.
2024-01-05 20:42:19 -08:00
Jōshin
c0eacf2eb1
clang-format 2024-01-05 23:21:52 -05:00
Justine Tunney
c60b150fcf
Release Cosmopolitan v3.2.1 2024-01-05 15:36:57 -08:00
Justine Tunney
2d93788ce3
Fix --ftrace with cosmo_dlopen()
This change ensures function call logging won't crash the process when
cosmo_dlopen() is called.
2024-01-05 15:13:07 -08:00
Justine Tunney
44a463e4d2
Invent *cosmo_dltramp() function 2024-01-05 09:11:51 -08:00
Justine Tunney
0de6a08988
Upgrade mono repo to cosmocc 3.2
The toolchain will now be downloaded going forward from multiple pinned
URLs which have shasums. Either wget or curl must be installed.

This change unblocks #1053
2024-01-05 08:02:04 -08:00
Jōshin
3315b6ef11
Store StrlenDevFd in n (#1056) 2024-01-04 23:17:37 -05:00
Justine Tunney
e93c8c4f8f
Fix typo 2024-01-04 13:59:36 -08:00
Justine Tunney
0e49bed660
Support 40 cosmo_dlopen() function parameters
Our dynamic linking implementation is now able to support functions with
dozens of parameters. In addition to having extra integral arguments you
can now pass vector registers using intrinsic types. Lastly, you can now
return multiple values, which is useful for functions returning structs.
2024-01-04 13:41:26 -08:00
Justine Tunney
a3deef70c2
Release Cosmopolitan v3.2 2024-01-04 09:39:48 -08:00
Justine Tunney
34ed1fcbea
Fix bugs with DNS library on Windows 2024-01-03 17:39:57 -08:00
Justine Tunney
1bb52c223b
Add missing build dependency 2024-01-03 17:04:21 -08:00
Jōshin
9801f2d1db
Remove COSMOPOLITAN_PROGRAM_EXECUTABLE (#1047) 2024-01-01 07:25:16 -08:00
Jōshin
68dbe5312f
Prepend getcwd to exename early in init (#1048) 2024-01-01 07:23:23 -08:00
Justine Tunney
2f89c2482a
Delete some dead code 2024-01-01 00:13:16 -08:00
Justine Tunney
9e6faa5256
Fix --ftrace on Windows 2024-01-01 00:00:42 -08:00
Jōshin
10b4693e37
Sanity-check set-id interpreter script paths (#1029) 2023-12-31 07:46:27 -08:00
Jōshin
14fe83facd
aarch64 loader passes os (#1042)
* Reorder Launch arguments, pass aarch64 os

Third and fourth arguments are now identical between cosmo and Launch.
By passing sp as argument 4, we save a bit of register juggling.

Fourth argument (os) is now always passed by the loader on aarch64. It
is not yet processed by cosmo. Pushing this change separately, as the
cosmo side turns out to be somewhat more involved.

* cosmo2 receives os from loader

FreeBSD aarch64 now traps early rather than pretending to be Linux.
o/aarch64/examples/env.com still works on Linux and Xnu.
2023-12-31 06:42:36 -08:00
Justine Tunney
d8ad34686a
Implement issetugid() on NetBSD 2023-12-30 14:58:16 -08:00
Justine Tunney
83107f78ed
Introduce FreeBSD ARM64 support
It's 100% passing test fleet. Solid as a rock.
2023-12-29 20:14:02 -08:00
Justine Tunney
43fe5956ad
Use DNS implementation from Musl Libc
Now that our socket system call polyfills are good enough to support
Musl's DNS library we should be using that rather than the barebones
domain name system implementation we rolled on our own. There's many
benefits to making this change. So many, that I myself wouldn't feel
qualified to enumerate them all. The Musl DNS code had to be changed
in order to support Windows of course, which looks very solid so far
2023-12-28 23:04:35 -08:00
Justine Tunney
ce0143e2a1
Fix madvise() on Windows 2023-12-27 22:41:46 -08:00
Justine Tunney
f51fd97644
Fix MODE=dbg build break 2023-12-27 22:24:27 -08:00
Jōshin
5c35863524
Rename __zipos_free -> __zipos_drop (#1043)
Removes the separate decref function, uses keep/drop in the internal
API.
2023-12-26 12:08:57 -05:00
Jōshin
25266b037b
Save a redundant load in zipos read/seek (#1037)
When h->pos has changed to something other than SIZE_MAX, we don't need
the extra atomic load.
2023-12-24 19:40:18 -05:00
Jōshin
7faffde303
Fix buffer overflow in TryPath (#1035)
Missed this when changing the code back to be like the old version.
com is now a parameter.

The only plausible way to trigger this would be to pass a loader
pathname close to MAX_PATH characters long, and then remove that
path prior to the first sys_faccessat.
2023-12-18 23:02:16 -05:00
Jōshin
1280797db2
Fix indentation, make llvm happy (#1032)
clang says "label at end of switch statement is a C2x extension."
2023-12-18 20:48:33 -05:00
Jōshin
2a11a09d98
Remove realpath/getcwd from loaders (#1024)
This implements proposals 1 and 2a from this gist:

https://gist.github.com/mrdomino/2222cab61715fd527e82e036ba4156b1

The only reason to use realpath from the loader was to try to prevent a
TOCTOU between the loader and the binary. But this is only a real issue
in set-id contexts, and in those cases there is already a canonical way
to do it: `/dev/fd`, passed by the kernel to the loader, so all we have
to do is pass that along to the binary.

Aside from realpath, there is no reason to absolutize the path we supply
to the binary, since it can call `getcwd` as well as we can, and on non-
M1 the binary is in a much better position to make that call.

Since we no longer absolutize the path, the binary does need to do this,
so we make its argv-parsing code generic and apply that to the different
possible places the path could come from. This means that `_` is finally
usable as a relative path, as a nice side benefit.

The M1 realpath code had a significant bug - it uses the wrong offset to
truncate the `.ape` in the `$prog.ape` case.

This PR also fixes a regression in `ape $progname` out of `$PATH` on the
two BSDs (Free and Net) that did not implement `RealPath`.
2023-12-18 15:01:16 -05:00
Jōshin
77facc4f7b
Cover more cases for detecting ape loader (#1022)
Fixes a regression in GetProgramExecutableName on Linux against old
loaders. In the loader case, /proc/self/exe gives the loader's path.
We tried to detect this by checking for `/usr/bin/ape`. But that is
only one of the possible places the loader could be.
2023-12-17 10:30:22 -05:00
Jōshin
2b315626f3
Revert retabbing of net/http and tinymath (#1020) 2023-12-16 23:59:11 -05:00
Jōshin
3a8e01a77a
more modeline errata (#1019)
Somehow or another, I previously had missed `BUILD.mk` files.

In the process I found a few straggler cases where the modeline was
different from the file, including one very involved manual fix where a
file had been treated like it was ts=2 and ts=8 on separate occasions.

The commit history in the PR shows the gory details; the BUILD.mk was
automated, everything else was mostly manual.
2023-12-16 23:07:10 -05:00
Jōshin
f94c11d978
Loader path security (#1012)
The ape loader now passes the program executable name directly as a
register. `x2` is used on aarch64, `%rdx` on x86_64. This is passed
as the third argument to `cosmo()` (M1) or `Launch` (non-M1) and is
assigned to the global `__program_executable_name`.

`GetProgramExecutableName` now returns this global's value, setting
it if it is initially null. `InitProgramExecutableName` first tries
exotic, secure methods: `KERN_PROC_PATHNAME` on FreeBSD/NetBSD, and
`/proc` on Linux. If those produce a reasonable response (i.e., not
`"/usr/bin/ape"`, which happens with the loader before this change),
that is used. Otherwise, if `issetugid()`, the empty string is used.
Otherwise, the old argv/envp parsing code is run.

The value returned from the loader is always the full absolute path
of the binary to be executed, having passed through `realpath`. For
the non-M1 loader, this necessitated writing `RealPath`, which uses
`readlinkat` of `"/proc/self/fd/[progfd]"` on Linux, `F_GETPATH` on
Xnu, and the `__realpath` syscall on OpenBSD. On FreeBSD/NetBSD, it
punts to `GetProgramExecutableName`, which is secure on those OSes.

With the loader, all platforms now have a secure program executable
name. With no loader or an old loader, everything still works as it
did, but setuid/setgid is not supported if the insecure pathfinding
code would have been needed.

Fixes #991.
2023-12-15 12:23:58 -05:00
Jōshin
8a10ccf9c4
Fix ZipOS deadlock/segfault (#1011)
This change adds a new stress test for ZipOS which helped
us improve the locking semantics in open() and close().
2023-12-14 19:59:20 -08:00
Jōshin
7a05fb43ac
Assume __zipos_close when closing a zipos fd 2023-12-13 10:07:32 -05:00
Jōshin
3b302e6379
Cleanup zipos vfork (#1004) 2023-12-13 01:36:44 -08:00
Jōshin
6cedbc746c
Fix type of refs 2023-12-13 03:03:25 -05:00