Commit graph

206 commits

Author SHA1 Message Date
Justine Tunney
ee6566a152
Further improve usability of cosmocc
- Support -s flag in cosmocc
- Support posix_spawn() setsid() feature
- Disable monorepo debug path prefix stripping
2023-07-10 05:55:00 -07:00
Justine Tunney
f7ae50462a
Make improvements
- Fix unused local variable errors
- Remove yoinks from sigaction() header
- Add nox87 and aarch64 to github actions
- Fix cosmocc -fportcosmo in linking mode
- It's now possible to build `make m=llvm o/llvm/libc`
2023-07-10 04:35:14 -07:00
Justine Tunney
41396ff48a
Make fixes and improvements
- Fix handling of precision in hex float formatting
- Enhance the cocmd interpreter for system() and popen()
- Manually ran the Lua unit tests, which are now passing
- Let stdio i/o operations happen when file is in error state
- We're now saving and restoring xmm in ftrace out of paranoia
2023-07-09 05:21:11 -07:00
Justine Tunney
3a8579252d
Move demangling utility to stdio
Closes #848
2023-07-07 10:47:50 -07:00
Justine Tunney
0d3c1c8b1a
Do work on curl/mbedtls/zstd
This change fixes stderr to be unbuffered. Added hardware AES on ARM64
to help safeguard against timing attacks. The curl.com command will be
somewhat more pleasant to use.
2023-07-07 10:13:35 -07:00
Justine Tunney
00acd81b2f
Delete more dead code 2023-07-06 09:12:28 -07:00
Justine Tunney
73c0faa1b5
Remove some dead code 2023-07-03 02:48:29 -07:00
Justine Tunney
0c630d95b5
Rewrite Cosmopolitan Ar
The build/bootstrap/ar.com program is now tinier. This change reduces
its size from 140kb to 53kb. Nothing was traded away. Cosmopolitan Ar
performance is now 2x better than llvm-ar largely thanks to using the
copy_file_range() system call. This change homebrews a new allocation
API that addresses the shortcomings of the C standard library design.
Using these new balloc() and reballoc() functions I managed to reduce
memory consumption so much that Cosmpolitan Ar should now use roughly
100x fewer bytes of peak resident memory compared to llvm-ar. Correct
behavior with better compatibility has been assured. Binary output is
now pretty much bit-identical to llvm-ar, as of this change. This can
and should be the living proof we need to show that a better world is
possible for software.
2023-07-02 10:19:16 -07:00
Justine Tunney
d7c79f43ef
Clean up more code
- Found some bugs in LLVM compiler-rt library
- The useless LIBC_STUBS package is now deleted
- Improve the overflow checking story even further
- Get chibicc tests working in MODE=dbg mode again
- The libc/isystem/ headers now have correctly named guards
2023-06-18 01:00:05 -07:00
Justine Tunney
b881c0ec9e
Remove printf() linking hack 2023-06-17 10:13:50 -07:00
Justine Tunney
4778cd4d27
Fix bugs in termios library and cleanup code
This change fixes an issue with the tcflow() magic numbers that was
causing bash to freeze up on Linux. While auditing termios polyfills,
several other issues were identified with XNU/BSD compatibility.

Out of an abundance of caution this change undefines as much surface
area from libc/calls/struct/termios.h as possible, so that autoconf
scripts are less likely to detect non-POSIX teletypewriter APIs that
haven't been polyfilled by Cosmopolitan.

This is a *breaking change* for your static archives in /opt/cosmos if
you use the cosmocc toolchain. That's because this change disables the
ioctl() undiamonding trick for code outside the monorepo, specifically
because it'll lead to brittle ABI breakages like this. If you're using
the cosmocc toolchain, you'll need to rebuild libraries like ncurses,
readline, etc. Yes diamonds cause bloat. To work around that, consider
using tcgetwinsize() instead of ioctl(TIOCGWINSZ) since it'll help you
avoid pulling every single ioctl-related polyfill into the linkage.

The cosmocc script was specifying -DNDEBUG for some reason. It's fixed.
2023-06-14 19:30:52 -07:00
Gabriel Ravier
50d8d953ce
Fix syscall(2) returning -errno instead of using POSIX errno scheme (#830)
As of now, the syscall function is implemented as alike to how the
linux kernel sycall ABI works, returning -errno upon errors without
setting the value of errno. However, this does not conform to the
expectations of most software, which expect it to return -1 and set
errno, which is how it works on other libcs, which document it
accordingly:

> The return value is defined by the system call being invoked.  In
> general, a 0 return value indicates success.  A -1 return value
> indicates an error, and an error number is stored in errno.
- Linux man-pages, syscall(2)

> The return value is the return value from the system call, unless
> the system call failed.  In that case, ‘syscall’ returns ‘-1’ and
> sets ‘errno’ to an error code that the system call returned.
- glibc manual, (libc)System Calls

> When the C-bit is set, syscall returns -1 and sets the external
> variable errno (see intro(2)).
- 4BSD manual, syscall(2)

> A -1 return value indicates an error, and an error code is stored in
> errno.
- 4.4BSD, FreeBSD, OpenBSD and NetBSD manuals (same quote is found in
  all of them), syscall(2)

This patch corrects the syscall function to work in the same way as in
other libcs.
2023-06-11 10:33:28 -07:00
Justine Tunney
8ff48201ca
Rewrite .zip.o file linker
This change takes an entirely new approach to the incremental linking of
pkzip executables. The assets created by zipobj.com are now treated like
debug data. After a .com.dbg is compiled, fixupobj.com should be run, so
it can apply fixups to the offsets and move the zip directory to the end
of the file. Since debug data doesn't get objcopy'd, a new tool has been
introduced called zipcopy.com which should be run after objcopy whenever
a .com file is created. This is all automated by the `cosmocc` toolchain
which is rapidly becoming the new recommended approach.

This change also introduces the new C23 checked arithmetic macros.
2023-06-10 09:29:44 -07:00
Justine Tunney
4a59210008
Introduce #include <cosmo.h> to toolchain users
This change improves the way internal APIs are being hidden behind the
`COSMO` define. The cosmo.h header will take care of defining that, so
that a separate define statement isn't needed. This change also does a
lot more to define which APIs are standard, and which belong to Cosmo.
2023-06-09 18:03:05 -07:00
Justine Tunney
9b55dbe417
Get GCC to mostly build with Cosmo 2023-06-09 06:41:34 -07:00
Justine Tunney
2eb7148474
Create a fake syscall() function due to Python 2023-06-09 05:18:38 -07:00
Justine Tunney
23e235b7a5
Fix bugs in cosmocc toolchain
This change integrates e58abc1110b335a3341e8ad5821ad8e3880d9bb2 from
https://github.com/ahgamut/musl-cross-make/ which fixes the issues we
were having with our C language extension for symbolic constants. This
change also performs some code cleanup and bug fixes to getaddrinfo().
It's now possible to compile projects like ncurses, readline and python
without needing to patch anything upstream, except maybe a line or two.
Pretty soon it should be possible to build a Linux distro on Cosmo.
2023-06-08 23:44:03 -07:00
Justine Tunney
daf4454a06
Validate privileged code relationships
- Work towards improving non-optimized build support
- Introduce MODE=zero which is -O0 without ASAN/UBSAN
- Use system GCC when ~/.cosmo.mk has USE_SYSTEM_TOOLCHAIN=1
- Have package.com check .privileged code doesn't call non-privileged
2023-06-08 04:38:06 -07:00
Justine Tunney
b8a6a989c0
Create ELF aliases for identical symbols
This change greatly reduces the number of modules that need to be
compiled. The only issue right now is that sometimes when viewing
symbol table entries, the aliased symbol is chosen.
2023-06-06 03:33:49 -07:00
Justine Tunney
eb40cb371d
Get --ftrace working on aarch64
This change implements a new approach to function call logging, that's
based on the GCC flag: -fpatchable-function-entry. Read the commentary
in build/config.mk to learn how it works.
2023-06-05 23:35:31 -07:00
Justine Tunney
35dcaca53c
Fix test fleet errors 2023-06-04 10:57:11 -07:00
Justine Tunney
8f522cb702
Make improvements
This change progresses our AARCH64 support:

- The AARCH64 build and tests are now passing
- Add 128-bit floating-point support to printf()
- Fix clone() so it initializes cosmo's x28 TLS register
- Fix TLS memory layout issue with aarch64 _Alignas vars
- Revamp microbenchmarking tools so they work on aarch64
- Make some subtle improvements to aarch64 crash reporting
- Make kisdangerous() memory checks more accurate on aarch64
- Remove sys_open() since it's not available on Linux AARCH64

This change makes general improvements to Cosmo and Redbean:

- Introduce GetHostIsa() function in Redbean
- You can now feature check using pledge(0, 0)
- You can now feature check using unveil("",0)
- Refactor some more x86-specific asm comments
- Refactor and write docs for some libm functions
- Make the mmap() API behave more similar to Linux
- Fix WIFSIGNALED() which wrongly returned true for zero
- Rename some obscure cosmo keywords from noFOO to dontFOO
2023-06-03 08:12:22 -07:00
Justine Tunney
290a49952e
Fix some more issues with aarch64 and llama.cpp 2023-05-10 07:34:26 -07:00
Justine Tunney
d04430f4ef
Get LIBC_MEM and LIBC_STDIO building with aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney
ca2860947f
Make progress towards aarch64 build 2023-05-10 04:20:46 -07:00
Justine Tunney
420f889ac3
Further optimize the math library
The sincosf() function is now twice as fast, thanks to ARM Limited. The
same might also be true of logf() and expm1f() which have been updated.
2023-04-28 01:20:47 -07:00
Justine Tunney
369f9740de
Run clang-format on most sources 2023-04-27 05:44:32 -07:00
Gabriel Ravier
9c6d78c26d
Fix vfprintf and derived functions not handling write errors (#798)
The C standard states:
> The fprintf function returns the number of characters transmitted,
> or a negative value if an output or encoding error occurred or if
> the implementation does not support a specified width length
> modifier.
- C Standard, 7.23.6.1.15. The fprintf function

However, cosmopolitan fails to return a negative value in the case of
an output error, meaning that a program such as:

 #include <stdio.h>

int main()
{
    FILE *fp = fopen("/dev/full", "w");

    setbuf(fp, NULL);
    printf("fprintf: %d\n", fprintf(fp, "test\n"));
    printf("fflush: %d\n", fflush(fp));
}

will, under cosmopolitan, print that no error occured in either of the
calls to fprintf and fflush.

This patch fixes this, along with the associated GitHub issue,
https://github.com/jart/cosmopolitan/issues/784
2023-03-31 09:57:29 -04:00
Justine Tunney
be3e109309
Fix build break 2023-01-08 15:35:08 -08:00
Justine Tunney
2bad436abf
Return error on fopen(NULL)
https://sqlite.org/forum/forumpost/d1c96a9032e564f8
2023-01-08 15:17:44 -08:00
Justine Tunney
ed161b240e
Clean up some code 2022-12-11 14:30:59 -08:00
Justine Tunney
f2af97711b
Make improvements
- Improve compatibility with Blink virtual machine
- Add non-POSIX APIs for joining threads and signal masks
- Never ever use anything except 32-bit integers for atomics
- Add some `#undef` statements to workaround `ctags` problems
2022-11-10 21:52:47 -08:00
Justine Tunney
bf7843833f
Rename hidden keyword to _Hide 2022-11-08 12:55:28 -08:00
Justine Tunney
3f0bcdc3ef
Improve cancellations, randomness, and time
- Exhaustively document cancellation points
- Rename SIGCANCEL to SIGTHR just like BSDs
- Further improve POSIX thread cancellations
- Ensure asynchronous cancellations work correctly
- Elevate the quality of getrandom() and getentropy()
- Make futexes cancel correctly on OpenBSD 6.x and 7.x
- Add reboot.com and shutdown.com to examples directory
- Remove underscore prefix from awesome timespec_*() APIs
- Create assertions that help verify our cancellation points
- Remove bad timespec APIs (cmp generalizes eq/ne/gt/gte/lt/lte)
2022-11-05 23:45:32 -07:00
Justine Tunney
0d7c265392
Fix backtraces on cancellation points 2022-11-04 20:22:13 -07:00
Justine Tunney
022536cab6
Make futexes cancellable by pthreads 2022-11-04 18:36:34 -07:00
Justine Tunney
2278327eba
Implement support for POSIX thread cancellations
This change makes some miracle modifications to the System Five system
call support, which lets us have safe, correct, and atomic handling of
thread cancellations. It all turned out to be cheaper than anticipated
because it wasn't necessary to modify the system call veneers. We were
able to encode the cancellability of each system call into the magnums
found in libc/sysv/syscalls.sh. Since cancellations are so waq, we are
also supporting a lovely Musl Libc mask feature for raising ECANCELED.
2022-11-04 01:04:43 -07:00
Justine Tunney
37d40e087f
Ignore SIGSYS on BSD by default 2022-11-03 09:32:12 -07:00
Justine Tunney
179e048bba
Make some small fixes to recent changes 2022-11-03 05:45:33 -07:00
Justine Tunney
89d1e5b8f2
Rearrange code and make a faster sha256sum program 2022-11-02 23:19:16 -07:00
Justine Tunney
e522aa3a07
Make more threading improvements
- ASAN memory morgue is now lockless
- Make C11 atomics header more portable
- Rewrote pthread keys support to be lockless
- Simplify Python's unicode table unpacking code
- Make crash report write(2) closer to being atomic
- Make it possible to strace/ftrace a single thread
- ASAN now checks nul-terminated strings fast and properly
- Windows fork() now restores TLS memory of calling thread
2022-11-01 23:28:26 -07:00
Justine Tunney
f7ff77d865
Make fixes and improvements
- Invent iso8601us() for faster timestamps
- Improve --strace descriptions of sigset_t
- Rebuild the Landlock Make bootstrap binary
- Introduce MODE=sysv for non-Windows builds
- Permit OFD fcntl() locks under pledge(flock)
- redbean can now protect your kernel from ddos
- Have vfork() fallback to sys_fork() not fork()
- Change kmalloc() to not die when out of memory
- Improve documentation for some termios functions
- Rewrite putenv() and friends to conform to POSIX
- Fix linenoise + strace verbosity issue on Windows
- Fix regressions in our ability to show backtraces
- Change redbean SetHeader() to no-op if value is nil
- Improve fcntl() so SQLite locks work in non-WAL mode
- Remove some unnecessary work during fork() on Windows
- Create redbean-based SSL reverse proxy for IPv4 TurfWar
- Fix ape/apeinstall.sh warning when using non-bash shells
- Add ProgramTrustedIp(), and IsTrustedIp() APIs to redbean
- Support $PWD, $UID, $GID, and $EUID in command interpreter
- Introduce experimental JTqFpD APE prefix for non-Windows builds
- Invent blackhole daemon for firewalling IP addresses via UNIX named socket
- Add ProgramTokenBucket(), AcquireToken(), and CountTokens() APIs to redbean
2022-10-19 07:19:19 -07:00
Justine Tunney
60cb435cb4
Implement pthread_atfork()
If threads are being used, then fork() will now acquire and release and
runtime locks so that fork() may be safely used from threads. This also
makes vfork() thread safe, because pthread mutexes will do nothing when
the process is a child of vfork(). More torture tests have been written
to confirm this all works like a charm. Additionally:

- Invent hexpcpy() api
- Rename nsync_malloc_() to kmalloc()
- Complete posix named semaphore implementation
- Make pthread_create() asynchronous signal safe
- Add rm, rmdir, and touch to command interpreter builtins
- Invent sigisprecious() and modify sigset functions to use it
- Add unit tests for posix_spawn() attributes and fix its bugs

One unresolved problem is the reclaiming of *NSYNC waiter memory in the
forked child processes, within apps which have threads waiting on locks
2022-10-16 12:25:13 -07:00
Justine Tunney
5af19b7eed
Make some foss compatibility improvements 2022-10-14 13:59:34 -07:00
Justine Tunney
f52f65b2e3
Make system() and popen() thread safe 2022-10-13 15:54:05 -07:00
Justine Tunney
0f89140882
Further improve cocmd interpreter 2022-10-12 10:44:54 -07:00
Justine Tunney
0cee831da3
Revamp posix_spawn() implementation 2022-10-12 05:26:58 -07:00
Justine Tunney
b41f91c658
Greatly expand system() shell code features
The cosmopolitan command interpreter now has 13 builtin commands,
variable support, support for ; / && / || syntax, asynchronous support,
and plenty of unit tests with bug fixes.

This change fixes a bug in posix_spawn() with null envp arg. strace
logging now uses atomic writes for scatter functions. Breaking change
renaming GetCpuCount() to _getcpucount(). TurfWar is now updated to use
the new token bucket algorithm. WIN32 affinity masks now inherit across
fork() and execve().
2022-10-11 21:30:31 -07:00
Justine Tunney
e557058ac8
Improve cosmo's conformance to libc-test
This change addresses various open source compatibility issues, so that
we pass 313/411 of the tests in https://github.com/jart/libc-test where
earlier today we were passing about 30/411 of them, due to header toil.
Please note that Glibc only passes 341/411 so 313 today is pretty good!

- Make the conformance of libc/isystem/ headers nearly perfect
- Import more of the remaining math library routines from Musl
- Fix inconsistencies with type signatures of calls like umask
- Write tests for getpriority/setpriority which work great now
- conform to `struct sockaddr *` on remaining socket functions
- Import a bunch of uninteresting stdlib functions e.g. rand48
- Introduce readdir_r, scandir, pthread_kill, sigsetjmp, etc..

Follow the instructions in our `tool/scripts/cosmocc` toolchain to run
these tests yourself. You use `make CC=cosmocc` on the test repository
2022-10-10 17:52:41 -07:00
Justine Tunney
7ae556463a
Rename rand64() to _rand64() 2022-10-10 04:12:06 -07:00