Commit graph

324 commits

Author SHA1 Message Date
Justine Tunney
4778cd4d27
Fix bugs in termios library and cleanup code
This change fixes an issue with the tcflow() magic numbers that was
causing bash to freeze up on Linux. While auditing termios polyfills,
several other issues were identified with XNU/BSD compatibility.

Out of an abundance of caution this change undefines as much surface
area from libc/calls/struct/termios.h as possible, so that autoconf
scripts are less likely to detect non-POSIX teletypewriter APIs that
haven't been polyfilled by Cosmopolitan.

This is a *breaking change* for your static archives in /opt/cosmos if
you use the cosmocc toolchain. That's because this change disables the
ioctl() undiamonding trick for code outside the monorepo, specifically
because it'll lead to brittle ABI breakages like this. If you're using
the cosmocc toolchain, you'll need to rebuild libraries like ncurses,
readline, etc. Yes diamonds cause bloat. To work around that, consider
using tcgetwinsize() instead of ioctl(TIOCGWINSZ) since it'll help you
avoid pulling every single ioctl-related polyfill into the linkage.

The cosmocc script was specifying -DNDEBUG for some reason. It's fixed.
2023-06-14 19:30:52 -07:00
tkchia
e47c0cc929
[metal] Clean up code for getting pointer to struct mman (#828)
Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2023-06-11 10:32:39 -07:00
Justine Tunney
2676ec55de
Make improvements to cosmocc toolchain 2023-06-10 15:50:01 -07:00
Justine Tunney
8ff48201ca
Rewrite .zip.o file linker
This change takes an entirely new approach to the incremental linking of
pkzip executables. The assets created by zipobj.com are now treated like
debug data. After a .com.dbg is compiled, fixupobj.com should be run, so
it can apply fixups to the offsets and move the zip directory to the end
of the file. Since debug data doesn't get objcopy'd, a new tool has been
introduced called zipcopy.com which should be run after objcopy whenever
a .com file is created. This is all automated by the `cosmocc` toolchain
which is rapidly becoming the new recommended approach.

This change also introduces the new C23 checked arithmetic macros.
2023-06-10 09:29:44 -07:00
Justine Tunney
4a59210008
Introduce #include <cosmo.h> to toolchain users
This change improves the way internal APIs are being hidden behind the
`COSMO` define. The cosmo.h header will take care of defining that, so
that a separate define statement isn't needed. This change also does a
lot more to define which APIs are standard, and which belong to Cosmo.
2023-06-09 18:03:05 -07:00
Justine Tunney
9b55dbe417
Get GCC to mostly build with Cosmo 2023-06-09 06:41:34 -07:00
Justine Tunney
4b2023ffab
Disable linker map generation and improve tinyness 2023-06-09 03:29:26 -07:00
Justine Tunney
23e235b7a5
Fix bugs in cosmocc toolchain
This change integrates e58abc1110b335a3341e8ad5821ad8e3880d9bb2 from
https://github.com/ahgamut/musl-cross-make/ which fixes the issues we
were having with our C language extension for symbolic constants. This
change also performs some code cleanup and bug fixes to getaddrinfo().
It's now possible to compile projects like ncurses, readline and python
without needing to patch anything upstream, except maybe a line or two.
Pretty soon it should be possible to build a Linux distro on Cosmo.
2023-06-08 23:44:03 -07:00
Justine Tunney
25678db2a0
Fix remaining regressions in ASAN support
See #829
2023-06-08 08:00:50 -07:00
Justine Tunney
4d629fd424
Fix stack abuse in llama.cc
This change also incorporates improvements for MODE=asan. It's been
confirmed that o/asan/third_party/ggml/llama.com will work.

Fixes #829
2023-06-08 07:12:26 -07:00
Justine Tunney
32682f0ce7
Remove some problematic APIs
In order to improve our chances of success building other open source
projects we shouldn't define APIs that'll lead any ./configure script
astray. For example:

- brk() and sbrk() can break mac/windows support
- syscall() is a superb way to break portability
- arch_prctl() is the greatest of all horror shows
2023-06-08 06:12:26 -07:00
Justine Tunney
daf4454a06
Validate privileged code relationships
- Work towards improving non-optimized build support
- Introduce MODE=zero which is -O0 without ASAN/UBSAN
- Use system GCC when ~/.cosmo.mk has USE_SYSTEM_TOOLCHAIN=1
- Have package.com check .privileged code doesn't call non-privileged
2023-06-08 04:38:06 -07:00
Justine Tunney
01fd655097
Get garbage collector working on aarch64
Garbage collection will now happen on arm64 when a function returns,
rather than kicking the can down the road to when the process exits.
This change also does some code cleanup and incorporates suggestions
2023-06-07 03:34:45 -07:00
Justine Tunney
b8a6a989c0
Create ELF aliases for identical symbols
This change greatly reduces the number of modules that need to be
compiled. The only issue right now is that sometimes when viewing
symbol table entries, the aliased symbol is chosen.
2023-06-06 03:33:49 -07:00
Justine Tunney
e1b83399bd
Fix memmove() alias on aarch64 with --ftrace 2023-06-06 00:39:55 -07:00
Justine Tunney
eb40cb371d
Get --ftrace working on aarch64
This change implements a new approach to function call logging, that's
based on the GCC flag: -fpatchable-function-entry. Read the commentary
in build/config.mk to learn how it works.
2023-06-05 23:35:31 -07:00
Justine Tunney
7558549d44
Test m=aarch64 on GitHub Actions 2023-06-05 15:23:00 -07:00
Justine Tunney
42b9b75749
Fix aarch64 build 2023-06-05 05:57:44 -07:00
Justine Tunney
f554dd800e
Make it possible to log kprintf() to file
It's now possible to compile Emacs using cosmocc. However we need to
troubleshoot why it's event loop isn't working correctly at runtime.
2023-06-05 04:16:15 -07:00
Justine Tunney
4aa1d09b9e
Improve aarch64 native support some more
This change introduces partial support for automating remote testing of
aarch64 binaries on Raspberry Pi and Apple Silicon.
2023-06-04 08:58:47 -07:00
Justine Tunney
bcf9af94bf
Get threads working well on MacOS Arm64
- Now using 10x better GCD semaphores
- We now generate Linux-like thread ids
- We now use fast system clock / sleep libraries
- The APE M1 loader now generates Linux-like stacks
2023-06-04 01:57:10 -07:00
Justine Tunney
b5eab2b0b7
Get POSIX threads working on Apple Silicon
It's now possible to run a working

    ape-m1 o/aarch64/third_party/ggml/llama.com

on Apple M1 hardware running XNU!
2023-06-03 18:33:01 -07:00
Justine Tunney
6ae18a10ba
Fix MODE=tiny build 2023-06-03 10:30:48 -07:00
Justine Tunney
8f522cb702
Make improvements
This change progresses our AARCH64 support:

- The AARCH64 build and tests are now passing
- Add 128-bit floating-point support to printf()
- Fix clone() so it initializes cosmo's x28 TLS register
- Fix TLS memory layout issue with aarch64 _Alignas vars
- Revamp microbenchmarking tools so they work on aarch64
- Make some subtle improvements to aarch64 crash reporting
- Make kisdangerous() memory checks more accurate on aarch64
- Remove sys_open() since it's not available on Linux AARCH64

This change makes general improvements to Cosmo and Redbean:

- Introduce GetHostIsa() function in Redbean
- You can now feature check using pledge(0, 0)
- You can now feature check using unveil("",0)
- Refactor some more x86-specific asm comments
- Refactor and write docs for some libm functions
- Make the mmap() API behave more similar to Linux
- Fix WIFSIGNALED() which wrongly returned true for zero
- Rename some obscure cosmo keywords from noFOO to dontFOO
2023-06-03 08:12:22 -07:00
Paul Kulchenko
5655c9a4e7
Extend Pledge with anet (same as inet, but with no connect) (#827)
* Add `anet` pledge for `inet` without connect

This is useful for configurations where it's desirable to start redbean
under these restrictions, but not to allow `connect` socket calls.

* Update message on protected/unpledged syscalls for clarity

* Update redbean to add reporting for unpledged sigaction

Previously it would abort without indicating what signal it failed to
install when sigaction is not pledged (although it fails all of them).

* Move GetHostIps before processing command line options

This allows using unix.pledge as part of the options without affecting
retrieving host IP addresses (which requires `connect`). It may still
fail under external `pledge` command as expected; in this case IPs
would need to be passed manually.

* Update tests for pledge anet promise
2023-06-03 07:50:29 -07:00
Justine Tunney
72f8bd10b7
Perform some code cleanup 2023-05-28 19:42:00 -07:00
tkchia
90645f4dac
[metal] Fix some issues that were breaking UEFI boot (#822)
Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2023-05-26 18:29:25 -07:00
Justine Tunney
1422e96b4e
Introduce native support for MacOS ARM64
There's a new program named ape/ape-m1.c which will be used to build an
embeddable binary that can load ape and elf executables. The support is
mostly working so far, but still chasing down ABI issues.
2023-05-20 04:17:03 -07:00
Justine Tunney
e7eb0b3070
Make more ML improvements
- Fix UX issues with llama.com
- Do housekeeping on libm code
- Add more vectorization to GGML
- Get GGJT quantizer programs working well
- Have the quantizer keep the output layer as f16c
- Prefetching improves performance 15% if you use fewer threads
2023-05-16 08:07:23 -07:00
Justine Tunney
210187cf77
Perform some code cleanup 2023-05-15 16:32:10 -07:00
Justine Tunney
cc1732bc42
Make AARCH64 harder, better, faster, stronger
- Perform some housekeeping on scalar math function code
- Import ARM's Optimized Routines for SIMD string processing
- Upgrade to latest Chromium zlib and enable more SIMD optimizations
2023-05-15 02:15:34 -07:00
Justine Tunney
550b52abf6
Port a lot more code to AARCH64
- Introduce epoll_pwait()
- Rewrite -ftrapv and ffs() libraries in C code
- Use more FreeBSD code in math function library
- Get significantly more tests passing on qemu-aarch64
- Fix many Musl long double functions that were broken on AARCH64
2023-05-14 09:37:26 -07:00
Justine Tunney
ba49e86e20
Get TEST_LIBC_CALLS passing on AARCH64 2023-05-13 02:41:41 -07:00
Justine Tunney
4a8a81eb9f
Fix llama.com interactive mode regressions 2023-05-13 00:09:38 -07:00
Justine Tunney
fd34ef732d
Make considerably more progress on AARCH64
- Utilities like pledge.com now build
- kprintf() will no longer balk at 48-bit addresses
- There's a new aarch64-dbg build mode that should work
- gc() and defer() are mostly pacified; avoid using them on aarch64
- THIRD_PART_STB now has Arm Neon intrinsics for fast image handling
2023-05-12 22:42:57 -07:00
Justine Tunney
414667b1c9
Get TEST_LIBC_STR passing on AARCH64
It's now possible to run commands like:

    make -j8 m=aarch64 o/aarch64/test/libc/str

Which will cross-compile and run the test suites in a qemu-aarch64
binary that's vendored in the third_party/qemu/ folder within your
x86_64 build environment.
2023-05-12 18:09:23 -07:00
Justine Tunney
1f2a5a8fc1
Implement crash reporting for AARCH64
The ShowCrashReports() feature for aarch64 should work even better than
the x86 crash reports. Thanks to the benefit of hindsight these reports
should be rock solid reliable and beautiful to read.

This change also improves the syscall polyfills for aarch64. Some of the
sys_foo() functions have been removed, usually because they're legacy or
downright footguns not worth building.
2023-05-12 05:47:54 -07:00
Justine Tunney
5e2f7f7ced
Get LIBC_TESTLIB building on AARCH64 2023-05-11 19:57:09 -07:00
Justine Tunney
ea607781f8
Fix the build 2023-05-10 10:14:15 -07:00
Justine Tunney
64aca4dc4f
Fix weird behavior issue w/ kprintf() on aarch64 2023-05-10 06:17:33 -07:00
Justine Tunney
f312f706f4
Bring MODE=tiny binary sizes down to 20kb minimum
aarch64 binaries start at 4kb.
2023-05-10 04:20:48 -07:00
Justine Tunney
5a455eaa0b
Work on magic numbers for aarch64 2023-05-10 04:20:48 -07:00
Justine Tunney
a0237a017c
Get llama.com working on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney
4c093155a3
Get llama.com building as an aarch64 native binary 2023-05-10 04:20:47 -07:00
Justine Tunney
d04430f4ef
Get LIBC_MEM and LIBC_STDIO building with aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney
ae0ee59614
Get aarch64 hello world working
$ m=aarch64-tiny
    $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64
    $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com
    hello world
    $ ls -hal o/$m/tool/hello/hello.com
    -rwxr-xr-x 1 jart jart 4.0K May  9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-10 04:20:47 -07:00
Justine Tunney
e5e3cdf447
Get LIBC_RUNTIME and LIBC_CALLS building on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney
036b9a0002
Make further progress on non-x86 support 2023-05-10 04:20:47 -07:00
Justine Tunney
aef9a69a60
Make more progress on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney
135080fd3e
Get libc/tinymath/ compiling on aarch64 2023-05-10 04:20:46 -07:00