Commit graph

172 commits

Author SHA1 Message Date
Justine Tunney
23e235b7a5
Fix bugs in cosmocc toolchain
This change integrates e58abc1110b335a3341e8ad5821ad8e3880d9bb2 from
https://github.com/ahgamut/musl-cross-make/ which fixes the issues we
were having with our C language extension for symbolic constants. This
change also performs some code cleanup and bug fixes to getaddrinfo().
It's now possible to compile projects like ncurses, readline and python
without needing to patch anything upstream, except maybe a line or two.
Pretty soon it should be possible to build a Linux distro on Cosmo.
2023-06-08 23:44:03 -07:00
Justine Tunney
22f81a8d50
Improve cosmocc / cosmoc++ toolchain scripts
- Get out of the red zone
- Generate --ftrace nops unless -Os is passed
- Intercept -o path to generate .com / .com.dbg appropriately
2023-06-08 14:29:22 -07:00
Justine Tunney
25678db2a0
Fix remaining regressions in ASAN support
See #829
2023-06-08 08:00:50 -07:00
Justine Tunney
4d629fd424
Fix stack abuse in llama.cc
This change also incorporates improvements for MODE=asan. It's been
confirmed that o/asan/third_party/ggml/llama.com will work.

Fixes #829
2023-06-08 07:12:26 -07:00
Justine Tunney
7512318a2a
Fix MODE=aarch64 build 2023-06-08 05:17:37 -07:00
Justine Tunney
daf4454a06
Validate privileged code relationships
- Work towards improving non-optimized build support
- Introduce MODE=zero which is -O0 without ASAN/UBSAN
- Use system GCC when ~/.cosmo.mk has USE_SYSTEM_TOOLCHAIN=1
- Have package.com check .privileged code doesn't call non-privileged
2023-06-08 04:38:06 -07:00
Justine Tunney
b94b29d79c
Prevent ftrace from misaligning functions 2023-06-06 06:00:31 -07:00
Justine Tunney
eb40cb371d
Get --ftrace working on aarch64
This change implements a new approach to function call logging, that's
based on the GCC flag: -fpatchable-function-entry. Read the commentary
in build/config.mk to learn how it works.
2023-06-05 23:35:31 -07:00
Justine Tunney
9cc3e37263
Upgrade to Cosmopolitan GCC 11.2.0 for aarch64 2023-06-05 02:07:28 -07:00
Justine Tunney
39f20dbb13
Upgrade to Cosmopolitan GCC 11.2.0 for x86_64 2023-06-05 02:06:18 -07:00
Justine Tunney
4aa1d09b9e
Improve aarch64 native support some more
This change introduces partial support for automating remote testing of
aarch64 binaries on Raspberry Pi and Apple Silicon.
2023-06-04 08:58:47 -07:00
Justine Tunney
8f522cb702
Make improvements
This change progresses our AARCH64 support:

- The AARCH64 build and tests are now passing
- Add 128-bit floating-point support to printf()
- Fix clone() so it initializes cosmo's x28 TLS register
- Fix TLS memory layout issue with aarch64 _Alignas vars
- Revamp microbenchmarking tools so they work on aarch64
- Make some subtle improvements to aarch64 crash reporting
- Make kisdangerous() memory checks more accurate on aarch64
- Remove sys_open() since it's not available on Linux AARCH64

This change makes general improvements to Cosmo and Redbean:

- Introduce GetHostIsa() function in Redbean
- You can now feature check using pledge(0, 0)
- You can now feature check using unveil("",0)
- Refactor some more x86-specific asm comments
- Refactor and write docs for some libm functions
- Make the mmap() API behave more similar to Linux
- Fix WIFSIGNALED() which wrongly returned true for zero
- Rename some obscure cosmo keywords from noFOO to dontFOO
2023-06-03 08:12:22 -07:00
Justine Tunney
1422e96b4e
Introduce native support for MacOS ARM64
There's a new program named ape/ape-m1.c which will be used to build an
embeddable binary that can load ape and elf executables. The support is
mostly working so far, but still chasing down ABI issues.
2023-05-20 04:17:03 -07:00
Justine Tunney
cc1732bc42
Make AARCH64 harder, better, faster, stronger
- Perform some housekeeping on scalar math function code
- Import ARM's Optimized Routines for SIMD string processing
- Upgrade to latest Chromium zlib and enable more SIMD optimizations
2023-05-15 02:15:34 -07:00
Justine Tunney
550b52abf6
Port a lot more code to AARCH64
- Introduce epoll_pwait()
- Rewrite -ftrapv and ffs() libraries in C code
- Use more FreeBSD code in math function library
- Get significantly more tests passing on qemu-aarch64
- Fix many Musl long double functions that were broken on AARCH64
2023-05-14 09:37:26 -07:00
Justine Tunney
fd34ef732d
Make considerably more progress on AARCH64
- Utilities like pledge.com now build
- kprintf() will no longer balk at 48-bit addresses
- There's a new aarch64-dbg build mode that should work
- gc() and defer() are mostly pacified; avoid using them on aarch64
- THIRD_PART_STB now has Arm Neon intrinsics for fast image handling
2023-05-12 22:42:57 -07:00
Justine Tunney
4c093155a3
Get llama.com building as an aarch64 native binary 2023-05-10 04:20:47 -07:00
Justine Tunney
ae0ee59614
Get aarch64 hello world working
$ m=aarch64-tiny
    $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64
    $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com
    hello world
    $ ls -hal o/$m/tool/hello/hello.com
    -rwxr-xr-x 1 jart jart 4.0K May  9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-10 04:20:47 -07:00
Justine Tunney
aef9a69a60
Make more progress on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney
ca2860947f
Make progress towards aarch64 build 2023-05-10 04:20:46 -07:00
Justine Tunney
8303e23b3a
Do some basic build tuning 2023-05-10 04:20:46 -07:00
Justine Tunney
57cc257f58
Vendor musl-cross-make gcc 9.2.0 aarch64 2023-05-10 04:20:46 -07:00
Justine Tunney
8b69acc1cc
Make mkdeps more helpful when files are missing 2023-02-23 08:11:08 -08:00
Justine Tunney
711bd095db
Import exuberant ctags 2022-11-13 13:26:28 -08:00
Justine Tunney
b407327972
Make fixes and improvements
- clock_nanosleep() is now much faster on OpenBSD and NetBSD
- Thread joining is now much faster on NetBSD
- FreeBSD timestamps are now more accurate
- Thread spawning now goes faster on XNU
- Clean up the clone() code
2022-11-08 10:11:46 -08:00
Justine Tunney
fc96af058b
Workaround MAP_GROWSDOWN unavailability on WSL 2022-11-02 01:38:06 -07:00
Justine Tunney
f7ff77d865
Make fixes and improvements
- Invent iso8601us() for faster timestamps
- Improve --strace descriptions of sigset_t
- Rebuild the Landlock Make bootstrap binary
- Introduce MODE=sysv for non-Windows builds
- Permit OFD fcntl() locks under pledge(flock)
- redbean can now protect your kernel from ddos
- Have vfork() fallback to sys_fork() not fork()
- Change kmalloc() to not die when out of memory
- Improve documentation for some termios functions
- Rewrite putenv() and friends to conform to POSIX
- Fix linenoise + strace verbosity issue on Windows
- Fix regressions in our ability to show backtraces
- Change redbean SetHeader() to no-op if value is nil
- Improve fcntl() so SQLite locks work in non-WAL mode
- Remove some unnecessary work during fork() on Windows
- Create redbean-based SSL reverse proxy for IPv4 TurfWar
- Fix ape/apeinstall.sh warning when using non-bash shells
- Add ProgramTrustedIp(), and IsTrustedIp() APIs to redbean
- Support $PWD, $UID, $GID, and $EUID in command interpreter
- Introduce experimental JTqFpD APE prefix for non-Windows builds
- Invent blackhole daemon for firewalling IP addresses via UNIX named socket
- Add ProgramTokenBucket(), AcquireToken(), and CountTokens() APIs to redbean
2022-10-19 07:19:19 -07:00
Justine Tunney
e557058ac8
Improve cosmo's conformance to libc-test
This change addresses various open source compatibility issues, so that
we pass 313/411 of the tests in https://github.com/jart/libc-test where
earlier today we were passing about 30/411 of them, due to header toil.
Please note that Glibc only passes 341/411 so 313 today is pretty good!

- Make the conformance of libc/isystem/ headers nearly perfect
- Import more of the remaining math library routines from Musl
- Fix inconsistencies with type signatures of calls like umask
- Write tests for getpriority/setpriority which work great now
- conform to `struct sockaddr *` on remaining socket functions
- Import a bunch of uninteresting stdlib functions e.g. rand48
- Introduce readdir_r, scandir, pthread_kill, sigsetjmp, etc..

Follow the instructions in our `tool/scripts/cosmocc` toolchain to run
these tests yourself. You use `make CC=cosmocc` on the test repository
2022-10-10 17:52:41 -07:00
Justine Tunney
5005f2e446
Rewrite brk() and sbrk()
This change reduces the .bss memory requirement for all executables by
O(64kb). The brk system calls are now fully tested and figured out and
might be useful for tiny programs that only target System Five.
2022-10-01 23:11:56 -07:00
Justine Tunney
994e1f4386
Improve pthread_join()
Since we're now on Windows 8, we can have clone() work as advertised on
Windows, where it sends a futex wake to the child tid. It's also likely
we no longer need to work around thread flakes on OpenBSD, in _wait0().
2022-09-16 14:02:06 -07:00
Justine Tunney
3f49889841
Make important improvements
- Fix preadv() and pwritev() for old distros
- Introduce _npassert() and _unassert() macros
- Prove that file locks work properly on Windows
- Support fcntl(F_DUPFD_CLOEXEC) on more systems
2022-09-14 22:39:08 -07:00
Justine Tunney
06f9a5b627
Get repository to build with GCC 11
See #594
2022-09-13 04:14:55 -07:00
Justine Tunney
9cf2bdc9a8
Rebuild compile.com
This updates the bootstrap binary for the build compile command wrapper,
which was recently updated by #602.
2022-09-13 01:50:20 -07:00
Justine Tunney
22b63d0b98
Add *NSYNC to libc/isystem 2022-09-13 01:47:12 -07:00
Justine Tunney
1729a8259c
Introduce MODE=fastbuild for 28% speedup 2022-09-09 08:59:59 -07:00
Justine Tunney
571c2c3c69
Fix race condition in makedirs() 2022-09-07 23:12:44 -07:00
Justine Tunney
af24f21556
Unsandbox assembly generation 2022-09-05 08:27:15 -07:00
Justine Tunney
c5c4dfcd21 Improve quality of raise(), abort(), and tkill()
This change fixes a nasty bug where SIG_IGN and SIG_DFL weren't working
as advertised on BSDs. This change also fixes the tkill() definition on
MacOS so it maps to __pthread_kill().
2022-09-03 20:17:54 -07:00
Justine Tunney
c5659b93f8 Create unit test framework for subprocesses 2022-09-03 18:02:01 -07:00
Justine Tunney
57b39d8793 Revert recent fixupobj.com change 2022-08-21 17:33:54 -07:00
Justine Tunney
bf757c4646 Fix _Thread_local with fork() on Windows 2022-08-21 15:51:44 -07:00
Justine Tunney
ee49b71be2 Fix build 2022-08-21 01:11:14 -07:00
Justine Tunney
73845be1f0 Restore zip.com and .symtab files
This change restores the .symtab symbol table files in our flagship
programs (e.g. redbean.com, python.com) needed to show backtraces. This
also rolls back earlier changes to zip.com w.r.t. temp directories since
the right way to do it turned out to be the -b DIR flag.

This change also improves the performance of zip.com. It turned out
mmap() wasn't being used, because zip.com was assuming a 4096-byte
granularity, but cosmo requires 65536. There was also a chance to speed
up stdio scanning using the unlocked functions.
2022-08-21 00:17:20 -07:00
Justine Tunney
d76dfadc7a Reduce GNU Make latency 17% for cosmo
fgets() is now 4x faster which makes Make 2% faster. Landlock Make now
has a builtin $(uniq ...) function that uses critbit trees rather than
functional programming. Since uniq is the most important function this
optimization makes our cold start latency 15% faster.
2022-08-19 15:29:24 -07:00
Justine Tunney
8835b82a7c Avoid sandboxing directory prerequisites
Landlock Make will no longer sandbox prerequisites that end with a
trailing slash. This means you can use use directory prerequisites
for detecting deleted files when using using globbing, without the
effect of unveiling the entire directory. When you do want make to
unveil directories, you can omit the trailing slash.
2022-08-19 13:02:25 -07:00
Justine Tunney
255d834f8f Put ape.macho back in build/bootstrap 2022-08-15 15:45:13 -07:00
Justine Tunney
f0701d2a24 Make improvements
- Polyfill pselect() on Windows
- Add -O NOFILE flag to pledge.com
- Polyfill ppoll() on NetBSD, XNU, and Windows
- Support negative numbers and errno in sizetol()
- Add .RSS, .NOFILE, and .MAXCORE to Landlock Make
- Fix issue with .PLEDGE preventing touching of output files
- Add __watch() function (like ftrace) for logging memory changes
2022-08-15 15:20:36 -07:00
Justine Tunney
0cf9716039 Improve build latency of repository 2022-08-14 22:19:46 -07:00
Justine Tunney
d36d0634db Add automatic TMPDIR setup/teardown to GNU Make
We now guarantee TMPDIR will be defined on a per build rule basis. It'll
be an absolute path. It'll be secure and unique. It'll be rm -rf'd after
the last shell script line in your build rule is executed. If $TMPDIR is
already defined, then it'll be created as a subdirectory of your $TMPDIR
and then replace the variable with the new definition. The Landlock Make
repository will be updated with examples shortly after this change which
shall be known as Landlock Make 1.1.1.

See #530
2022-08-14 02:03:41 -07:00
Justine Tunney
2e54c8f9fc Change default build mode to use -O2 not -Og 2022-08-14 01:18:15 -07:00