Commit graph

257 commits

Author SHA1 Message Date
Justine Tunney
4aa1d09b9e
Improve aarch64 native support some more
This change introduces partial support for automating remote testing of
aarch64 binaries on Raspberry Pi and Apple Silicon.
2023-06-04 08:58:47 -07:00
Justine Tunney
8f522cb702
Make improvements
This change progresses our AARCH64 support:

- The AARCH64 build and tests are now passing
- Add 128-bit floating-point support to printf()
- Fix clone() so it initializes cosmo's x28 TLS register
- Fix TLS memory layout issue with aarch64 _Alignas vars
- Revamp microbenchmarking tools so they work on aarch64
- Make some subtle improvements to aarch64 crash reporting
- Make kisdangerous() memory checks more accurate on aarch64
- Remove sys_open() since it's not available on Linux AARCH64

This change makes general improvements to Cosmo and Redbean:

- Introduce GetHostIsa() function in Redbean
- You can now feature check using pledge(0, 0)
- You can now feature check using unveil("",0)
- Refactor some more x86-specific asm comments
- Refactor and write docs for some libm functions
- Make the mmap() API behave more similar to Linux
- Fix WIFSIGNALED() which wrongly returned true for zero
- Rename some obscure cosmo keywords from noFOO to dontFOO
2023-06-03 08:12:22 -07:00
Justine Tunney
1422e96b4e
Introduce native support for MacOS ARM64
There's a new program named ape/ape-m1.c which will be used to build an
embeddable binary that can load ape and elf executables. The support is
mostly working so far, but still chasing down ABI issues.
2023-05-20 04:17:03 -07:00
Justine Tunney
cc1732bc42
Make AARCH64 harder, better, faster, stronger
- Perform some housekeeping on scalar math function code
- Import ARM's Optimized Routines for SIMD string processing
- Upgrade to latest Chromium zlib and enable more SIMD optimizations
2023-05-15 02:15:34 -07:00
Justine Tunney
fd34ef732d
Make considerably more progress on AARCH64
- Utilities like pledge.com now build
- kprintf() will no longer balk at 48-bit addresses
- There's a new aarch64-dbg build mode that should work
- gc() and defer() are mostly pacified; avoid using them on aarch64
- THIRD_PART_STB now has Arm Neon intrinsics for fast image handling
2023-05-12 22:42:57 -07:00
Justine Tunney
1f6f9e6701
Remove division from matrix multiplication
This change reduces llama.com CPU cycles systemically by 2.5% according
to the Linux Kernel `perf stat -Bddd` utility.
2023-05-10 21:19:54 -07:00
Justine Tunney
ae0ee59614
Get aarch64 hello world working
$ m=aarch64-tiny
    $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64
    $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com
    hello world
    $ ls -hal o/$m/tool/hello/hello.com
    -rwxr-xr-x 1 jart jart 4.0K May  9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-10 04:20:47 -07:00
Justine Tunney
7e46645193
Get LIBC_FMT to build for aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney
036b9a0002
Make further progress on non-x86 support 2023-05-10 04:20:47 -07:00
Justine Tunney
7c9ef924bf
Add more sorting algorithms 2023-04-27 05:44:44 -07:00
Justine Tunney
ba180e754d
Improve cosmocc toolchain
- Decouple zlib from libc
- Add some underscores to mostly internal names
2023-03-05 23:52:49 -08:00
Justine Tunney
dfabcd84c1
Fix sysinfo()
The system call wrapper was wrongfully reinterpreting kernel data. The
examples/sysinfo.c program is now updated to show how to correctly use
what's returned.
2023-02-23 06:54:58 -08:00
Justine Tunney
2b6261a52d
Make some system call fixes
- Fix minor ABI issue with SIOCGIFCONF
- Fix ABI translation issues with statfs() on BSDs
- Fix SQLite angled header line
2023-02-12 22:16:34 -08:00
Ronsor
9a46c5afc4 Add a new example showing how to use hiredis. 2023-01-01 16:30:10 -08:00
Ronsor
f6b3019796
Make curl example into a complete tool (#706)
You can now handily use `curl` with popen()/system() by
placing `STATIC_YOINK("_curl")` in your main source file.
2022-12-21 16:12:02 -08:00
tkchia
0da47c51de
[metal] Allow programs larger than 440 KiB to run in bare metal mode (#685)
* [metal] Copy program pages to extended memory at startup
* [metal] Reclaim base memory pages for later app use
* [metal] Load program pages beyond 1st 440 KiB to extended memory

o//examples/hellolua.com now runs correctly under QEMU (in
legacy BIOS mode).

* [metal] Place GDT in read/write segment

The CPU absolutely needs to alter the GDT when loading the
task register (via ltr).  To account for this, I move the
GDT into a read/write data section.  There is still a "rump"
read-only GDT in the text section that is used by the real
mode bootloader.

We also delay the loading of the task register (ltr) until
after the IDT and TSS are finally set up.

* [metal] Get examples/vga2.c serial output working for UEFI boot
* [metal] Get examples/vga2.c VGA output working for UEFI boot
* [metal] Allow munmap() to reclaim dynamically allocated pages
* Place TLS sections right after .text, not after embedded zip file

Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-12-17 17:51:20 -08:00
Justine Tunney
b407327972
Make fixes and improvements
- clock_nanosleep() is now much faster on OpenBSD and NetBSD
- Thread joining is now much faster on NetBSD
- FreeBSD timestamps are now more accurate
- Thread spawning now goes faster on XNU
- Clean up the clone() code
2022-11-08 10:11:46 -08:00
Justine Tunney
c995838e5c
Make improvements
- Clean up sigaction() code
- Add a port scanner example
- Introduce a ParseCidr() API
- Clean up our futex abstraction code
- Fix a harmless integer overflow in ParseIp()
- Use kernel semaphores on NetBSD to make threads much faster
2022-11-07 02:26:06 -08:00
Justine Tunney
3f0bcdc3ef
Improve cancellations, randomness, and time
- Exhaustively document cancellation points
- Rename SIGCANCEL to SIGTHR just like BSDs
- Further improve POSIX thread cancellations
- Ensure asynchronous cancellations work correctly
- Elevate the quality of getrandom() and getentropy()
- Make futexes cancel correctly on OpenBSD 6.x and 7.x
- Add reboot.com and shutdown.com to examples directory
- Remove underscore prefix from awesome timespec_*() APIs
- Create assertions that help verify our cancellation points
- Remove bad timespec APIs (cmp generalizes eq/ne/gt/gte/lt/lte)
2022-11-05 23:45:32 -07:00
Justine Tunney
f7ff77d865
Make fixes and improvements
- Invent iso8601us() for faster timestamps
- Improve --strace descriptions of sigset_t
- Rebuild the Landlock Make bootstrap binary
- Introduce MODE=sysv for non-Windows builds
- Permit OFD fcntl() locks under pledge(flock)
- redbean can now protect your kernel from ddos
- Have vfork() fallback to sys_fork() not fork()
- Change kmalloc() to not die when out of memory
- Improve documentation for some termios functions
- Rewrite putenv() and friends to conform to POSIX
- Fix linenoise + strace verbosity issue on Windows
- Fix regressions in our ability to show backtraces
- Change redbean SetHeader() to no-op if value is nil
- Improve fcntl() so SQLite locks work in non-WAL mode
- Remove some unnecessary work during fork() on Windows
- Create redbean-based SSL reverse proxy for IPv4 TurfWar
- Fix ape/apeinstall.sh warning when using non-bash shells
- Add ProgramTrustedIp(), and IsTrustedIp() APIs to redbean
- Support $PWD, $UID, $GID, and $EUID in command interpreter
- Introduce experimental JTqFpD APE prefix for non-Windows builds
- Invent blackhole daemon for firewalling IP addresses via UNIX named socket
- Add ProgramTokenBucket(), AcquireToken(), and CountTokens() APIs to redbean
2022-10-19 07:19:19 -07:00
Justine Tunney
3d2d04b25e
Make cli commands yoinkable into cosmo shell
This lets our system() and popen() commands function sort of like
BusyBox and ToyBox. By default the Cosmopolitan Shell is lightweight.
But if you use STATIC_YOINK then you can pull the individual commands
you want into the linkage, and they'll be included in a single binary.
For example the demo binary embeds `tr` and `sed` and ends up ~140kb.
2022-10-14 16:37:29 -07:00
Justine Tunney
5af19b7eed
Make some foss compatibility improvements 2022-10-14 13:59:34 -07:00
Justine Tunney
b41f91c658
Greatly expand system() shell code features
The cosmopolitan command interpreter now has 13 builtin commands,
variable support, support for ; / && / || syntax, asynchronous support,
and plenty of unit tests with bug fixes.

This change fixes a bug in posix_spawn() with null envp arg. strace
logging now uses atomic writes for scatter functions. Breaking change
renaming GetCpuCount() to _getcpucount(). TurfWar is now updated to use
the new token bucket algorithm. WIN32 affinity masks now inherit across
fork() and execve().
2022-10-11 21:30:31 -07:00
Justine Tunney
c19ac8729d
Reinvent the classic unix wall command 2022-10-11 21:22:03 -07:00
Connor
9209ea63a7
Integrate upstream dash changes into unbourne shell (#656) 2022-10-10 20:29:28 -07:00
Justine Tunney
e557058ac8
Improve cosmo's conformance to libc-test
This change addresses various open source compatibility issues, so that
we pass 313/411 of the tests in https://github.com/jart/libc-test where
earlier today we were passing about 30/411 of them, due to header toil.
Please note that Glibc only passes 341/411 so 313 today is pretty good!

- Make the conformance of libc/isystem/ headers nearly perfect
- Import more of the remaining math library routines from Musl
- Fix inconsistencies with type signatures of calls like umask
- Write tests for getpriority/setpriority which work great now
- conform to `struct sockaddr *` on remaining socket functions
- Import a bunch of uninteresting stdlib functions e.g. rand48
- Introduce readdir_r, scandir, pthread_kill, sigsetjmp, etc..

Follow the instructions in our `tool/scripts/cosmocc` toolchain to run
these tests yourself. You use `make CC=cosmocc` on the test repository
2022-10-10 17:52:41 -07:00
Justine Tunney
7ae556463a
Rename rand64() to _rand64() 2022-10-10 04:12:06 -07:00
Justine Tunney
9849b4c7ba
Add *NSYNC unit test suite
This change also fixes the clock_nanosleep() api and polyfills futexes
on Windows, Mac, and NetBSD using exponential backoff.
2022-10-07 21:34:15 -07:00
tkchia
d3efa47f81
Show "screen of death" on VGA console upon program crash (#650)
* [metal] Refactoring: separate out sys_writev_vga() and _vga_init() modules

* [metal] Read VGA info from BDA before long mode entry, not after

If using a pre-existing VGA text console, the VGA initialization
code now retrieves the cursor position & character height from
the BIOS data area while still in real mode — rather than
reading from the BIOS data area only after entering long mode.

(This should help make the code more correct, if Cosmopolitan
were to support UEFI graphics output in the future.  If the
program were booted via UEFI, then the long mode IsMetal()
code would still be activated, but the BIOS data area might
not have been initialized in that case.)

This change also means that there are now a few more fields
in the `struct mman`.

* [metal] VGA console can now show "screen of death" upon a crash

There is now a new function _klog_vga(), which can be called
by kprintf() to output system messages — e.g. information about
CPU exceptions — on the VGA screen.

* [metal] CPU exception handler now dumps cr2 value
* [metal] Add demo of program crash reporting w/ bare metal VGA TTY
* [metal] Reduce size of "screen of death" code
2022-10-06 05:36:15 -07:00
tkchia
edb8fef06c
Add basic character output for VGA graphics mode console (#649)
* Test output of colors for VGA graphics modes in examples/vga.c
* [metal] Character output in VGA graphics modes is mostly working
* [metal] Mention magic key to switch video mode, at bootup
2022-10-05 06:46:50 -07:00
Gavin Hayes
87708c5d6e
Change accept type to struct sockaddr * (#630) 2022-09-20 07:49:16 -07:00
Justine Tunney
2cc1d5ac4c
Improve redbean method=get parameter handling 2022-09-19 20:04:39 -07:00
Justine Tunney
c7a8cd21e9
Improve system call wrappers
This change improves copy_file_range(), sendfile(), splice(), openpty(),
closefrom(), close_range(), fadvise() and posix_fadvise() in addition to
writing tests that confirm things like errno and seeking behavior across
platforms. We now less aggressively polyfill behavior with some of these
functions when the platform support isn't available. Please see:

https://justine.lol/cosmopolitan/functions.html
2022-09-19 15:06:25 -07:00
Justine Tunney
3f49889841
Make important improvements
- Fix preadv() and pwritev() for old distros
- Introduce _npassert() and _unassert() macros
- Prove that file locks work properly on Windows
- Support fcntl(F_DUPFD_CLOEXEC) on more systems
2022-09-14 22:39:08 -07:00
Justine Tunney
1ad2f530f9
Add script.com and whois.com courtesy of FreeBSD 2022-09-13 20:26:13 -07:00
Justine Tunney
446a1f7c5a
Fix the build 2022-09-13 02:28:07 -07:00
tkchia
df08b541af
Bare metal VGA: implement "status report" escape codes (#613)
* Bare metal VGA: implement "status report" escape codes
* Minor fix to pseudoteletypewriter code

Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-09-13 02:14:10 -07:00
Justine Tunney
22b63d0b98
Add *NSYNC to libc/isystem 2022-09-13 01:47:12 -07:00
Justine Tunney
6f7d0cb1c3
Pay off more technical debt
This makes breaking changes to add underscores to many non-standard
function names provided by the c library. MODE=tiny is now tinier and
we now use smaller locks that are better for tiny apps in this mode.
Some headers have been renamed to be in the same folder as the build
package, so it'll be easier to know which build dependency is needed.
Certain old misguided interfaces have been removed. Intel intrinsics
headers are now listed in libc/isystem (but not in the amalgamation)
to help further improve open source compatibility. Header complexity
has also been reduced. Lastly, more shell scripts are now available.
2022-09-12 23:36:56 -07:00
Justine Tunney
9b89a7c51d
Add *NSYNC mu_test 2022-09-11 18:56:29 -07:00
Justine Tunney
b5cb71ab84
Use *NSYNC for POSIX threads locking APIs
Condition variables, barriers, and r/w locks now work very well.
2022-09-11 11:04:50 -07:00
Justine Tunney
155b378a39
Tidy up the threading implementation
The organization of the source files is now much more rational.
Old experiments that didn't work out are now deleted. Naming of
things like files is now more intuitive.
2022-09-10 02:56:25 -07:00
Justine Tunney
3265324e00
Don't relocate file descriptor memory
This change fixes #496 where ASAN spotted a race condition that could
happen in multithreaded programs, with more than OPEN_MAX descriptors
when using ZipOS or Windows NT, which require tracking open file info
and this change fixes that table so it never relocates, thus allowing
us to continue to enjoy the benefits of avoiding locks while reading.
2022-09-09 16:54:28 -07:00
Justine Tunney
c3208eb9d5
Add more POSIX threads APIs 2022-09-09 11:30:33 -07:00
Justine Tunney
9f963dc597
Clean up some of the threading code 2022-09-08 12:31:56 -07:00
Justine Tunney
0e2b1bfeed
Make garbage collection thread safe
- You can now use _gc(malloc()) in multithreaded programs
- This change fixes a bug where fork() on NT disabled TLS
- Fixed TLS code morphing on XNU/NT, for R8-R15 registers
2022-09-08 02:33:01 -07:00
tkchia
3fdb1c14f1
Add VGA support on bare metal (#588)
If your main module has this declaration:

    STATIC_YOINK("vga_console");

Then a VGA driver will be linked into your executable which
displays your stdio characters on the PC display, whereas
before we could only use the serial port. Your display is an
ANSI terminal and it's still a work in progress.
2022-09-06 18:41:08 -07:00
Justine Tunney
12d9e1e128
Improve quality of our ANSI C clock() function
It now works most excellently across all supported operating
sytsems (earlier it didn't work on NT and XNU). Demo code is
available in examples/clock.c and this change also adds some
of the newer ANSI C time functions like timespec_get(), plus
timespec_getres() which hasn't even come out yet as it's C23
2022-09-05 23:03:49 -07:00
Justine Tunney
b9dc74b672
Do some work on unbuffer command 2022-09-04 02:25:34 -07:00
Justine Tunney
b66bd064d8 Improve quality of uname/gethostname/getdomainname 2022-09-03 20:20:40 -07:00