Commit graph

250 commits

Author SHA1 Message Date
Justine Tunney
7e46645193
Get LIBC_FMT to build for aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney
036b9a0002
Make further progress on non-x86 support 2023-05-10 04:20:47 -07:00
Justine Tunney
7c9ef924bf
Add more sorting algorithms 2023-04-27 05:44:44 -07:00
Justine Tunney
ba180e754d
Improve cosmocc toolchain
- Decouple zlib from libc
- Add some underscores to mostly internal names
2023-03-05 23:52:49 -08:00
Justine Tunney
dfabcd84c1
Fix sysinfo()
The system call wrapper was wrongfully reinterpreting kernel data. The
examples/sysinfo.c program is now updated to show how to correctly use
what's returned.
2023-02-23 06:54:58 -08:00
Justine Tunney
2b6261a52d
Make some system call fixes
- Fix minor ABI issue with SIOCGIFCONF
- Fix ABI translation issues with statfs() on BSDs
- Fix SQLite angled header line
2023-02-12 22:16:34 -08:00
Ronsor
9a46c5afc4 Add a new example showing how to use hiredis. 2023-01-01 16:30:10 -08:00
Ronsor
f6b3019796
Make curl example into a complete tool (#706)
You can now handily use `curl` with popen()/system() by
placing `STATIC_YOINK("_curl")` in your main source file.
2022-12-21 16:12:02 -08:00
tkchia
0da47c51de
[metal] Allow programs larger than 440 KiB to run in bare metal mode (#685)
* [metal] Copy program pages to extended memory at startup
* [metal] Reclaim base memory pages for later app use
* [metal] Load program pages beyond 1st 440 KiB to extended memory

o//examples/hellolua.com now runs correctly under QEMU (in
legacy BIOS mode).

* [metal] Place GDT in read/write segment

The CPU absolutely needs to alter the GDT when loading the
task register (via ltr).  To account for this, I move the
GDT into a read/write data section.  There is still a "rump"
read-only GDT in the text section that is used by the real
mode bootloader.

We also delay the loading of the task register (ltr) until
after the IDT and TSS are finally set up.

* [metal] Get examples/vga2.c serial output working for UEFI boot
* [metal] Get examples/vga2.c VGA output working for UEFI boot
* [metal] Allow munmap() to reclaim dynamically allocated pages
* Place TLS sections right after .text, not after embedded zip file

Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-12-17 17:51:20 -08:00
Justine Tunney
b407327972
Make fixes and improvements
- clock_nanosleep() is now much faster on OpenBSD and NetBSD
- Thread joining is now much faster on NetBSD
- FreeBSD timestamps are now more accurate
- Thread spawning now goes faster on XNU
- Clean up the clone() code
2022-11-08 10:11:46 -08:00
Justine Tunney
c995838e5c
Make improvements
- Clean up sigaction() code
- Add a port scanner example
- Introduce a ParseCidr() API
- Clean up our futex abstraction code
- Fix a harmless integer overflow in ParseIp()
- Use kernel semaphores on NetBSD to make threads much faster
2022-11-07 02:26:06 -08:00
Justine Tunney
3f0bcdc3ef
Improve cancellations, randomness, and time
- Exhaustively document cancellation points
- Rename SIGCANCEL to SIGTHR just like BSDs
- Further improve POSIX thread cancellations
- Ensure asynchronous cancellations work correctly
- Elevate the quality of getrandom() and getentropy()
- Make futexes cancel correctly on OpenBSD 6.x and 7.x
- Add reboot.com and shutdown.com to examples directory
- Remove underscore prefix from awesome timespec_*() APIs
- Create assertions that help verify our cancellation points
- Remove bad timespec APIs (cmp generalizes eq/ne/gt/gte/lt/lte)
2022-11-05 23:45:32 -07:00
Justine Tunney
f7ff77d865
Make fixes and improvements
- Invent iso8601us() for faster timestamps
- Improve --strace descriptions of sigset_t
- Rebuild the Landlock Make bootstrap binary
- Introduce MODE=sysv for non-Windows builds
- Permit OFD fcntl() locks under pledge(flock)
- redbean can now protect your kernel from ddos
- Have vfork() fallback to sys_fork() not fork()
- Change kmalloc() to not die when out of memory
- Improve documentation for some termios functions
- Rewrite putenv() and friends to conform to POSIX
- Fix linenoise + strace verbosity issue on Windows
- Fix regressions in our ability to show backtraces
- Change redbean SetHeader() to no-op if value is nil
- Improve fcntl() so SQLite locks work in non-WAL mode
- Remove some unnecessary work during fork() on Windows
- Create redbean-based SSL reverse proxy for IPv4 TurfWar
- Fix ape/apeinstall.sh warning when using non-bash shells
- Add ProgramTrustedIp(), and IsTrustedIp() APIs to redbean
- Support $PWD, $UID, $GID, and $EUID in command interpreter
- Introduce experimental JTqFpD APE prefix for non-Windows builds
- Invent blackhole daemon for firewalling IP addresses via UNIX named socket
- Add ProgramTokenBucket(), AcquireToken(), and CountTokens() APIs to redbean
2022-10-19 07:19:19 -07:00
Justine Tunney
3d2d04b25e
Make cli commands yoinkable into cosmo shell
This lets our system() and popen() commands function sort of like
BusyBox and ToyBox. By default the Cosmopolitan Shell is lightweight.
But if you use STATIC_YOINK then you can pull the individual commands
you want into the linkage, and they'll be included in a single binary.
For example the demo binary embeds `tr` and `sed` and ends up ~140kb.
2022-10-14 16:37:29 -07:00
Justine Tunney
5af19b7eed
Make some foss compatibility improvements 2022-10-14 13:59:34 -07:00
Justine Tunney
b41f91c658
Greatly expand system() shell code features
The cosmopolitan command interpreter now has 13 builtin commands,
variable support, support for ; / && / || syntax, asynchronous support,
and plenty of unit tests with bug fixes.

This change fixes a bug in posix_spawn() with null envp arg. strace
logging now uses atomic writes for scatter functions. Breaking change
renaming GetCpuCount() to _getcpucount(). TurfWar is now updated to use
the new token bucket algorithm. WIN32 affinity masks now inherit across
fork() and execve().
2022-10-11 21:30:31 -07:00
Justine Tunney
c19ac8729d
Reinvent the classic unix wall command 2022-10-11 21:22:03 -07:00
Connor
9209ea63a7
Integrate upstream dash changes into unbourne shell (#656) 2022-10-10 20:29:28 -07:00
Justine Tunney
e557058ac8
Improve cosmo's conformance to libc-test
This change addresses various open source compatibility issues, so that
we pass 313/411 of the tests in https://github.com/jart/libc-test where
earlier today we were passing about 30/411 of them, due to header toil.
Please note that Glibc only passes 341/411 so 313 today is pretty good!

- Make the conformance of libc/isystem/ headers nearly perfect
- Import more of the remaining math library routines from Musl
- Fix inconsistencies with type signatures of calls like umask
- Write tests for getpriority/setpriority which work great now
- conform to `struct sockaddr *` on remaining socket functions
- Import a bunch of uninteresting stdlib functions e.g. rand48
- Introduce readdir_r, scandir, pthread_kill, sigsetjmp, etc..

Follow the instructions in our `tool/scripts/cosmocc` toolchain to run
these tests yourself. You use `make CC=cosmocc` on the test repository
2022-10-10 17:52:41 -07:00
Justine Tunney
7ae556463a
Rename rand64() to _rand64() 2022-10-10 04:12:06 -07:00
Justine Tunney
9849b4c7ba
Add *NSYNC unit test suite
This change also fixes the clock_nanosleep() api and polyfills futexes
on Windows, Mac, and NetBSD using exponential backoff.
2022-10-07 21:34:15 -07:00
tkchia
d3efa47f81
Show "screen of death" on VGA console upon program crash (#650)
* [metal] Refactoring: separate out sys_writev_vga() and _vga_init() modules

* [metal] Read VGA info from BDA before long mode entry, not after

If using a pre-existing VGA text console, the VGA initialization
code now retrieves the cursor position & character height from
the BIOS data area while still in real mode — rather than
reading from the BIOS data area only after entering long mode.

(This should help make the code more correct, if Cosmopolitan
were to support UEFI graphics output in the future.  If the
program were booted via UEFI, then the long mode IsMetal()
code would still be activated, but the BIOS data area might
not have been initialized in that case.)

This change also means that there are now a few more fields
in the `struct mman`.

* [metal] VGA console can now show "screen of death" upon a crash

There is now a new function _klog_vga(), which can be called
by kprintf() to output system messages — e.g. information about
CPU exceptions — on the VGA screen.

* [metal] CPU exception handler now dumps cr2 value
* [metal] Add demo of program crash reporting w/ bare metal VGA TTY
* [metal] Reduce size of "screen of death" code
2022-10-06 05:36:15 -07:00
tkchia
edb8fef06c
Add basic character output for VGA graphics mode console (#649)
* Test output of colors for VGA graphics modes in examples/vga.c
* [metal] Character output in VGA graphics modes is mostly working
* [metal] Mention magic key to switch video mode, at bootup
2022-10-05 06:46:50 -07:00
Gavin Hayes
87708c5d6e
Change accept type to struct sockaddr * (#630) 2022-09-20 07:49:16 -07:00
Justine Tunney
2cc1d5ac4c
Improve redbean method=get parameter handling 2022-09-19 20:04:39 -07:00
Justine Tunney
c7a8cd21e9
Improve system call wrappers
This change improves copy_file_range(), sendfile(), splice(), openpty(),
closefrom(), close_range(), fadvise() and posix_fadvise() in addition to
writing tests that confirm things like errno and seeking behavior across
platforms. We now less aggressively polyfill behavior with some of these
functions when the platform support isn't available. Please see:

https://justine.lol/cosmopolitan/functions.html
2022-09-19 15:06:25 -07:00
Justine Tunney
3f49889841
Make important improvements
- Fix preadv() and pwritev() for old distros
- Introduce _npassert() and _unassert() macros
- Prove that file locks work properly on Windows
- Support fcntl(F_DUPFD_CLOEXEC) on more systems
2022-09-14 22:39:08 -07:00
Justine Tunney
1ad2f530f9
Add script.com and whois.com courtesy of FreeBSD 2022-09-13 20:26:13 -07:00
Justine Tunney
446a1f7c5a
Fix the build 2022-09-13 02:28:07 -07:00
tkchia
df08b541af
Bare metal VGA: implement "status report" escape codes (#613)
* Bare metal VGA: implement "status report" escape codes
* Minor fix to pseudoteletypewriter code

Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-09-13 02:14:10 -07:00
Justine Tunney
22b63d0b98
Add *NSYNC to libc/isystem 2022-09-13 01:47:12 -07:00
Justine Tunney
6f7d0cb1c3
Pay off more technical debt
This makes breaking changes to add underscores to many non-standard
function names provided by the c library. MODE=tiny is now tinier and
we now use smaller locks that are better for tiny apps in this mode.
Some headers have been renamed to be in the same folder as the build
package, so it'll be easier to know which build dependency is needed.
Certain old misguided interfaces have been removed. Intel intrinsics
headers are now listed in libc/isystem (but not in the amalgamation)
to help further improve open source compatibility. Header complexity
has also been reduced. Lastly, more shell scripts are now available.
2022-09-12 23:36:56 -07:00
Justine Tunney
9b89a7c51d
Add *NSYNC mu_test 2022-09-11 18:56:29 -07:00
Justine Tunney
b5cb71ab84
Use *NSYNC for POSIX threads locking APIs
Condition variables, barriers, and r/w locks now work very well.
2022-09-11 11:04:50 -07:00
Justine Tunney
155b378a39
Tidy up the threading implementation
The organization of the source files is now much more rational.
Old experiments that didn't work out are now deleted. Naming of
things like files is now more intuitive.
2022-09-10 02:56:25 -07:00
Justine Tunney
3265324e00
Don't relocate file descriptor memory
This change fixes #496 where ASAN spotted a race condition that could
happen in multithreaded programs, with more than OPEN_MAX descriptors
when using ZipOS or Windows NT, which require tracking open file info
and this change fixes that table so it never relocates, thus allowing
us to continue to enjoy the benefits of avoiding locks while reading.
2022-09-09 16:54:28 -07:00
Justine Tunney
c3208eb9d5
Add more POSIX threads APIs 2022-09-09 11:30:33 -07:00
Justine Tunney
9f963dc597
Clean up some of the threading code 2022-09-08 12:31:56 -07:00
Justine Tunney
0e2b1bfeed
Make garbage collection thread safe
- You can now use _gc(malloc()) in multithreaded programs
- This change fixes a bug where fork() on NT disabled TLS
- Fixed TLS code morphing on XNU/NT, for R8-R15 registers
2022-09-08 02:33:01 -07:00
tkchia
3fdb1c14f1
Add VGA support on bare metal (#588)
If your main module has this declaration:

    STATIC_YOINK("vga_console");

Then a VGA driver will be linked into your executable which
displays your stdio characters on the PC display, whereas
before we could only use the serial port. Your display is an
ANSI terminal and it's still a work in progress.
2022-09-06 18:41:08 -07:00
Justine Tunney
12d9e1e128
Improve quality of our ANSI C clock() function
It now works most excellently across all supported operating
sytsems (earlier it didn't work on NT and XNU). Demo code is
available in examples/clock.c and this change also adds some
of the newer ANSI C time functions like timespec_get(), plus
timespec_getres() which hasn't even come out yet as it's C23
2022-09-05 23:03:49 -07:00
Justine Tunney
b9dc74b672
Do some work on unbuffer command 2022-09-04 02:25:34 -07:00
Justine Tunney
b66bd064d8 Improve quality of uname/gethostname/getdomainname 2022-09-03 20:20:40 -07:00
Gavin Hayes
263711965f
Change sigaction_f to match sysv signature (#585) 2022-09-02 05:08:35 -07:00
Justine Tunney
83d41e4588 Clean up some code 2022-08-20 12:32:51 -07:00
Justine Tunney
c2211c9e63 Polyfill statfs() and fstatfs() on Windows 2022-08-17 19:01:51 -07:00
Justine Tunney
f7ee9d7d99 Polyfill statfs() and fstatfs() on BSD distros 2022-08-17 14:54:03 -07:00
Justine Tunney
f0701d2a24 Make improvements
- Polyfill pselect() on Windows
- Add -O NOFILE flag to pledge.com
- Polyfill ppoll() on NetBSD, XNU, and Windows
- Support negative numbers and errno in sizetol()
- Add .RSS, .NOFILE, and .MAXCORE to Landlock Make
- Fix issue with .PLEDGE preventing touching of output files
- Add __watch() function (like ftrace) for logging memory changes
2022-08-15 15:20:36 -07:00
Justine Tunney
7ab15e0b23 Add .PLEDGE/.CPU/.MEMORY/etc. to Landlock Make 1.2 2022-08-14 20:16:44 -07:00
Justine Tunney
0ea0d33a77 Reduce build graph by another eight percent 2022-08-13 13:11:56 -07:00