Commit graph

47 commits

Author SHA1 Message Date
Justine Tunney
40291c9db3 Improve signal handling and math
- Polyfill ucontext_t on FreeBSD/OpenBSD/NetBSD
- Add tests confirming signals can edit CPU state
- Work towards supporting ZIP filesystem on bare metal
- Add more tinymath unit tests for POSIX conformance
- Add X87 and SSE status flags to crash report
- Fix some bugs in blinkenlights
- Fix llvm build breakage
2021-02-25 18:33:33 -08:00
Justine Tunney
edd9297eba Support malloc() on bare metal
Your Actually Portable Executables now contains a simple virtual memory
that works similarly to the Linux Kernel in the sense that it maps your
physical memory to negative addresses. This is needed to support mmap()
and malloc(). This functionality has zero code size impact. For example
the MODE=tiny LIFE.COM executable is still only 12KB in size.

The APE bootloader code has also been simplified to improve readibility
and further elevate the elegance by which we're able to support so many
platforms thereby enhancing verifiability so that we may engender trust
in this bootloading process.
2021-02-24 00:53:24 -08:00
Justine Tunney
ac3b1dfb21 Parse EFI command line arguments (#12) 2021-02-21 23:33:44 -08:00
Justine Tunney
537c21338b Add UEFI support
This is mutually exclusive with Windows support. Documentation for how
to use it has been written in libc/runtime/efimain.c
2021-02-21 21:33:04 -08:00
Justine Tunney
4a5698b5c9 Implement better hypot function 2021-02-21 11:06:18 -08:00
Justine Tunney
e345b42d78 Don't scrub PATH variable since Clang needs it 2021-02-20 12:54:13 -08:00
Justine Tunney
e85aeda4ba Fix some more build issues (#43)
We're now scrubbing environment variables in compile.com since gnu make
was not behaving as expected. It also appears there was a regression in
recent revisions that caused ASAN to be turned off for most binaries in
dbg mode, which has now been fixed. Cosmopolitan is fully ASAN hardened
down to the lowest level libraries and it doesn't need any interceptors
2021-02-20 12:39:39 -08:00
Justine Tunney
3e1fd1d962 Add tool for parsing strace output 2021-02-20 10:45:55 -08:00
Justine Tunney
b740cca642 Improve build system
- Reduce full build latency from ~20s to ~18s
- Bring back silent mode if `make V=0` is passed
- Demodernize utimes() polyfill so it works RHEL5
- Delete some old shell scripts that are no longer needed
- Truncate long lines when outputting builds to Emacs buffers
2021-02-19 23:03:34 -08:00
Justine Tunney
c797f139bb Set LC_ALL=C in build config (#43)
It's most likely the case (and SHOULD be the case) that C is the same
locale as C.UTF-8. The only exception might be Python's gonzo unicode
handling which whines when unicode is used without authorization, but
that shouldn't be an issue since we don't need Python to build.
2021-02-19 17:16:45 -08:00
Justine Tunney
adebc5e4c3 Remove support vector config shortcuts
These were used to build the "How Fat Does a Fat Binary Need to Be?"
blog post, but they're not needed anymore.
2021-02-19 17:16:17 -08:00
Justine Tunney
bfa8581537 Trim down executable sizes 2021-02-11 08:37:18 -08:00
Justine Tunney
e75ffde09e Get codebase completely working with LLVM
You can now build Cosmopolitan with Clang:

    make -j8 MODE=llvm
    o/llvm/examples/hello.com

The assembler and linker code is now friendly to LLVM too.
So it's not needed to configure Clang to use binutils under
the hood. If you love LLVM then you can now use pure LLVM.
2021-02-09 02:57:32 -08:00
Justine Tunney
0e36cb3ac4 Improve dead code elimination 2021-02-08 04:04:42 -08:00
Justine Tunney
2f3bd90216 Apply some touchups 2021-02-07 07:02:46 -08:00
Justine Tunney
c843243322 Implement more security stuff
- Support deterministic stacks on OpenBSD
- Support OpenBSD system call origin verification
- Fix overrun by one in chibicc string token allocator
- Get all chibicc tests passing under Address Sanitizer
2021-02-02 20:21:06 -08:00
Justine Tunney
cbfd4ccd1e Make more functions friendly to Address Sanitizer 2021-02-02 03:45:31 -08:00
Justine Tunney
1ff9ab95ac Make C memory safe like Rust
This change enables Address Sanitizer systemically w/ `make MODE=dbg`.
Our version of Rust's `unsafe` keyword is named `noasan` which is used
for two functions that do aligned memory chunking, like `strcpy.c` and
we need to fix the tiny DEFLATE code, but that's it everything else is
fabulous you can have all the fischer price security blankets you need

Best of all is we're now able to use the ASAN data in Blinkenlights to
colorize the memory dumps. See the screenshot below of a test program:

  https://justine.lol/blinkenlights/asan.png

Which is operating on float arrays stored on the stack, with red areas
indicating poisoned memory, and the green areas indicate valid memory.
2021-02-01 03:58:46 -08:00
Justine Tunney
e6481efa80 Use make SILENT=0 by default
Silent build configs are good for the core developers of a project
because they understand the build config, however it's an obstacle
for newcomers who'll be better served by seeing the build commands
2021-01-31 11:30:42 -08:00
Justine Tunney
d7ac16a9ed Work towards improving signals and processes 2021-01-27 19:34:02 -08:00
Justine Tunney
de703b182c Fix amalgamated release objects for older distros
We now pass -Wa,--nocompress-debug-sections to GCC so it won't use ELF
compression, since that causes "unable to initialize decompress status
for section .debug_aranges" when linked with past releases of binutils

Please note: this issue only impacts users who download cosmopolitan.a
from the website. Building from source isn't impacted, since we vendor
GCC 9.2.0 and Binutils 2.32 static binaries in the third_party folder.

Thanks @vshymanskyy for reporting this!
See #32
2021-01-27 16:21:01 -08:00
Justine Tunney
45b72485ad Fix XNU / FreeBSD / OpenBSD / RHEL5 / NT bugs
For the first time ever, all tests in this codebase now pass, when
run automatically on macos, freebsd, openbsd, rhel5, rhel7, alpine
and windows via the network using the runit and runitd build tools

- Fix vfork exec path etc.
- Add XNU opendir() support
- Add OpenBSD opendir() support
- Add Linux history to syscalls.sh
- Use copy_file_range on FreeBSD 13+
- Fix system calls with 7+ arguments
- Fix Windows with greater than 16 FDs
- Fix RUNIT.COM and RUNITD.COM flakiness
- Fix OpenBSD munmap() when files are mapped
- Fix long double so it's actually long on Windows
- Fix OpenBSD truncate() and ftruncate() thunk typo
- Let Windows fcntl() be used on socket files descriptors
- Fix Windows fstat() which had an accidental printf statement
- Fix RHEL5 CLOCK_MONOTONIC by not aliasing to CLOCK_MONOTONIC_RAW

This is wonderful. I never could have dreamed it would be possible
to get it working so well on so many platforms with tiny binaries.

Fixes #31
Fixes #25
Fixes #14
2021-01-25 18:31:17 -08:00
Justine Tunney
78add3cb17 Fix AR.COM on RHEL7 (#24)
Doing this required improving the copyfd() polyfill needed by
copy_file_range() so it'll work harder to avoid short writes.
2021-01-17 10:28:52 -08:00
Justine Tunney
565edd051f Remove dependency on ragel command 2021-01-17 09:42:52 -08:00
Justine Tunney
9f68d6eee9 Fix link order in cosmopolitan.a
It turned out that the linker was doing the wrong with the amalgamation
library concerning weak stubs. A regression test has been added and new
binaries have been uploaded to https://justine.lol/cosmopolitan/

Ideally this should be fixed by building a tool that turns multiple .a
files into a single .a file with deduplication. As a workaround for now
the cosmopolitan.a build is restructured to not include LIBC_STUBS which
meant technical debt needed to be paid off where non-stub interfaces
were moved to LIBC_INTRIN and LIBC_NEXGEN32E.

Thank @PerfectProductions in #31 for the report!
2021-01-16 12:05:41 -08:00
Justine Tunney
1bc3a25505 Improve documentation
The Cosmo API documentation page is pretty good now
https://justine.lol/cosmopolitan/documentation.html
2020-12-27 07:02:35 -08:00
Justine Tunney
9df2cef4c4 Enhance chibicc 2020-12-09 04:00:48 -08:00
Justine Tunney
8da931a7f6 Add chibicc
This program popped up on Hacker News recently. It's the only modern
compiler I've ever seen that doesn't have dependencies and is easily
modified. So I added all of the missing GNU extensions I like to use
which means it might be possible soon to build on non-Linux and have
third party not vendor gcc binaries.
2020-12-06 16:20:21 -08:00
Justine Tunney
ea0b5d9d1c Get Cosmopolitan into releasable state
A new rollup tool now exists for flattening out the headers in a way
that works better for our purposes than cpp. A lot of the API clutter
has been removed. APIs that aren't a sure thing in terms of general
recommendation are now marked internal.

There's now a smoke test for the amalgamation archive and gigantic
header file. So we can now guarantee you can use this project on the
easiest difficulty setting without the gigantic repository.

A website is being created, which is currently a work in progress:
https://justine.storage.googleapis.com/cosmopolitan/index.html
2020-11-25 08:19:00 -08:00
Justine Tunney
db33973e0a Get fork() working on Windows
This is done without using Microsoft's internal APIs. MAP_PRIVATE
mappings are copied to the subprocess via a pipe, since Microsoft
doesn't want us to have proper COW pages. MAP_SHARED mappings are
remapped without needing to do any copying. Global variables need
copying along with the stack and the whole heap of anonymous mem.
This actually improves the reliability of the redbean http server
although one shouldn't expect 10k+ connections on a home computer
that isn't running software built to serve like Linux or FreeBSD.
2020-11-13 03:14:39 -08:00
Justine Tunney
aea89fe832 Make minor improvements
- Work towards simplifying ape.S startup process
- Rewrote ar because it took minutes to build cosmopolitan.a
2020-11-09 15:41:11 -08:00
Justine Tunney
feed0d2b0e Add minor improvements and cleanup 2020-10-27 03:39:46 -07:00
Justine Tunney
c45e46f871 Add fixes performance and static web server 2020-10-05 23:11:49 -07:00
Justine Tunney
b6793d42d5 Add The LISP Challenge
This change introduces a 2.5kb program that's comes pretty close so far
to bootstrapping John McCarthy's metacircular evaluator on bare metal.
2020-10-01 01:20:13 -07:00
Justine Tunney
23d333c090 Make more improvements
This change includes many bug fixes, for the NT polyfills, strings,
memory, boot, and math libraries which were discovered by adding more
tools for recreational programming, such as PC emulation. Lemon has also
been vendored because it works so well at parsing languages.
2020-09-28 01:20:34 -07:00
Justine Tunney
7327c345f9 Get address sanitizer mostly working 2020-09-03 05:44:37 -07:00
Justine Tunney
1f1f3cd477 Fix bug in code written a priori 2020-08-31 21:02:14 -07:00
Justine Tunney
f213556366 Add binfmt_misc workaround detector to builds
This is a recently introduced Linux Kernel feature that gives people
like Debian package mantainers the power to arbitrarily redefine how
executables are interpreted by the kernel. If your system gets tuned
this way and you're not able to disable it, then you need to restore
default behavior for the APE MZqFpD prefix as follows:

    sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"

This prefix will cover all .com executables built with this tooling.
Please don't run the above command unless you're certain you need it.
See #2 for additional context.
2020-08-31 20:09:23 -07:00
Justine Tunney
bd29223891 Fix bugs and have emulator emulate itself 2020-08-31 05:17:31 -07:00
Justine Tunney
e86cff8ba0 Fix printvideo regression and minor improvements 2020-08-26 09:41:07 -07:00
Justine Tunney
f4f4caab0e Add x86_64-linux-gnu emulator
I wanted a tiny scriptable meltdown proof way to run userspace programs
and visualize how program execution impacts memory. It helps to explain
how things like Actually Portable Executable works. It can show you how
the GCC generated code is going about manipulating matrices and more. I
didn't feel fully comfortable with Qemu and Bochs because I'm not smart
enough to understand them. I wanted something like gVisor but with much
stronger levels of assurances. I wanted a single binary that'll run, on
all major operating systems with an embedded GPL barrier ZIP filesystem
that is tiny enough to transpile to JavaScript and run in browsers too.

https://justine.storage.googleapis.com/emulator625.mp4
2020-08-25 04:43:42 -07:00
Justine Tunney
72b654cb6c Add NES emulator
It's now possible to play classic video game roms with teletypewriters.
https://justine.storage.googleapis.com/nesemu.png
2020-07-02 13:46:08 -07:00
Justine Tunney
ac00be1a4e Make small fixes and oops ran clang-format on dtoa 2020-06-30 19:55:47 -07:00
Justine Tunney
d51409ccd9 Add glob and some finer tuning of documentation 2020-06-21 15:23:35 -07:00
Justine Tunney
799e24a87b Reduce build log verbosity with preprocessor trick 2020-06-18 23:07:58 -07:00
Justine Tunney
b4269930f7 Add scouts honor escape hatch for source embedding 2020-06-15 19:01:28 -07:00
Justine Tunney
c91b3c5006 Initial import 2020-06-15 07:18:57 -07:00