Commit graph

82 commits

Author SHA1 Message Date
Alexander Nicholi
ec9bfd8c56
finish intellisense support and sync with upstream 2021-02-03 13:50:08 -05:00
Justine Tunney
4e56d89dcd Eliminate some flakes
- Get ASAN working on Windows.

- Deleting directories and then recreating them with the same name in a
  short period of time appears to be a no-no on Windows.

- There's no reason to call FlushFileBuffers on close() for pipes, and
  it's harmful since it might block indefinitely for no good reason.
2021-02-03 06:25:27 -08:00
Justine Tunney
27c899af56 Make mmap() work better
- Mapping file offsets now works on Windows
- Mapping stack memory now works on OpenBSD
2021-02-03 00:10:12 -08:00
Justine Tunney
23a14b537c Delete LIBC_CALLS_HEFTY
- fork() no longer requires malloc()
- readdir() moved to LIBC_STDIO
- Custom APIs moved to LIBC_X
2021-02-02 22:17:53 -08:00
Justine Tunney
c843243322 Implement more security stuff
- Support deterministic stacks on OpenBSD
- Support OpenBSD system call origin verification
- Fix overrun by one in chibicc string token allocator
- Get all chibicc tests passing under Address Sanitizer
2021-02-02 20:21:06 -08:00
Alexander Nicholi
7642710155
wip on intellisense (again) 2021-02-02 11:14:45 -05:00
Alexander Nicholi
9841e2186a
wip vscode intellisense support 2021-02-02 07:29:00 -05:00
Justine Tunney
cbfd4ccd1e Make more functions friendly to Address Sanitizer 2021-02-02 03:45:31 -08:00
Justine Tunney
3ab76b2312 Fix cosmopolitan.h for C++
We now have an integration test that runs the amalgamated sources
through a C++ compiler, to prevent the recurrence of such issues.

Fixes #38
Thanks @gbbnfhb for the report!
2021-02-01 04:04:44 -08:00
Justine Tunney
1ff9ab95ac Make C memory safe like Rust
This change enables Address Sanitizer systemically w/ `make MODE=dbg`.
Our version of Rust's `unsafe` keyword is named `noasan` which is used
for two functions that do aligned memory chunking, like `strcpy.c` and
we need to fix the tiny DEFLATE code, but that's it everything else is
fabulous you can have all the fischer price security blankets you need

Best of all is we're now able to use the ASAN data in Blinkenlights to
colorize the memory dumps. See the screenshot below of a test program:

  https://justine.lol/blinkenlights/asan.png

Which is operating on float arrays stored on the stack, with red areas
indicating poisoned memory, and the green areas indicate valid memory.
2021-02-01 03:58:46 -08:00
Justine Tunney
fdc3fa9148 Remove sun musl fdlibm math library
The libm code from musl wasn't being used since most of these functions
are implemented using x87 which goes faster than a library intended for
risc machines.
2021-01-31 18:25:16 -08:00
Justine Tunney
39bb91fc44 Improve fcntl(F_{SET,GET}FL) implementation (#32) 2021-01-30 13:10:32 -08:00
Justine Tunney
393a642c22 Increase Windows stack size (#32)
We're now using the same values CMD.EXE uses.
2021-01-30 10:08:28 -08:00
Justine Tunney
95173645a1 Implement getcwd() for XNU 2021-01-30 08:54:12 -08:00
Justine Tunney
417797d218 Support dirfd relative iops on Windows
We always favor calling functions like openat(), fstatat(), etc. because
Linux, XNU, FreeBSD, and OpenBSD all elected to support them, while some
systems like Android love them so much, that they stopped supporting the
old interfaces.

This change ensures that when dirfd is actually a dirfd and not AT_FDCWD
we'll do the right thing on Windows NT. We use an API that's been around
since Vista to accomplish that.

This change also adds exponential backoff to chdir() on Windows since it
seems almost as flaky on Windows 7 as the rmdir() function.
2021-01-30 01:49:43 -08:00
Justine Tunney
b8d26e2418 Fix fork() on Windows after chdir() 2021-01-29 23:19:29 -08:00
Justine Tunney
21e1023d28 Reduce memory requirements for execve() 2021-01-29 22:00:10 -08:00
Justine Tunney
eaca5b3e81 Add exponential backoff to rmdir() on Windows 2021-01-29 21:48:40 -08:00
Justine Tunney
bf8b1623c8 Normalize mkdir() error codes 2021-01-29 21:21:06 -08:00
Justine Tunney
caee314a50 Make PATH search do the right thing w/ empty path 2021-01-29 01:27:09 -08:00
Justine Tunney
a5f3456333 Refix readdir() on OpenBSD
We now have better tests which are catching these kinds of bugs.
2021-01-29 01:25:14 -08:00
Justine Tunney
f7c7b949fd Explore comfort level with posix_spawn
Justine says nay for the time being. Only XNU implements this. It's not
clear what ABI XNU is using but it's obviously not the one in the POSIX
documentation link below. Since all platforms implement vfork, it might
be better to empirically gauge the intersection of consensus which will
have better performance than this interface.

https://pubs.opengroup.org/onlinepubs/009696699/xrat/xsh_chap03.html
2021-01-29 01:24:54 -08:00
Justine Tunney
b2260c2278 Remove printfs accidentally in release :( 2021-01-29 00:07:53 -08:00
Justine Tunney
971bc8147f Check for EINTR in poll$nt()
This is a bandaid that lets CTRL-C work in daemons until a better
solution for signals on Windows can be implemented.
2021-01-28 15:53:06 -08:00
Justine Tunney
d8fffd2123 Change stack address on Windows
I rebooted Windows 7 and noticed that KERNEL32.DLL got relocated.
Microsoft needs to change and embrace the dream that was MULTICS.
Until then we'll need a better way to work around their dominion.
2021-01-28 13:42:53 -08:00
Justine Tunney
d7ac16a9ed Work towards improving signals and processes 2021-01-27 19:34:02 -08:00
Justine Tunney
45b72485ad Fix XNU / FreeBSD / OpenBSD / RHEL5 / NT bugs
For the first time ever, all tests in this codebase now pass, when
run automatically on macos, freebsd, openbsd, rhel5, rhel7, alpine
and windows via the network using the runit and runitd build tools

- Fix vfork exec path etc.
- Add XNU opendir() support
- Add OpenBSD opendir() support
- Add Linux history to syscalls.sh
- Use copy_file_range on FreeBSD 13+
- Fix system calls with 7+ arguments
- Fix Windows with greater than 16 FDs
- Fix RUNIT.COM and RUNITD.COM flakiness
- Fix OpenBSD munmap() when files are mapped
- Fix long double so it's actually long on Windows
- Fix OpenBSD truncate() and ftruncate() thunk typo
- Let Windows fcntl() be used on socket files descriptors
- Fix Windows fstat() which had an accidental printf statement
- Fix RHEL5 CLOCK_MONOTONIC by not aliasing to CLOCK_MONOTONIC_RAW

This is wonderful. I never could have dreamed it would be possible
to get it working so well on so many platforms with tiny binaries.

Fixes #31
Fixes #25
Fixes #14
2021-01-25 18:31:17 -08:00
Justine Tunney
78add3cb17 Fix AR.COM on RHEL7 (#24)
Doing this required improving the copyfd() polyfill needed by
copy_file_range() so it'll work harder to avoid short writes.
2021-01-17 10:28:52 -08:00
Justine Tunney
92b794002b Fix pmovmskb mnemonic in memeqmask
This change unbreaks test/libc/nexgen32e/memeqmask_test.c which appears
to currently be the only reference to this api. It was originally meant
to be used for speeding up terminal video rendering. We might delete it

Fixes #26
2021-01-16 21:52:14 -08:00
Justine Tunney
8fa47acecc Fix memcpy on sandybridge and ivybridge
This bug impacts folks who purchased Intel chips made in 2011-2012.
We're now using `vxorps` instead of `vpxor` which is great since it
means we do not need to change `X86_HAVE(AVX)` to `X86_HAVE(AVX2)`,
because AVX2 is only available on Haswell and later.

Fixes #16
2021-01-16 21:36:55 -08:00
Justine Tunney
2ad0d05d31 Avoid mapping above 4gb on Windows 7 x64 (#19)
Windows 7 support is now confirmed to be working. However it comes at
the cost of needing to workaround this tragically oppressive behavior
which is imposed for reasons unknown.

Fixes #19
2021-01-16 19:18:37 -08:00
Justine Tunney
f0600a898c Fix metal bugs so deathstar.com runs in qemu (#20)
- Remove XD bit in page tables
- Fix cylinder+head+sector arithmetic
- Implement fstat() for serial file descriptors on metal

Here's how to boot an Actually Portable Executable in QEMU:

    make -j12 o//tool/viz/deathstar.com
    qemu-system-x86_64 -serial stdio -fda o//tool/viz/deathstar.com

Here's a screenshot of DEATHSTAR.COM booted in QEMU:
https://justine.lol/cosmopolitan/cosmo-metal-qemu.png

Thus metal support is in much better shape now, but still incomplete.
Only a few system calls have been polyfilled. To figure out which ones
your program needs, simply boot it in the blinkenlights emulator with a
breakpoint, and press CTRL-C to continue to the system call breakpoint.
If it doesn't break then you should be good. (Note: to emulate normally
you can press 'c' and use CTRL-T and ALT-T to tune the speed.)

    m=tiny
    make -j12 SILENT=0 MODE=$m          \
      o/$m/tool/build/blinkenlights.com \
      o/$m/tool/viz/deathstar.com
    o/$m/tool/build/blinkenlights.com   \
      -r -t -b systemfive.linux         \
      o/$m/tool/viz/deathstar.com

Thank @Theldus for the bug report that made this change possible.
Fixes #20 which explains this change further.
2021-01-16 17:52:15 -08:00
Justine Tunney
58d9659d53 Remove call to SetDefaultDllDirectories (#19)
This API isn't present in Windows 7 SP1.
2021-01-16 17:50:29 -08:00
Justine Tunney
9f68d6eee9 Fix link order in cosmopolitan.a
It turned out that the linker was doing the wrong with the amalgamation
library concerning weak stubs. A regression test has been added and new
binaries have been uploaded to https://justine.lol/cosmopolitan/

Ideally this should be fixed by building a tool that turns multiple .a
files into a single .a file with deduplication. As a workaround for now
the cosmopolitan.a build is restructured to not include LIBC_STUBS which
meant technical debt needed to be paid off where non-stub interfaces
were moved to LIBC_INTRIN and LIBC_NEXGEN32E.

Thank @PerfectProductions in #31 for the report!
2021-01-16 12:05:41 -08:00
Justine Tunney
04f1d89f84 Replace .pushsection directives (#30) 2021-01-10 13:36:31 -08:00
Justine Tunney
5eddadafbd Fix strtol 2020-12-29 22:40:52 -08:00
Justine Tunney
1df136323b Add timingsafe_memcmp #13 2020-12-28 20:01:15 -08:00
Justine Tunney
0e85b136ae Fix Windows 7 support (#19)
This change pays off technical debt with the function -> DLL mappings in
libc/nt/master.sh, which was originally defined based on binary analysis
on Windows 10. It's now been updated so the kernel32/kernelbase/advapi32
imports should be exactly as they are written, on the MSDN documentation
and that wouldn't have been easy without Geoff Chappell's work thank him

https://www.geoffchappell.com/studies/windows/win32/index.htm
2020-12-28 13:52:02 -08:00
Justine Tunney
12065100e1 Add l suffix to yoink nop 2020-12-28 11:38:38 -08:00
Justine Tunney
de09bec215 Fix strlcpy 2020-12-27 21:35:38 -08:00
Justine Tunney
37a4c70c36 Change license 2020-12-27 17:18:44 -08:00
Justine Tunney
548dcb9f08 Further refine documentation 2020-12-27 17:05:03 -08:00
Justine Tunney
1bc3a25505 Improve documentation
The Cosmo API documentation page is pretty good now
https://justine.lol/cosmopolitan/documentation.html
2020-12-27 07:02:35 -08:00
Justine Tunney
13437dd19b Auto-generate some documentation 2020-12-26 02:09:07 -08:00
Justine Tunney
830334d767 Hunt down some small bugs 2020-12-24 16:01:48 -08:00
Justine Tunney
95b142e4e5 Make minor improvements 2020-12-23 23:42:56 -08:00
Justine Tunney
921f63da68 Fix build bug 2020-12-19 13:37:31 -08:00
Justine Tunney
b562d6fdb3 Make minor improvements 2020-12-19 11:21:04 -08:00
Justine Tunney
1fc91f3580 Fold conv package into fmt
Both packages had nearly identical dependency requirements, so merging
them should help reduce the complexity of the build graph.
2020-12-09 16:52:00 -08:00
Justine Tunney
9df2cef4c4 Enhance chibicc 2020-12-09 04:00:48 -08:00