mirrors/cosmopolitan
1
0
Fork 0
mirror of https://github.com/jart/cosmopolitan.git synced 2025-07-28 05:20:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Make improvements

Browse source 
- Every unit test now passes on Apple Silicon. The final piece of this
  puzzle was porting our POSIX threads cancelation support, since that
  works differently on ARM64 XNU vs. AMD64. Our semaphore support on
  Apple Silicon is also superior now compared to AMD64, thanks to the
  grand central dispatch library which lets *NSYNC locks go faster.

- The Cosmopolitan runtime is now more stable, particularly on Windows.
  To do this, thread local storage is mandatory at all runtime levels,
  and the innermost packages of the C library is no longer being built
  using ASAN. TLS is being bootstrapped with a 128-byte TIB during the
  process startup phase, and then later on the runtime re-allocates it
  either statically or dynamically to support code using _Thread_local.
  fork() and execve() now do a better job cooperating with threads. We
  can now check how much stack memory is left in the process or thread
  when functions like kprintf() / execve() etc. call alloca(), so that
  ENOMEM can be raised, reduce a buffer size, or just print a warning.

- POSIX signal emulation is now implemented the same way kernels do it
  with pthread_kill() and raise(). Any thread can interrupt any other
  thread, regardless of what it's doing. If it's blocked on read/write
  then the killer thread will cancel its i/o operation so that EINTR can
  be returned in the mark thread immediately. If it's doing a tight CPU
  bound operation, then that's also interrupted by the signal delivery.
  Signal delivery works now by suspending a thread and pushing context
  data structures onto its stack, and redirecting its execution to a
  trampoline function, which calls SetThreadContext(GetCurrentThread())
  when it's done.

- We're now doing a better job managing locks and handles. On NetBSD we
  now close semaphore file descriptors in forked children. Semaphores on
  Windows can now be canceled immediately, which means mutexes/condition
  variables will now go faster. Apple Silicon semaphores can be canceled
  too. We're now using Apple's pthread_yield() funciton. Apple _nocancel
  syscalls are now used on XNU when appropriate to ensure pthread_cancel
  requests aren't lost. The MbedTLS library has been updated to support
  POSIX thread cancelations. See tool/build/runitd.c for an example of
  how it can be used for production multi-threaded tls servers. Handles
  on Windows now leak less often across processes. All i/o operations on
  Windows are now overlapped, which means file pointers can no longer be
  inherited across dup() and fork() for the time being.

- We now spawn a thread on Windows to deliver SIGCHLD and wakeup wait4()
  which means, for example, that posix_spawn() now goes 3x faster. POSIX
  spawn is also now more correct. Like Musl, it's now able to report the
  failure code of execve() via a pipe although our approach favors using
  shared memory to do that on systems that have a true vfork() function.

- We now spawn a thread to deliver SIGALRM to threads when setitimer()
  is used. This enables the most precise wakeups the OS makes possible.

- The Cosmopolitan runtime now uses less memory. On NetBSD for example,
  it turned out the kernel would actually commit the PT_GNU_STACK size
  which caused RSS to be 6mb for every process. Now it's down to ~4kb.
  On Apple Silicon, we reduce the mandatory upstream thread size to the
  smallest possible size to reduce the memory overhead of Cosmo threads.
  The examples directory has a program called greenbean which can spawn
  a web server on Linux with 10,000 worker threads and have the memory
  usage of the process be ~77mb. The 1024 byte overhead of POSIX-style
  thread-local storage is now optional; it won't be allocated until the
  pthread_setspecific/getspecific functions are called. On Windows, the
  threads that get spawned which are internal to the libc implementation
  use reserve rather than commit memory, which shaves a few hundred kb.

- sigaltstack() is now supported on Windows, however it's currently not
  able to be used to handle stack overflows, since crash signals are
  still generated by WIN32. However the crash handler will still switch
  to the alt stack, which is helpful in environments with tiny threads.

- Test binaries are now smaller. Many of the mandatory dependencies of
  the test runner have been removed. This ensures many programs can do a
  better job only linking the the thing they're testing. This caused the
  test binaries for LIBC_FMT for example, to decrease from 200kb to 50kb

- long double is no longer used in the implementation details of libc,
  except in the APIs that define it. The old code that used long double
  for time (instead of struct timespec) has now been thoroughly removed.

- ShowCrashReports() is now much tinier in MODE=tiny. Instead of doing
  backtraces itself, it'll just print a command you can run on the shell
  using our new `cosmoaddr2line` program to view the backtrace.

- Crash report signal handling now works in a much better way. Instead
  of terminating the process, it now relies on SA_RESETHAND so that the
  default SIG_IGN behavior can terminate the process if necessary.

- Our pledge() functionality has now been fully ported to AARCH64 Linux.
This commit is contained in:
Justine Tunney 2023-09-18 20:44:45 -07:00
parent c4eb838516
commit ec480f5aa0
No known key found for this signature in database
GPG key ID: BE714B4575D6E328
638 changed files with 7925 additions and 8282 deletions
Download patch file Download diff file Expand all files Collapse all files

101
libc/runtime/clone.c
View file

@ -33,6 +33,8 @@
#include "libc/intrin/weaken.h"
#include "libc/limits.h"
#include "libc/macros.internal.h"
#include "libc/mem/alloca.h"
#include "libc/nt/enum/processcreationflags.h"
#include "libc/nt/runtime.h"
#include "libc/nt/signals.h"
#include "libc/nt/synchronization.h"
@ -40,10 +42,12 @@
#include "libc/nt/thunk/msabi.h"
#include "libc/runtime/internal.h"
#include "libc/runtime/runtime.h"
#include "libc/runtime/stack.h"
#include "libc/runtime/syslib.internal.h"
#include "libc/sock/internal.h"
#include "libc/stdalign.internal.h"
#include "libc/str/str.h"
#include "libc/sysv/consts/clone.h"
#include "libc/sysv/consts/futex.h"
#include "libc/sysv/consts/nr.h"
#include "libc/sysv/consts/nrlinux.h"
@ -85,6 +89,8 @@ struct CloneArgs {
void *arg;
};
int __stack_call(void *, int, long, long, int (*)(void *, int), void *);
static struct CloneArgs *AllocateCloneArgs(char *stk, size_t stksz) {
return (struct CloneArgs *)(((uintptr_t)(stk + stksz) -
sizeof(struct CloneArgs)) &
@ -100,29 +106,15 @@ __msabi extern typeof(TlsSetValue) *const __imp_TlsSetValue;
__msabi extern typeof(ExitThread) *const __imp_ExitThread;
__msabi extern typeof(WakeByAddressAll) *const __imp_WakeByAddressAll;
int WinThreadLaunch(void *arg, // rdi
int tid, // rsi
int (*func)(void *, int), // rdx
intptr_t rsp); // rcx
// we can't log this function because:
// 1. windows owns the backtrace pointer right now
// 2. ftrace unwinds rbp to determine depth
// 3. tid in tls for ftrace isn't set yet
// we can't use address sanitizer because:
// 1. __asan_handle_no_return wipes stack
// 2. windows owns the stack memory right now
// we need win32 raw imports because:
// 1. generated thunks are function logged
dontasan dontinstrument static textwindows wontreturn void //
WinThreadEntry(int rdi, // rcx
int rsi, // rdx
int rdx, // r8
struct CloneArgs *wt) { // r9
static textwindows dontinstrument wontreturn void //
WinThreadEntry(int rdi, // rcx
int rsi, // rdx
int rdx, // r8
struct CloneArgs *wt) { // r9
int rc;
if (wt->tls) __set_tls_win32(wt->tls);
*wt->ctid = wt->tid;
rc = WinThreadLaunch(wt->arg, wt->tid, wt->func, (intptr_t)wt);
rc = __stack_call(wt->arg, wt->tid, 0, 0, wt->func, wt);
// we can now clear ctid directly since we're no longer using our own
// stack memory, which can now be safely free'd by the parent thread.
*wt->ztid = 0;
@ -130,7 +122,7 @@ WinThreadEntry(int rdi, // rcx
// since we didn't indirect this function through NT2SYSV() it's not
// safe to simply return, and as such, we need ExitThread().
__imp_ExitThread(rc);
notpossible;
__builtin_unreachable();
}
static textwindows errno_t CloneWindows(int (*func)(void *, int), char *stk,
@ -144,8 +136,12 @@ static textwindows errno_t CloneWindows(int (*func)(void *, int), char *stk,
wt->func = func;
wt->arg = arg;
wt->tls = flags & CLONE_SETTLS ? tls : 0;
if ((h = CreateThread(0, 65536, (void *)WinThreadEntry, wt, 0, &wt->utid))) {
CloseHandle(h);
if ((h = CreateThread(0, 65536, (void *)WinThreadEntry, wt,
kNtStackSizeParamIsAReservation, &wt->utid))) {
if (flags & CLONE_SETTLS) {
struct CosmoTib *tib = tls;
tib->tib_syshand = h;
}
if (flags & CLONE_PARENT_SETTID) {
*ptid = wt->tid;
}
@ -196,9 +192,6 @@ XnuThreadMain(void *pthread, // rdi
func(arg, tid);
// avoid signal handler being triggered after we trash our stack
_sigblockall();
// we no longer use the stack after this point
// %rax = int bsdthread_terminate(%rdi = void *stackaddr,
// %rsi = size_t freesize,
@ -210,7 +203,7 @@ XnuThreadMain(void *pthread, // rdi
: "=m"(*wt->ztid)
: "a"(0x2000000 | 361), "D"(0), "S"(0), "d"(0L)
: "rcx", "r10", "r11", "memory");
notpossible;
__builtin_unreachable();
}
static errno_t CloneXnu(int (*fn)(void *), char *stk, size_t stksz, int flags,
@ -236,8 +229,6 @@ static wontreturn void FreebsdThreadMain(void *p) {
struct CloneArgs *wt = p;
*wt->ctid = wt->tid;
wt->func(wt->arg, wt->tid);
// avoid signal handler being triggered after we trash our stack
_sigblockall();
// we no longer use the stack after this point
// void thr_exit(%rdi = long *state);
asm volatile("movl\t$0,%0\n\t" // *wt->ztid = 0
@ -250,7 +241,7 @@ static wontreturn void FreebsdThreadMain(void *p) {
: "=m"(*wt->ztid)
: "a"(454), "D"(wt->ztid), "S"(UMTX_OP_WAKE), "d"(INT_MAX)
: "rcx", "r8", "r9", "r10", "r11", "memory");
notpossible;
__builtin_unreachable();
}
static errno_t CloneFreebsd(int (*func)(void *, int), char *stk, size_t stksz,
@ -290,7 +281,7 @@ static errno_t CloneFreebsd(int (*func)(void *, int), char *stk, size_t stksz,
// we can't use address sanitizer because:
// 1. __asan_handle_no_return wipes stack [todo?]
dontasan static wontreturn void OpenbsdThreadMain(void *p) {
static wontreturn void OpenbsdThreadMain(void *p) {
struct CloneArgs *wt = p;
*wt->ctid = wt->tid;
wt->func(wt->arg, wt->tid);
@ -304,7 +295,7 @@ dontasan static wontreturn void OpenbsdThreadMain(void *p) {
: "a"(83), "m"(__oldstack), "D"(wt->ztid),
"S"(2 /* FUTEX_WAKE */), "d"(INT_MAX)
: "rcx", "r11", "memory");
notpossible;
__builtin_unreachable();
}
static errno_t CloneOpenbsd(int (*func)(void *, int), char *stk, size_t stksz,
@ -353,8 +344,6 @@ static wontreturn void NetbsdThreadMain(void *arg, // rdi
ax = sys_gettid();
*ctid = ax;
func(arg, ax);
// avoid signal handler being triggered after we trash our stack
_sigblockall();
// we no longer use the stack after this point
// %eax = int __lwp_exit(void);
asm volatile("movl\t$0,%2\n\t" // *wt->ztid = 0
@ -362,7 +351,7 @@ static wontreturn void NetbsdThreadMain(void *arg, // rdi
: "=a"(ax), "=d"(dx), "=m"(*ztid)
: "0"(310)
: "rcx", "r11", "memory");
notpossible;
__builtin_unreachable();
}
static int CloneNetbsd(int (*func)(void *, int), char *stk, size_t stksz,
@ -449,21 +438,10 @@ static int CloneNetbsd(int (*func)(void *, int), char *stk, size_t stksz,
// APPLE SILICON
static void *SiliconThreadMain(void *arg) {
register struct CloneArgs *wt asm("x21") = arg;
struct CloneArgs *wt = arg;
asm volatile("mov\tx28,%0" : /* no outputs */ : "r"(wt->tls));
*wt->ctid = wt->this;
register void *x0 asm("x0") = wt->arg;
register int x1 asm("w1") = wt->this;
register void *x28 asm("x28") = wt->tls;
asm volatile("mov\tx19,x29\n\t" // save frame pointer
"mov\tx20,sp\n\t" // save stack pointer
"mov\tx29,#0\n\t" // reset backtrace
"mov\tsp,%3\n\t" // switch stack
"blr\t%2\n\t" // wt->func(wt->arg, tid)
"mov\tx29,x19\n\t" // restore frame pointer
"mov\tsp,x20" // restore stack pointer
: "+r"(x0)
: "r"(x1), "r"(wt->func), "r"(wt), "r"(x28)
: "x19", "x20", "memory");
__stack_call(wt->arg, wt->this, 0, 0, wt->func, wt);
*wt->ztid = 0;
return 0;
}
@ -471,9 +449,11 @@ static void *SiliconThreadMain(void *arg) {
static errno_t CloneSilicon(int (*fn)(void *, int), char *stk, size_t stksz,
int flags, void *arg, void *tls, int *ptid,
int *ctid) {
void *attr;
errno_t res;
unsigned tid;
pthread_t th;
size_t babystack;
struct CloneArgs *wt;
static atomic_uint tids;
wt = AllocateCloneArgs(stk, stksz);
@ -484,14 +464,23 @@ static errno_t CloneSilicon(int (*fn)(void *, int), char *stk, size_t stksz,
wt->tls = flags & CLONE_SETTLS ? tls : 0;
wt->func = fn;
wt->arg = arg;
if (!(res = __syslib->pthread_create(&th, 0, SiliconThreadMain, wt)) &&
babystack = __syslib->__pthread_stack_min;
#pragma GCC push_options
#pragma GCC diagnostic ignored "-Walloca-larger-than="
attr = alloca(__syslib->__sizeof_pthread_attr_t);
#pragma GCC pop_options
unassert(!__syslib->__pthread_attr_init(attr));
unassert(!__syslib->__pthread_attr_setguardsize(attr, 0));
unassert(!__syslib->__pthread_attr_setstacksize(attr, babystack));
if (!(res = __syslib->__pthread_create(&th, attr, SiliconThreadMain, wt)) &&
(flags & CLONE_PARENT_SETTID)) {
*ptid = tid;
if (flags & CLONE_SETTLS) {
struct CosmoTib *tib = tls;
tib[-1].tib_pthread = th;
tib[-1].tib_syshand = th;
}
}
unassert(!__syslib->__pthread_attr_destroy(attr));
return res;
}
@ -620,9 +609,10 @@ static int CloneLinux(int (*func)(void *arg, int rc), char *stk, size_t stksz,
* child has started.
*
* - `CLONE_PARENT_SETTID` must be specified if you intend to set
* the `ptid` argument, which is updated at the most opportune
* moment. On all platforms except XNU, this happens before
* clone() returns. On XNU, it happens once the thread starts.
* the `ptid` argument, and it is updated at the most opportune
* moment. On all platforms except XNU x86, this happens before
* clone() returns. But since it might not be available yet you
* need to use pthread_getunique_np() to obtain it.
*
* - `CLONE_CHILD_CLEARTID` causes `*ctid = 0` upon child thread
* termination. This is used to implement join so that the parent
@ -653,9 +643,6 @@ errno_t clone(void *func, void *stk, size_t stksz, int flags, void *arg,
if (!func) {
rc = EINVAL;
} else if (!IsTiny() &&
((flags & CLONE_VM) && (stksz < 4096 || (stksz & 15)))) {
rc = EINVAL;
} else if (IsAsan() &&
(((flags & CLONE_SETTLS) && !__asan_is_valid(tls, 64)) ||
((flags & CLONE_PARENT_SETTID) &&

1069
libc/runtime/cocmd.c
View file

File diff suppressed because it is too large Load diff

13
libc/runtime/cosmo2.c
View file

@ -84,6 +84,17 @@ wontreturn textstartup void cosmo(long *sp, struct Syslib *m1) {
// its used by --strace and also kprintf() %T
kStartTsc = rdtsc();
// enable enough tls to survive until it can be allocated properly
struct CosmoTib tib = {
.tib_self = &tib,
.tib_self2 = &tib,
.tib_sigmask = -1,
.tib_sigstack_size = 57344,
.tib_sigstack_addr = (char *)__builtin_frame_address(0) - 57344,
.tib_tid = 1,
};
__set_tls(&tib);
// extracts arguments from old sysv stack abi
int argc = *sp;
char **argv = (char **)(sp + 1);
@ -118,7 +129,7 @@ wontreturn textstartup void cosmo(long *sp, struct Syslib *m1) {
}
// check system call abi compatibility
if (SupportsXnu() && __syslib && __syslib->version < SYSLIB_VERSION) {
if (SupportsXnu() && __syslib && __syslib->__version < SYSLIB_VERSION) {
sys_write(2, "need newer ape loader\n", 22);
_Exit(127);
}

68
libc/runtime/daemon.c
View file

@ -1,68 +0,0 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
│vi: set net ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2022 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/assert.h"
#include "libc/calls/calls.h"
#include "libc/calls/syscall_support-sysv.internal.h"
#include "libc/dce.h"
#include "libc/nt/enum/processcreationflags.h"
#include "libc/paths.h"
#include "libc/runtime/runtime.h"
#include "libc/sysv/consts/o.h"
/**
* Runs process in background.
*
* On Unix this calls fork() and setsid(). On Windows this is
* implemented using CreateProcess(kNtDetachedProcess).
*
* @return 0 on success, or -1 w/ errno
*/
int daemon(int nochdir, int noclose) {
int fd;
switch (_fork(kNtDetachedProcess)) {
case -1:
return -1;
case 0:
break;
default:
_Exit(0);
}
if (!IsWindows()) {
if (setsid() == -1) {
return -1;
}
}
if (!nochdir) {
unassert(!chdir("/"));
}
if (!noclose && (fd = open(_PATH_DEVNULL, O_RDWR)) != -1) {
unassert(dup2(fd, 0) == 0);
unassert(dup2(fd, 1) == 1);
unassert(dup2(fd, 2) == 2);
if (fd > 2) {
unassert(!close(fd));
}
}
return 0;
}

2
libc/runtime/efimain.greg.c
View file

@ -141,7 +141,7 @@ static void EfiInitVga(struct mman *mm, EFI_SYSTEM_TABLE *SystemTable) {
*
* @see libc/dce.h
*/
__msabi dontasan EFI_STATUS EfiMain(EFI_HANDLE ImageHandle,
__msabi EFI_STATUS EfiMain(EFI_HANDLE ImageHandle,
EFI_SYSTEM_TABLE *SystemTable) {
struct mman *mm;
uint32_t DescVersion;

30
libc/runtime/enable_tls.c
View file

@ -27,20 +27,25 @@
#include "libc/intrin/dll.h"
#include "libc/intrin/weaken.h"
#include "libc/macros.internal.h"
#include "libc/nt/files.h"
#include "libc/nt/runtime.h"
#include "libc/nt/synchronization.h"
#include "libc/nt/thread.h"
#include "libc/runtime/internal.h"
#include "libc/runtime/runtime.h"
#include "libc/runtime/syslib.internal.h"
#include "libc/str/locale.h"
#include "libc/str/str.h"
#include "libc/thread/posixthread.internal.h"
#include "libc/thread/thread.h"
#include "libc/thread/tls.h"
#include "third_party/make/gnumake.h"
#define I(x) ((uintptr_t)x)
extern unsigned char __tls_mov_nt_rax[];
extern unsigned char __tls_add_nt_rax[];
static struct PosixThread _pthread_main;
_Alignas(TLS_ALIGNMENT) static char __static_tls[6016];
/**
@ -191,7 +196,15 @@ textstartup void __enable_tls(void) {
tib->tib_strace = __strace;
tib->tib_ftrace = __ftrace;
tib->tib_locale = (intptr_t)&__c_dot_utf8_locale;
tib->tib_pthread = (pthread_t)&_pthread_main;
tib->tib_pthread = (pthread_t)&_pthread_static;
if (IsWindows()) {
intptr_t threadhand, pseudo = GetCurrentThread();
DuplicateHandle(GetCurrentProcess(), pseudo, GetCurrentProcess(),
&threadhand, 0, false, kNtDuplicateSameAccess);
atomic_store_explicit(&tib->tib_syshand, threadhand, memory_order_relaxed);
} else if (IsXnuSilicon()) {
tib->tib_syshand = __syslib->__pthread_self();
}
if (IsLinux() || IsXnuSilicon()) {
// gnu/systemd guarantees pid==tid for the main thread so we can
// avoid issuing a superfluous system call at startup in program
@ -202,11 +215,14 @@ textstartup void __enable_tls(void) {
atomic_store_explicit(&tib->tib_tid, tid, memory_order_relaxed);
// initialize posix threads
_pthread_main.tib = tib;
_pthread_main.flags = PT_STATIC;
_pthread_main.list.prev = _pthread_main.list.next = //
_pthread_list = __veil("r", &_pthread_main.list);
atomic_store_explicit(&_pthread_main.ptid, tid, memory_order_relaxed);
_pthread_static.tib = tib;
_pthread_static.pt_flags = PT_STATIC;
dll_init(&_pthread_static.list);
_pthread_list = &_pthread_static.list;
atomic_store_explicit(&_pthread_static.ptid, tid, memory_order_relaxed);
if (IsWindows()) {
npassert((_pthread_static.semaphore = CreateSemaphore(0, 0, 1, 0)));
}
// copy in initialized data section
if (I(_tdata_size)) {

64
libc/runtime/finddebugbinary.c
View file

@ -17,7 +17,11 @@
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "ape/sections.internal.h"
#include "libc/atomic.h"
#include "libc/calls/blockcancel.internal.h"
#include "libc/calls/blocksigs.internal.h"
#include "libc/calls/calls.h"
#include "libc/cosmo.h"
#include "libc/elf/tinyelf.internal.h"
#include "libc/errno.h"
#include "libc/intrin/bits.h"
@ -29,12 +33,19 @@
#include "libc/sysv/consts/o.h"
#include "libc/sysv/consts/prot.h"
static bool IsMyDebugBinaryImpl(const char *path) {
int fd;
static struct {
atomic_uint once;
const char *res;
char buf[PATH_MAX];
} g_comdbg;
static bool IsMyDebugBinary(const char *path) {
void *map;
int64_t size;
uintptr_t value;
bool res = false;
int fd, e = errno;
BLOCK_CANCELLATIONS;
if ((fd = open(path, O_RDONLY | O_CLOEXEC, 0)) != -1) {
// sanity test that this .com.dbg file (1) is an elf image, and (2)
// contains the same number of bytes of code as our .com executable
@ -49,16 +60,26 @@ static bool IsMyDebugBinaryImpl(const char *path) {
}
close(fd);
}
ALLOW_CANCELLATIONS;
errno = e;
return res;
}
static bool IsMyDebugBinary(const char *path) {
int e;
bool res;
e = errno;
res = IsMyDebugBinaryImpl(path);
errno = e;
return res;
static void FindDebugBinaryInit(void) {
char *p = GetProgramExecutableName();
size_t n = strlen(p);
if ((n > 4 && READ32LE(p + n - 4) == READ32LE(".dbg")) ||
IsMyDebugBinary(p)) {
g_comdbg.res = p;
} else if (n + 4 < ARRAYLEN(g_comdbg.buf)) {
mempcpy(mempcpy(g_comdbg.buf, p, n), ".dbg", 5);
if (IsMyDebugBinary(g_comdbg.buf)) {
g_comdbg.res = g_comdbg.buf;
}
}
if (!g_comdbg.res) {
g_comdbg.res = getenv("COMDBG");
}
}
/**
@ -67,27 +88,6 @@ static bool IsMyDebugBinary(const char *path) {
* @return path to debug binary, or NULL
*/
const char *FindDebugBinary(void) {
static bool once;
static char *res;
static char buf[PATH_MAX];
char *p;
size_t n;
if (!once) {
p = GetProgramExecutableName();
n = strlen(p);
if ((n > 4 && READ32LE(p + n - 4) == READ32LE(".dbg")) ||
IsMyDebugBinary(p)) {
res = p;
} else if (n + 4 < ARRAYLEN(buf)) {
mempcpy(mempcpy(buf, p, n), ".dbg", 5);
if (IsMyDebugBinary(buf)) {
res = buf;
}
}
if (!res) {
res = getenv("COMDBG");
}
once = true;
}
return res;
cosmo_once(&g_comdbg.once, FindDebugBinaryInit);
return g_comdbg.res;
}

454
libc/runtime/fork-nt.c
View file

@ -1,454 +0,0 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
│vi: set net ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/calls/internal.h"
#include "libc/calls/ntspawn.h"
#include "libc/calls/state.internal.h"
#include "libc/calls/struct/fd.internal.h"
#include "libc/calls/syscall-nt.internal.h"
#include "libc/calls/syscall_support-nt.internal.h"
#include "libc/calls/wincrash.internal.h"
#include "libc/errno.h"
#include "libc/fmt/itoa.h"
#include "libc/intrin/atomic.h"
#include "libc/intrin/directmap.internal.h"
#include "libc/intrin/handlock.internal.h"
#include "libc/intrin/kprintf.h"
#include "libc/intrin/strace.internal.h"
#include "libc/intrin/weaken.h"
#include "libc/macros.internal.h"
#include "libc/mem/alloca.h"
#include "libc/mem/mem.h"
#include "libc/nexgen32e/nt2sysv.h"
#include "libc/nt/console.h"
#include "libc/nt/createfile.h"
#include "libc/nt/enum/accessmask.h"
#include "libc/nt/enum/creationdisposition.h"
#include "libc/nt/enum/fileflagandattributes.h"
#include "libc/nt/enum/filemapflags.h"
#include "libc/nt/enum/pageflags.h"
#include "libc/nt/enum/processcreationflags.h"
#include "libc/nt/enum/startf.h"
#include "libc/nt/errors.h"
#include "libc/nt/files.h"
#include "libc/nt/ipc.h"
#include "libc/nt/memory.h"
#include "libc/nt/process.h"
#include "libc/nt/runtime.h"
#include "libc/nt/signals.h"
#include "libc/nt/struct/ntexceptionpointers.h"
#include "libc/nt/synchronization.h"
#include "libc/runtime/internal.h"
#include "libc/runtime/memtrack.internal.h"
#include "libc/runtime/runtime.h"
#include "libc/runtime/symbols.internal.h"
#include "libc/sock/internal.h"
#include "libc/str/str.h"
#include "libc/sysv/consts/limits.h"
#include "libc/sysv/consts/map.h"
#include "libc/sysv/consts/o.h"
#include "libc/sysv/consts/prot.h"
#include "libc/sysv/errfuns.h"
#include "libc/thread/tls.h"
#ifdef __x86_64__
extern int64_t __wincrashearly;
bool32 __onntconsoleevent(uint32_t);
void sys_setitimer_nt_reset(void);
static textwindows wontreturn void AbortFork(const char *func) {
#ifdef SYSDEBUG
kprintf("fork() %s() failed with win32 error %d\n", func, GetLastError());
#endif
ExitProcess(11);
}
static textwindows char16_t *ParseInt(char16_t *p, int64_t *x) {
*x = 0;
while (*p == ' ') p++;
while ('0' <= *p && *p <= '9') {
*x *= 10;
*x += *p++ - '0';
}
return p;
}
static inline textwindows ssize_t ForkIo(int64_t h, char *p, size_t n,
bool32 (*f)()) {
size_t i;
uint32_t x;
for (i = 0; i < n; i += x) {
if (!f(h, p + i, n - i, &x, NULL)) {
return __winerr();
}
}
return i;
}
static dontinline textwindows bool ForkIo2(int64_t h, void *buf, size_t n,
bool32 (*fn)(), const char *sf,
bool ischild) {
ssize_t rc = ForkIo(h, buf, n, fn);
if (ischild) __tls_enabled_set(false); // prevent tls crash in kprintf
NTTRACE("%s(%ld, %p, %'zu) → %'zd% m", sf, h, buf, n, rc);
return rc != -1;
}
static dontinline textwindows bool WriteAll(int64_t h, void *buf, size_t n) {
bool ok;
// kprintf("WriteAll(%ld, %p, %zu);\n", h, buf, n);
ok = ForkIo2(h, buf, n, WriteFile, "WriteFile", false);
#ifndef NDEBUG
if (ok) ok = ForkIo2(h, &n, sizeof(n), WriteFile, "WriteFile", false);
#endif
#ifdef SYSDEBUG
if (!ok) {
kprintf("failed to write %zu bytes to forked child: %d\n", n,
GetLastError());
}
#endif
// Sleep(10);
return ok;
}
static textwindows dontinline void ReadOrDie(int64_t h, void *buf, size_t n) {
// kprintf("ReadOrDie(%ld, %p, %zu);\n", h, buf, n);
if (!ForkIo2(h, buf, n, ReadFile, "ReadFile", true)) {
AbortFork("ReadFile1");
}
#ifndef NDEBUG
size_t got;
if (!ForkIo2(h, &got, sizeof(got), ReadFile, "ReadFile", true)) {
AbortFork("ReadFile2");
}
if (got != n) {
AbortFork("ReadFile_SIZE_CHECK");
}
#endif
// Sleep(10);
}
static textwindows int64_t MapOrDie(uint32_t prot, uint64_t size) {
int64_t h;
for (;;) {
if ((h = CreateFileMapping(-1, 0, prot, size >> 32, size, 0))) {
return h;
}
if (GetLastError() == kNtErrorAccessDenied) {
switch (prot) {
case kNtPageExecuteWritecopy:
prot = kNtPageWritecopy;
continue;
case kNtPageExecuteReadwrite:
prot = kNtPageReadwrite;
continue;
case kNtPageExecuteRead:
prot = kNtPageReadonly;
continue;
default:
break;
}
}
AbortFork("MapOrDie");
}
}
static textwindows void ViewOrDie(int64_t h, uint32_t access, size_t pos,
size_t size, void *base) {
TryAgain:
if (!MapViewOfFileEx(h, access, pos >> 32, pos, size, base)) {
if ((access & kNtFileMapExecute) &&
GetLastError() == kNtErrorAccessDenied) {
access &= ~kNtFileMapExecute;
goto TryAgain;
}
AbortFork("ViewOrDie");
}
}
static textwindows int OnForkCrash(struct NtExceptionPointers *ep) {
kprintf("error: fork() child crashed!%n"
"\tExceptionCode = %#x%n"
"\tRip = %x%n",
ep->ExceptionRecord->ExceptionCode,
ep->ContextRecord ? ep->ContextRecord->Rip : -1);
ExitProcess(11);
}
textwindows void WinMainForked(void) {
jmp_buf jb;
int64_t reader;
int64_t savetsc;
char *addr, *shad;
uint64_t size, upsize;
struct MemoryInterval *maps;
char16_t fvar[21 + 1 + 21 + 1];
uint32_t i, varlen, oldprot, savepid;
long mapcount, mapcapacity, specialz;
struct StdinRelay stdin;
struct Fds *fds = __veil("r", &g_fds);
stdin = fds->stdin;
// check to see if the process was actually forked
// this variable should have the pipe handle numba
varlen = GetEnvironmentVariable(u"_FORK", fvar, ARRAYLEN(fvar));
if (!varlen || varlen >= ARRAYLEN(fvar)) return;
NTTRACE("WinMainForked()");
SetEnvironmentVariable(u"_FORK", NULL);
#ifdef SYSDEBUG
int64_t oncrash = AddVectoredExceptionHandler(1, NT2SYSV(OnForkCrash));
#endif
ParseInt(fvar, &reader);
// read the cpu state from the parent process & plus
// read the list of mappings from the parent process
// this is stored in a special secretive memory map!
// read ExtendMemoryIntervals for further details :|
maps = (void *)kMemtrackStart;
ReadOrDie(reader, jb, sizeof(jb));
ReadOrDie(reader, &mapcount, sizeof(_mmi.i));
ReadOrDie(reader, &mapcapacity, sizeof(_mmi.n));
specialz = ROUNDUP(mapcapacity * sizeof(_mmi.p[0]), kMemtrackGran);
ViewOrDie(MapOrDie(kNtPageReadwrite, specialz), kNtFileMapWrite, 0, specialz,
maps);
ReadOrDie(reader, maps, mapcount * sizeof(_mmi.p[0]));
if (IsAsan()) {
shad = (char *)(((intptr_t)maps >> 3) + 0x7fff8000);
size = ROUNDUP(specialz >> 3, FRAMESIZE);
ViewOrDie(MapOrDie(kNtPageReadwrite, size), kNtFileMapWrite, 0, size, maps);
ReadOrDie(reader, shad, (mapcount * sizeof(_mmi.p[0])) >> 3);
}
// read the heap mappings from the parent process
for (i = 0; i < mapcount; ++i) {
addr = (char *)((uint64_t)maps[i].x << 16);
size = maps[i].size;
if ((maps[i].flags & MAP_TYPE) != MAP_SHARED) {
upsize = ROUNDUP(size, FRAMESIZE);
// we don't need to close the map handle because sys_mmap_nt
// doesn't mark it inheritable across fork() for MAP_PRIVATE
ViewOrDie((maps[i].h = MapOrDie(kNtPageExecuteReadwrite, upsize)),
kNtFileMapWrite | kNtFileMapExecute, 0, upsize, addr);
ReadOrDie(reader, addr, size);
} else {
// we can however safely inherit MAP_SHARED with zero copy
ViewOrDie(maps[i].h,
maps[i].readonlyfile ? kNtFileMapRead | kNtFileMapExecute
: kNtFileMapWrite | kNtFileMapExecute,
maps[i].offset, size, addr);
}
}
// read the .data and .bss program image sections
savepid = __pid;
savetsc = kStartTsc;
ReadOrDie(reader, __data_start, __data_end - __data_start);
ReadOrDie(reader, __bss_start, __bss_end - __bss_start);
__pid = savepid;
kStartTsc = savetsc;
__threaded = false;
__tls_index = 0;
__tls_enabled_set(false);
// apply fixups and reapply memory protections
_mmi.p = maps;
_mmi.n = specialz / sizeof(_mmi.p[0]);
for (i = 0; i < mapcount; ++i) {
if (!VirtualProtect((void *)((uint64_t)maps[i].x << 16), maps[i].size,
__prot2nt(maps[i].prot, maps[i].iscow), &oldprot)) {
AbortFork("VirtualProtect");
}
}
// mitosis complete
if (!CloseHandle(reader)) {
AbortFork("CloseHandle");
}
// rewrap the stdin named pipe hack
// since the handles closed on fork
fds->stdin = stdin;
fds->p[0].handle = GetStdHandle(kNtStdInputHandle);
fds->p[1].handle = GetStdHandle(kNtStdOutputHandle);
fds->p[2].handle = GetStdHandle(kNtStdErrorHandle);
// untrack children of parent since we specify with both
// CreateProcess() and CreateThread() as non-inheritable
for (i = 0; i < fds->n; ++i) {
if (fds->p[i].kind == kFdProcess) {
fds->p[i].kind = 0;
atomic_store_explicit(&fds->f, MIN(i, fds->f), memory_order_relaxed);
}
}
// restore the crash reporting stuff
#ifdef SYSDEBUG
RemoveVectoredExceptionHandler(oncrash);
#endif
if (_weaken(__wincrash)) {
if (!IsTiny()) {
RemoveVectoredExceptionHandler(__wincrashearly);
}
AddVectoredExceptionHandler(1, (void *)_weaken(__wincrash));
}
if (_weaken(__onntconsoleevent)) {
SetConsoleCtrlHandler(_weaken(__onntconsoleevent), 1);
}
// jump back into function below
longjmp(jb, 1);
}
static void __hand_inherit(bool32 bInherit) {
for (int i = 0; i < _mmi.i; ++i) {
if ((_mmi.p[i].flags & MAP_TYPE) == MAP_SHARED) {
SetHandleInformation(_mmi.p[i].h, kNtHandleFlagInherit, bInherit);
}
}
for (int i = 0; i < g_fds.n; ++i) {
if (g_fds.p[i].kind == kFdEmpty) continue;
SetHandleInformation(g_fds.p[i].handle, kNtHandleFlagInherit, bInherit);
if (g_fds.p[i].kind == kFdConsole) {
SetHandleInformation(g_fds.p[i].extra, kNtHandleFlagInherit, bInherit);
}
}
}
textwindows int sys_fork_nt(uint32_t dwCreationFlags) {
jmp_buf jb;
uint32_t op;
char **args;
char ok, threaded;
struct CosmoTib *tib;
char16_t pipename[64];
int64_t reader, writer;
struct NtStartupInfo startinfo;
int i, pid, untrackpid, rc = -1;
char *p, forkvar[6 + 21 + 1 + 21 + 1];
struct NtProcessInformation procinfo;
threaded = __threaded;
tib = __tls_enabled ? __get_tls() : 0;
if (!setjmp(jb)) {
pid = untrackpid = __reservefd_unlocked(-1);
reader = CreateNamedPipe(__create_pipe_name(pipename), kNtPipeAccessInbound,
kNtPipeTypeByte | kNtPipeReadmodeByte, 1, PIPE_BUF,
PIPE_BUF, 0, &kNtIsInheritable);
writer = CreateFile(pipename, kNtGenericWrite, 0, 0, kNtOpenExisting, 0, 0);
if (pid != -1 && reader != -1 && writer != -1) {
p = stpcpy(forkvar, "_FORK=");
p = FormatUint64(p, reader);
bzero(&startinfo, sizeof(startinfo));
startinfo.cb = sizeof(struct NtStartupInfo);
startinfo.dwFlags = kNtStartfUsestdhandles;
startinfo.hStdInput = __getfdhandleactual(0);
startinfo.hStdOutput = __getfdhandleactual(1);
startinfo.hStdError = __getfdhandleactual(2);
args = __argv;
#ifdef SYSDEBUG
// If --strace was passed to this program, then propagate it the
// forked process since the flag was removed by __intercept_flag
if (strace_enabled(0) > 0) {
int n;
char **args2;
for (n = 0; args[n];) ++n;
args2 = alloca((n + 2) * sizeof(char *));
for (i = 0; i < n; ++i) args2[i] = args[i];
args2[i++] = "--strace";
args2[i] = 0;
args = args2;
}
#endif
__hand_inherit(true);
int spawnrc =
ntspawn(GetProgramExecutableName(), args, environ, forkvar, 0, 0,
true, dwCreationFlags, 0, &startinfo, &procinfo);
__hand_inherit(false);
if (spawnrc != -1) {
CloseHandle(procinfo.hThread);
ok = WriteAll(writer, jb, sizeof(jb)) &&
WriteAll(writer, &_mmi.i, sizeof(_mmi.i)) &&
WriteAll(writer, &_mmi.n, sizeof(_mmi.n)) &&
WriteAll(writer, _mmi.p, _mmi.i * sizeof(_mmi.p[0]));
if (IsAsan() && ok) {
ok = WriteAll(writer, (char *)(((intptr_t)_mmi.p >> 3) + 0x7fff8000),
(_mmi.i * sizeof(_mmi.p[0])) >> 3);
}
for (i = 0; i < _mmi.i && ok; ++i) {
if ((_mmi.p[i].flags & MAP_TYPE) != MAP_SHARED) {
char *p = (char *)((uint64_t)_mmi.p[i].x << 16);
// XXX: forking destroys thread guard pages currently
VirtualProtect(
p, _mmi.p[i].size,
__prot2nt(_mmi.p[i].prot | PROT_READ, _mmi.p[i].iscow), &op);
ok = WriteAll(writer, p, _mmi.p[i].size);
}
}
if (ok) ok = WriteAll(writer, __data_start, __data_end - __data_start);
if (ok) ok = WriteAll(writer, __bss_start, __bss_end - __bss_start);
if (ok) {
if (!CloseHandle(writer)) ok = false;
writer = -1;
}
if (ok) {
g_fds.p[pid].kind = kFdProcess;
g_fds.p[pid].handle = procinfo.hProcess;
g_fds.p[pid].flags = O_CLOEXEC;
g_fds.p[pid].zombie = false;
untrackpid = -1;
rc = pid;
} else {
TerminateProcess(procinfo.hProcess, 9);
CloseHandle(procinfo.hProcess);
}
}
}
if (reader != -1) CloseHandle(reader);
if (writer != -1) CloseHandle(writer);
if (rc == -1 && errno != ENOMEM) {
eagain(); // posix fork() only specifies two errors
}
} else {
rc = 0;
// re-apply code morphing for thread-local storage
if (tib && _weaken(__set_tls) && _weaken(__morph_tls)) {
_weaken(__set_tls)(tib);
_weaken(__morph_tls)();
__tls_enabled_set(true);
}
// re-apply code morphing for synchronization nops
if (threaded && !__threaded && _weaken(__enable_threads)) {
_weaken(__enable_threads)();
}
// re-apply code morphing for function tracing
if (ftrace_stackdigs) {
_weaken(__hook)(_weaken(ftrace_hook), _weaken(GetSymbolTable)());
}
// reset alarms
if (_weaken(sys_setitimer_nt_reset)) {
_weaken(sys_setitimer_nt_reset)();
}
}
if (untrackpid != -1) {
__releasefd(untrackpid);
}
return rc;
}
#endif /* __x86_64__ */

71
libc/runtime/fork-sysv.c
View file

@ -1,71 +0,0 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
│vi: set net ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2022 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/calls/syscall-sysv.internal.h"
#include "libc/dce.h"
#include "libc/runtime/syslib.internal.h"
#include "libc/sysv/consts/sig.h"
#include "libc/sysv/errfuns.h"
int sys_fork(void) {
#ifdef __x86_64__
axdx_t ad;
int ax, dx;
ad = __sys_fork();
ax = ad.ax;
dx = ad.dx;
if (IsXnu() && ax != -1) {
// eax always returned with childs pid
// edx is 0 for parent and 1 for child
ax &= dx - 1;
}
return ax;
#elif defined(__aarch64__)
if (IsLinux()) {
int flags = 17; // SIGCHLD
void *child_stack = 0;
void *parent_tidptr = 0;
void *newtls = 0;
void *child_tidptr = 0;
register long r0 asm("x0") = (long)flags;
register long r1 asm("x1") = (long)child_stack;
register long r2 asm("x2") = (long)parent_tidptr;
register long r3 asm("x3") = (long)newtls;
register long r4 asm("x4") = (long)child_tidptr;
register int res_x0 asm("x0");
asm volatile("mov\tx8,%1\n\t"
"svc\t0"
: "=r"(res_x0)
: "i"(220), "r"(r0), "r"(r1), "r"(r2), "r"(r3), "r"(r4)
: "x8", "x16", "memory");
return _sysret(res_x0);
} else if (__syslib) {
return _sysret(__syslib->fork());
} else {
return enosys();
}
#else
return enosys();
#endif
}

91
libc/runtime/fork.c
View file

@ -1,91 +0,0 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
│vi: set net ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/assert.h"
#include "libc/calls/blocksigs.internal.h"
#include "libc/calls/calls.h"
#include "libc/calls/struct/sigset.h"
#include "libc/calls/struct/sigset.internal.h"
#include "libc/calls/syscall-nt.internal.h"
#include "libc/calls/syscall-sysv.internal.h"
#include "libc/dce.h"
#include "libc/intrin/atomic.h"
#include "libc/intrin/strace.internal.h"
#include "libc/intrin/weaken.h"
#include "libc/nt/process.h"
#include "libc/runtime/internal.h"
#include "libc/sysv/consts/sig.h"
#include "libc/thread/posixthread.internal.h"
#include "libc/thread/tls.h"
int _fork(uint32_t dwCreationFlags) {
struct CosmoTib *tib;
struct PosixThread *pt;
int ax, dx, tid, parent;
(void)parent;
BLOCK_SIGNALS;
if (__threaded && _weaken(_pthread_onfork_prepare)) {
_weaken(_pthread_onfork_prepare)();
}
if (!IsWindows()) {
ax = sys_fork();
} else {
ax = sys_fork_nt(dwCreationFlags);
}
if (!ax) {
if (!IsWindows()) {
dx = sys_getpid().ax;
} else {
dx = GetCurrentProcessId();
}
parent = __pid;
__pid = dx;
if (__tls_enabled) {
tib = __get_tls();
tid = IsLinux() || IsXnuSilicon() ? dx : sys_gettid();
atomic_store_explicit(&tib->tib_tid, tid, memory_order_relaxed);
if ((pt = (struct PosixThread *)tib->tib_pthread)) {
atomic_store_explicit(&pt->ptid, tid, memory_order_relaxed);
}
}
if (__threaded && _weaken(_pthread_onfork_child)) {
_weaken(_pthread_onfork_child)();
}
STRACE("fork() → 0 (child of %d)", parent);
} else {
if (__threaded && _weaken(_pthread_onfork_parent)) {
_weaken(_pthread_onfork_parent)();
}
STRACE("fork() → %d% m", ax);
}
ALLOW_SIGNALS;
return ax;
}
/**
* Creates new process.
*
* @return 0 to child, child pid to parent, or -1 w/ errno
* @raise EAGAIN if `RLIMIT_NPROC` was exceeded or system lacked resources
* @raise ENOMEM if we require more vespene gas
* @asyncsignalsafe
* @threadsafe
*/
int fork(void) {
return _fork(0);
}

21
libc/runtime/ftracer.c
View file

@ -26,6 +26,8 @@
#include "libc/runtime/internal.h"
#include "libc/runtime/runtime.h"
#include "libc/runtime/stack.h"
#include "libc/thread/posixthread.internal.h"
#include "libc/thread/thread.h"
#include "libc/thread/tls.h"
#include "libc/thread/tls2.internal.h"
@ -74,30 +76,39 @@ __funline int GetNestingLevel(struct CosmoFtrace *ft, struct StackFrame *sf) {
* @see ftrace_install()
*/
privileged void ftracer(void) {
uintptr_t fn;
long stackuse;
uintptr_t fn, st;
struct CosmoTib *tib;
struct StackFrame *sf;
struct CosmoFtrace *ft;
struct PosixThread *pt;
sf = __builtin_frame_address(0);
st = (uintptr_t)__argv - sizeof(uintptr_t);
if (__ftrace <= 0) return;
if (__tls_enabled) {
tib = __get_tls_privileged();
if (tib->tib_ftrace <= 0) return;
ft = &tib->tib_ftracer;
if ((char *)sf >= tib->tib_sigstack_addr &&
(char *)sf <= tib->tib_sigstack_addr + tib->tib_sigstack_size) {
st = (uintptr_t)tib->tib_sigstack_addr + tib->tib_sigstack_size;
} else if ((pt = (struct PosixThread *)tib->tib_pthread) &&
pt->attr.__stacksize) {
st = (uintptr_t)pt->attr.__stackaddr + pt->attr.__stacksize;
}
} else {
ft = &g_ftrace;
}
stackuse = st - (intptr_t)sf;
if (_cmpxchg(&ft->ft_once, false, true)) {
ft->ft_lastaddr = -1;
ft->ft_skew = GetNestingLevelImpl(__builtin_frame_address(0));
ft->ft_skew = GetNestingLevelImpl(sf);
}
if (_cmpxchg(&ft->ft_noreentry, false, true)) {
sf = __builtin_frame_address(0);
sf = sf->next;
fn = sf->addr + DETOUR_SKEW;
if (fn != ft->ft_lastaddr) {
stackuse = GetStackAddr() + GetStackSize() - (intptr_t)sf;
kprintf("%rFUN %6P %'13T %'*ld %*s%t\n", ftrace_stackdigs, stackuse,
kprintf("%rFUN %6P %'16T %'*ld %*s%t\n", ftrace_stackdigs, stackuse,
GetNestingLevel(ft, sf) * 2, "", fn);
ft->ft_lastaddr = fn;
}

8
libc/runtime/getdosargv.c
View file

@ -35,7 +35,7 @@ struct DosArgv {
wint_t wc;
};
textwindows dontasan void DecodeDosArgv(int ignore, struct DosArgv *st) {
textwindows void DecodeDosArgv(int ignore, struct DosArgv *st) {
wint_t x, y;
for (;;) {
if (!(x = *st->s++)) break;
@ -51,7 +51,7 @@ textwindows dontasan void DecodeDosArgv(int ignore, struct DosArgv *st) {
st->wc = x;
}
static textwindows dontasan void AppendDosArgv(wint_t wc, struct DosArgv *st) {
static textwindows void AppendDosArgv(wint_t wc, struct DosArgv *st) {
uint64_t w;
w = tpenc(wc);
do {
@ -60,7 +60,7 @@ static textwindows dontasan void AppendDosArgv(wint_t wc, struct DosArgv *st) {
} while (w >>= 8);
}
static textwindows dontasan int Count(int c, struct DosArgv *st) {
static textwindows int Count(int c, struct DosArgv *st) {
int ignore, n = 0;
asm("" : "=g"(ignore));
while (st->wc == c) {
@ -87,7 +87,7 @@ static textwindows dontasan int Count(int c, struct DosArgv *st) {
// @see test/libc/dosarg_test.c
// @see libc/runtime/ntspawn.c
// @note kudos to Simon Tatham for figuring out quoting behavior
textwindows dontasan int GetDosArgv(const char16_t *cmdline, char *buf,
textwindows int GetDosArgv(const char16_t *cmdline, char *buf,
size_t size, char **argv, size_t max) {
bool inquote;
int i, argc, slashes, quotes, ignore;

16
libc/runtime/getdosenviron.c
View file

@ -21,13 +21,15 @@
#include "libc/str/str.h"
#include "libc/str/utf16.h"
#define abi textwindows dontinstrument
#define ToUpper(c) ((c) >= 'a' && (c) <= 'z' ? (c) - 'a' + 'A' : (c))
forceinline int IsAlpha(int c) {
__funline int IsAlpha(int c) {
return ('A' <= c && c <= 'Z') || ('a' <= c && c <= 'z');
}
forceinline char *MemChr(const char *s, unsigned char c, unsigned long n) {
__funline char *MemChr(const char *s, unsigned char c, unsigned long n) {
for (; n; --n, ++s) {
if ((*s & 255) == c) {
return (char *)s;
@ -36,8 +38,7 @@ forceinline char *MemChr(const char *s, unsigned char c, unsigned long n) {
return 0;
}
static textwindows dontasan dontinstrument axdx_t
Recode16to8(char *dst, size_t dstsize, const char16_t *src) {
static abi axdx_t Recode16to8(char *dst, size_t dstsize, const char16_t *src) {
bool v;
axdx_t r;
uint64_t w;
@ -70,7 +71,7 @@ Recode16to8(char *dst, size_t dstsize, const char16_t *src) {
return r;
}
textwindows dontinstrument dontasan void FixPath(char *path) {
static abi void FixPath(char *path) {
char *p;
// turn backslash into slash
@ -109,9 +110,8 @@ textwindows dontinstrument dontasan void FixPath(char *path) {
// @param envp stores NULL-terminated string pointer list (optional)
// @param max is the pointer count capacity of envp
// @return number of variables decoded, excluding NULL-terminator
textwindows dontasan dontinstrument int GetDosEnviron(const char16_t *env,
char *buf, size_t size,
char **envp, size_t max) {
abi int GetDosEnviron(const char16_t *env, char *buf, size_t size, char **envp,
size_t max) {
int i;
char *p;
axdx_t r;

2
libc/runtime/getmemtracksize.c
View file

@ -18,7 +18,7 @@
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/runtime/memtrack.internal.h"
dontasan size_t __get_memtrack_size(struct MemoryIntervals *mm) {
size_t __get_memtrack_size(struct MemoryIntervals *mm) {
size_t i, n;
for (n = i = 0; i < mm->i; ++i) {
n += ((size_t)(mm->p[i].y - mm->p[i].x) + 1) << 16;

2
libc/runtime/getsymbolbyaddr.c
View file

@ -23,7 +23,7 @@
/**
* Returns name of symbol at address.
*/
dontasan char *GetSymbolByAddr(int64_t addr) {
char *GetSymbolByAddr(int64_t addr) {
/* asan runtime depends on this function */
int i;
struct SymbolTable *st;

4
libc/runtime/internal.h
View file

@ -44,8 +44,8 @@ int GetDosEnviron(const char16_t *, char *, size_t, char **, size_t);
bool __intercept_flag(int *, char *[], const char *);
int sys_mprotect_nt(void *, size_t, int);
int __inflate(void *, size_t, const void *, size_t);
void *__mmap_unlocked(void *, size_t, int, int, int, int64_t) dontasan;
int __munmap_unlocked(char *, size_t) dontasan;
void *__mmap_unlocked(void *, size_t, int, int, int, int64_t);
int __munmap_unlocked(char *, size_t);
void __on_arithmetic_overflow(void);
void __init_fds(int, char **, char **);

2
libc/runtime/isheap.c
View file

@ -26,7 +26,7 @@
* @assume stack addresses are always greater than heap addresses
* @assume stack memory isn't stored beneath %rsp (-mno-red-zone)
*/
optimizesize dontasan bool _isheap(void *p) {
optimizesize bool _isheap(void *p) {
intptr_t x, y;
x = kAutomapStart;
y = x + kAutomapSize;

78
libc/runtime/enable_threads.c → libc/runtime/isstackoverflow.c
View file

@ -1,7 +1,7 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
│vi: set net ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2022 Justine Alexandra Roberts Tunney │
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
@ -16,48 +16,48 @@
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "ape/sections.internal.h"
#include "libc/calls/syscall-sysv.internal.h"
#include "libc/intrin/strace.internal.h"
#include "libc/calls/struct/siginfo.h"
#include "libc/calls/struct/ucontext.internal.h"
#include "libc/calls/ucontext.h"
#include "libc/intrin/kprintf.h"
#include "libc/runtime/runtime.h"
#include "libc/sysv/consts/auxv.h"
#include "libc/sysv/consts/sig.h"
#include "libc/thread/posixthread.internal.h"
#include "libc/thread/thread.h"
#include "libc/thread/tls.h"
extern int __threadcalls_end[] __attribute__((__weak__));
extern int __threadcalls_start[] __attribute__((__weak__));
/**
* Returns true if signal is likely a stack overflow.
*/
char __is_stack_overflow(siginfo_t *si, void *ucontext) {
static privileged void FixupLockNops(void) {
__morph_begin();
/*
* _NOPL("__threadcalls", func)
*
* The big ugly macro above is used by Cosmopolitan Libc to unser
* locking primitive (e.g. flockfile, funlockfile) have zero impact on
* performance and binary size when threads aren't actually in play.
*
* we have this
*
* 0f 1f 05 b1 19 00 00 nopl func(%rip)
*
* we're going to turn it into this
*
* 67 67 e8 b1 19 00 00 addr32 addr32 call func
*
* This is cheap and fast because the big ugly macro stored in the
* binary the offsets of all the instructions we need to change.
*/
for (int *p = __threadcalls_start; p < __threadcalls_end; ++p) {
__executable_start[*p + 0] = 0x67;
__executable_start[*p + 1] = 0x67;
__executable_start[*p + 2] = 0xe8;
// check if signal has the information we need
ucontext_t *uc = ucontext;
if (!si) return false;
if (!uc) return false;
if (si->si_signo != SIGSEGV && si->si_signo != SIGBUS) return false;
// with threads we know exactly where the guard page is
int pagesz = getauxval(AT_PAGESZ);
uintptr_t addr = (uintptr_t)si->si_addr;
struct PosixThread *pt = _pthread_self();
if (pt->attr.__stacksize) {
uintptr_t stack = (uintptr_t)pt->attr.__stackaddr;
uintptr_t guard = pt->attr.__guardsize;
uintptr_t bot, top;
if (guard) {
bot = stack;
top = bot + guard;
} else {
bot = stack - pagesz;
top = stack;
}
return addr >= bot && addr < top;
}
__morph_end();
}
void __enable_threads(void) {
if (__threaded) return;
#ifdef __x86_64__
STRACE("__enable_threads()");
FixupLockNops();
#endif
__threaded = __tls_enabled ? __get_tls()->tib_tid : sys_gettid();
// it's easy to guess with the main stack
// even though it's hard to know its exact boundaries
uintptr_t sp = uc->uc_mcontext.SP;
return addr <= sp && addr >= sp - pagesz;
}

2
libc/runtime/memtrack.internal.h
View file

@ -43,10 +43,8 @@ struct MemoryIntervals {
extern struct MemoryIntervals _mmi;
void __mmi_init(void);
void __mmi_lock(void);
void __mmi_unlock(void);
void __mmi_funlock(void);
bool IsMemtracked(int, int);
void PrintSystemMappings(int);
unsigned __find_memory(const struct MemoryIntervals *, int) nosideeffect;

4
libc/runtime/memtracknt.c
View file

@ -23,14 +23,14 @@
#include "libc/runtime/memtrack.internal.h"
#include "libc/runtime/runtime.h"
static inline dontasan void *GetFrameAddr(int f) {
static inline void *GetFrameAddr(int f) {
intptr_t a;
a = f;
a *= FRAMESIZE;
return (void *)a;
}
dontasan void __release_memory_nt(struct MemoryIntervals *mm, int l, int r) {
void __release_memory_nt(struct MemoryIntervals *mm, int l, int r) {
int i;
for (i = l; i <= r; ++i) {
UnmapViewOfFile(GetFrameAddr(mm->p[i].x));

45
libc/runtime/mmap.c
View file

@ -77,7 +77,7 @@ static wontreturn void __mmap_die(const char *s) {
_Exit(199);
}
static dontasan inline bool __overlaps_existing_mapping(char *p, size_t n) {
static inline bool __overlaps_existing_mapping(char *p, size_t n) {
int a, b, i;
unassert(n > 0);
a = FRAME(p);
@ -91,7 +91,7 @@ static dontasan inline bool __overlaps_existing_mapping(char *p, size_t n) {
return false;
}
static dontasan bool __choose_memory(int x, int n, int align, int *res) {
static bool __choose_memory(int x, int n, int align, int *res) {
// TODO: improve performance
int i, start, end;
unassert(align > 0);
@ -152,12 +152,12 @@ static dontasan bool __choose_memory(int x, int n, int align, int *res) {
return false;
}
dontasan static bool __auto_map(int count, int align, int *res) {
static bool __auto_map(int count, int align, int *res) {
return __choose_memory(FRAME(kAutomapStart), count, align, res) &&
*res + count <= FRAME(kAutomapStart + (kAutomapSize - 1));
}
static dontasan void *__finish_memory(void *addr, size_t size, int prot,
static void *__finish_memory(void *addr, size_t size, int prot,
int flags, int fd, int64_t off, int f,
int x, int n, struct DirectMap dm) {
if (!IsWindows() && (flags & MAP_FIXED)) {
@ -178,7 +178,7 @@ static dontasan void *__finish_memory(void *addr, size_t size, int prot,
return addr;
}
static dontasan void *__map_memory(void *addr, size_t size, int prot, int flags,
static void *__map_memory(void *addr, size_t size, int prot, int flags,
int fd, int64_t off, int f, int x, int n) {
struct DirectMap dm;
dm = sys_mmap(addr, size, prot, f, fd, off);
@ -200,7 +200,7 @@ static dontasan void *__map_memory(void *addr, size_t size, int prot, int flags,
* This is useful on Windows since it allows us to partially unmap or
* punch holes into existing mappings.
*/
static textwindows dontinline dontasan void *__map_memories(
static textwindows dontinline void *__map_memories(
char *addr, size_t size, int prot, int flags, int fd, int64_t off, int f,
int x, int n) {
size_t i, m;
@ -238,14 +238,13 @@ static textwindows dontinline dontasan void *__map_memories(
return addr;
}
dontasan inline void *__mmap_unlocked(void *addr, size_t size, int prot,
inline void *__mmap_unlocked(void *addr, size_t size, int prot,
int flags, int fd, int64_t off) {
int a, f, n, x;
char *p = addr;
struct DirectMap dm;
size_t requested_size;
bool needguard, clashes;
unsigned long page_size;
int a, f, n, x, pagesize;
size_t virtualused, virtualneed;
if (VERY_UNLIKELY(!size)) {
@ -273,7 +272,7 @@ dontasan inline void *__mmap_unlocked(void *addr, size_t size, int prot,
}
requested_size = size;
page_size = getauxval(AT_PAGESZ);
pagesize = getauxval(AT_PAGESZ);
if (flags & MAP_ANONYMOUS) {
fd = -1;
off = 0;
@ -288,7 +287,7 @@ dontasan inline void *__mmap_unlocked(void *addr, size_t size, int prot,
} else if (VERY_UNLIKELY(off < 0)) {
STRACE("mmap negative offset");
return VIP(einval());
} else if (off & ((IsWindows() ? FRAMESIZE : page_size) - 1)) {
} else if (off & ((IsWindows() ? FRAMESIZE : pagesize) - 1)) {
STRACE("mmap offset isn't properly aligned");
return VIP(einval());
} else if (VERY_UNLIKELY(INT64_MAX - size < off)) {
@ -358,31 +357,21 @@ dontasan inline void *__mmap_unlocked(void *addr, size_t size, int prot,
// starting page and the bottom guard page, since that would stop
// our stack page from growing down.
npassert(!sys_munmap(p, size));
// by default MAP_GROWSDOWN will auto-allocate 10mb of pages. it's
// supposed to stop growing if an adjacent allocation exists, to
// prevent your stacks from overlapping on each other. we're not
// able to easily assume a mapping beneath this one exists. even
// if we could, the linux kernel requires for muh security reasons
// that stacks be at least 1mb away from each other, so it's not
// possible to avoid this call if our goal is to have 60kb stacks
// with 4kb guards like a sane multithreaded production system.
// however this 1mb behavior oddly enough is smart enough to not
// apply if the mapping is a manually-created guard page.
int e = errno;
if ((dm = sys_mmap(p + size - SIGSTKSZ, SIGSTKSZ, prot,
int guardsize = pagesize, e = errno;
if ((dm = sys_mmap(p + size - guardsize, guardsize, prot,
f | MAP_GROWSDOWN_linux, fd, off))
.addr != MAP_FAILED) {
npassert(sys_mmap(p, GetGuardSize(), PROT_NONE,
npassert(sys_mmap(p, pagesize, PROT_NONE,
MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0)
.addr == p);
dm.addr = p;
p = __finish_memory(p, size, prot, flags, fd, off, f, x, n, dm);
if (IsAsan() && p != MAP_FAILED) {
__asan_poison(p, GetGuardSize(), kAsanStackOverflow);
__asan_poison(p, pagesize, kAsanStackOverflow);
}
return p;
} else if (errno == ENOTSUP) {
// WSL doesn't support MAP_GROWSDOWN
// WSL and Blink don't support MAP_GROWSDOWN
needguard = true;
errno = e;
}
@ -406,9 +395,9 @@ dontasan inline void *__mmap_unlocked(void *addr, size_t size, int prot,
kAsanMmapSizeOverrun);
}
if (needguard) {
unassert(!mprotect(p, page_size, PROT_NONE));
unassert(!mprotect(p, pagesize, PROT_NONE));
if (IsAsan()) {
__asan_poison(p, page_size, kAsanStackOverflow);
__asan_poison(p, pagesize, kAsanStackOverflow);
}
}
}

2
libc/runtime/msync-nt.c
View file

@ -25,7 +25,7 @@
#include "libc/sysv/consts/o.h"
#include "libc/sysv/consts/prot.h"
dontasan textwindows int sys_msync_nt(char *addr, size_t size, int flags) {
textwindows int sys_msync_nt(char *addr, size_t size, int flags) {
int i, rc = 0;
char *a, *b, *x, *y;
__mmi_lock();

6
libc/runtime/munmap.c
View file

@ -39,7 +39,7 @@
#define ALIGNED(p) (!(IP(p) & (FRAMESIZE - 1)))
#define FRAME(x) ((int)((intptr_t)(x) >> 16))
static dontasan void __munmap_shadow(char *p, size_t n) {
static void __munmap_shadow(char *p, size_t n) {
intptr_t a, b, x, y;
KERNTRACE("__munmap_shadow(%p, %'zu)", p, n);
a = ((intptr_t)p >> 3) + 0x7fff8000;
@ -66,7 +66,7 @@ static dontasan void __munmap_shadow(char *p, size_t n) {
// our api supports doing things like munmap(0, 0x7fffffffffff) but some
// platforms (e.g. openbsd) require that we know the specific intervals
// or else it returns EINVAL. so we munmap a piecewise.
static dontasan void __munmap_impl(char *p, size_t n) {
static void __munmap_impl(char *p, size_t n) {
char *q;
size_t m;
intptr_t a, b, c;
@ -112,7 +112,7 @@ static dontasan void __munmap_impl(char *p, size_t n) {
}
}
dontasan int __munmap_unlocked(char *p, size_t n) {
int __munmap_unlocked(char *p, size_t n) {
unassert(!__vforked);
if (UNLIKELY(!n)) {
STRACE("munmap n is 0");

28
libc/runtime/printargs.c
View file

@ -82,7 +82,7 @@ static const char *FindNameById(const struct IdName *names, unsigned long id) {
return NULL;
}
static dontasan void PrintDependencies(const char *prologue) {
static void PrintDependencies(const char *prologue) {
struct NtLinkedList *head = &NtGetPeb()->Ldr->InLoadOrderModuleList;
struct NtLinkedList *ldr = head->Next;
do {
@ -93,10 +93,10 @@ static dontasan void PrintDependencies(const char *prologue) {
} while ((ldr = ldr->Next) && ldr != head);
}
static dontasan void Print(const char *prologue) {
static void Print(const char *prologue) {
}
static dontasan const char *ConvertCcToStr(int cc) {
static const char *ConvertCcToStr(int cc) {
if (cc == _POSIX_VDISABLE) {
return "_POSIX_VDISABLE";
} else {
@ -115,7 +115,16 @@ static dontasan const char *ConvertCcToStr(int cc) {
*
* @param prologue needs to be a .rodata kprintf string
*/
dontasan textstartup void __printargs(const char *prologue) {
textstartup void __printargs(const char *prologue) {
#pragma GCC push_options
#pragma GCC diagnostic ignored "-Wframe-larger-than="
union {
char path[PATH_MAX];
struct pollfd pfds[128];
} u;
CheckLargeStackAllocation(&u, sizeof(u));
#pragma GCC pop_options
const struct AuxiliaryValue {
const char *fmt;
@ -172,10 +181,6 @@ dontasan textstartup void __printargs(const char *prologue) {
struct sched_param sp;
struct termios termios;
const struct AuxiliaryValue *auxinfo;
union {
char path[PATH_MAX];
struct pollfd pfds[128];
} u;
(void)x;
@ -275,7 +280,9 @@ dontasan textstartup void __printargs(const char *prologue) {
if (X86_HAVE(LA57)) kprintf(" LA57");
if (X86_HAVE(FSGSBASE)) kprintf(" FSGSBASE");
#elif defined(__aarch64__)
PRINT(" AARCH64");
kprintf(" AARCH64\n");
#else
kprintf("\n");
#endif
PRINT("");
@ -391,6 +398,7 @@ dontasan textstartup void __printargs(const char *prologue) {
PRINT(" ☼ %p __oldstack ptr", __oldstack);
PRINT(" ☼ %p __oldstack bot", ROUNDDOWN(__oldstack, foss_stack_size));
PRINT(" ☼ %p __builtin_frame_address(0)", __builtin_frame_address(0));
PRINT(" ☼ %p GetStackPointer()", GetStackPointer());
PRINT("");
PRINT("ARGUMENTS (%p)", __argv);
@ -448,7 +456,7 @@ dontasan textstartup void __printargs(const char *prologue) {
PRINT(" ☼ %s = %d", "geteuid()", geteuid());
PRINT(" ☼ %s = %d", "getgid()", getgid());
PRINT(" ☼ %s = %d", "getegid()", getegid());
PRINT(" ☼ %s = %#s", "kTmpPath", kTmpPath);
PRINT(" ☼ %s = %#s", "__get_tmpdir()", __get_tmpdir());
#ifdef __x86_64__
PRINT(" ☼ %s = %#s", "kNtSystemDirectory", kNtSystemDirectory);
PRINT(" ☼ %s = %#s", "kNtWindowsDirectory", kNtWindowsDirectory);

5
libc/runtime/runtime.h
View file

@ -79,7 +79,6 @@ extern int __strace;
extern int __ftrace;
extern uint64_t __syscount;
extern uint64_t kStartTsc;
extern char kTmpPath[];
extern const char kNtSystemDirectory[];
extern const char kNtWindowsDirectory[];
extern size_t __virtualmax;
@ -138,6 +137,10 @@ int __get_arg_max(void) pureconst;
int __get_cpu_count(void) pureconst;
long __get_avphys_pages(void) pureconst;
long __get_phys_pages(void) pureconst;
long __get_minsigstksz(void) pureconst;
void __get_main_stack(void **, size_t *, int *);
long __get_safe_size(long, long);
char *__get_tmpdir(void);
#endif /* _COSMO_SOURCE */
COSMOPOLITAN_C_END_

60
libc/runtime/runtime.mk
View file

@ -68,43 +68,11 @@ $(LIBC_RUNTIME_A).pkg: \
o/$(MODE)/libc/runtime/cosmo2.o: private \
CFLAGS += -O0
o/$(MODE)/libc/runtime/ftracer.o: private \
CFLAGS += \
-x-no-pg \
-ffreestanding \
-fno-sanitize=all
o/$(MODE)/libc/runtime/cosmo2.o \
o/$(MODE)/libc/runtime/fork-nt.o \
o/$(MODE)/libc/runtime/enable_tls.o \
o/$(MODE)/libc/runtime/printmemoryintervals.o \
o/$(MODE)/libc/runtime/findmemoryinterval.o \
o/$(MODE)/libc/runtime/sys_mprotect.greg.o \
o/$(MODE)/libc/runtime/getdosargv.o \
o/$(MODE)/libc/runtime/getdosenviron.o \
o/$(MODE)/libc/runtime/hook.greg.o \
o/$(MODE)/libc/runtime/ismemtracked.greg.o \
o/$(MODE)/libc/runtime/memtracknt.o \
o/$(MODE)/libc/runtime/memtrack.greg.o \
o/$(MODE)/libc/runtime/metalprintf.greg.o \
o/$(MODE)/libc/runtime/printargs.greg.o \
o/$(MODE)/libc/runtime/mman.greg.o \
o/$(MODE)/libc/runtime/print.greg.o \
o/$(MODE)/libc/runtime/stackchkfail.o \
o/$(MODE)/libc/runtime/stackchkfaillocal.o \
o/$(MODE)/libc/runtime/winmain.greg.o \
o/$(MODE)/libc/runtime/interceptflag.greg.o \
o/$(MODE)/libc/runtime/opensymboltable.o: private \
CFLAGS += \
-Os \
-ffreestanding \
$(NO_MAGIC)
# must use alloca()
# can't use asan or any runtime services
o/$(MODE)/libc/runtime/fork-nt.o: private \
CPPFLAGS += \
-DSTACK_FRAME_UNLIMITED
$(LIBC_RUNTIME_A_OBJS): private \
COPTS += \
-fno-sanitize=all \
-Wframe-larger-than=4096 \
-Walloca-larger-than=4096
o/$(MODE)/libc/runtime/qsort.o: private \
CFLAGS += \
@ -125,22 +93,6 @@ o/$(MODE)/libc/runtime/enable_tls.o: private \
-mcmodel=large
endif
# privileged functions
o/$(MODE)/libc/runtime/morph.o \
o/$(MODE)/libc/runtime/getsymbol.o \
o/$(MODE)/libc/runtime/enable_threads.o \
o/$(MODE)/libc/runtime/morph_tls.o: private \
CFLAGS += \
-ffreestanding \
-fno-sanitize=all \
-fno-stack-protector
# TODO(jart): We need a way to avoid WinThreadEntry() being hooked.
o/$(MODE)/libc/runtime/clone.o: private \
COPTS += \
-fno-sanitize=all \
-fpatchable-function-entry=0,0
o/$(MODE)/libc/runtime/.cosmo.zip.o: private \
ZIPOBJ_FLAGS += \
-B
@ -150,8 +102,6 @@ o/$(MODE)/libc/runtime/init.o: libc/runtime/init.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/libc/runtime/wipe.o: libc/runtime/wipe.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/libc/runtime/vfork.o: libc/runtime/vfork.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/libc/runtime/clone-linux.o: libc/runtime/clone-linux.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/libc/runtime/ftrace-hook.o: libc/runtime/ftrace-hook.S

1
libc/runtime/set_tls.c
View file

@ -34,7 +34,6 @@ textstartup void __set_tls(struct CosmoTib *tib) {
#ifdef __x86_64__
// ask the operating system to change the x86 segment register
if (IsWindows()) {
__tls_index = TlsAlloc();
npassert(0 <= __tls_index && __tls_index < 64);
asm("mov\t%1,%%gs:%0" : "=m"(*((long *)0x1480 + __tls_index)) : "r"(tib));
} else if (IsFreebsd()) {

23
libc/runtime/stack.h
View file

@ -5,8 +5,6 @@
/**
* Returns preferred size and alignment of thread stack.
*
* This will always be equal to `PTHREAD_STACK_MIN`.
*/
#define GetStackSize() 262144
@ -73,8 +71,7 @@ extern char ape_stack_align[] __attribute__((__weak__));
* process too, then you'll need STATIC_STACK_ALIGN(GetStackSize())
* which will burn O(256kb) of memory to ensure thread invariants.
*/
#define GetStackAddr() \
(((intptr_t)__builtin_frame_address(0) - 1) & -GetStackSize())
#define GetStackAddr() ((GetStackPointer() - 1) & -GetStackSize())
#define GetStaticStackSize() ((uintptr_t)ape_stack_memsz)
@ -86,9 +83,8 @@ extern char ape_stack_align[] __attribute__((__weak__));
* which will burn O(256kb) of memory to ensure thread invariants,
* which make this check exceedingly fast.
*/
#define HaveStackMemory(n) \
((intptr_t)__builtin_frame_address(0) >= \
GetStackAddr() + GetGuardSize() + (n))
#define HaveStackMemory(n) \
(GetStackPointer() >= GetStackAddr() + GetGuardSize() + (n))
/**
* Extends stack memory by poking large allocations.
@ -146,6 +142,19 @@ int FreeCosmoStack(void *) libcesque;
#define GetStaticStackAddr(ADDEND) (GetStackAddr() + ADDEND)
#endif
#define GetStackPointer() \
({ \
uintptr_t __sp; \
__asm__(__mov_sp : "=r"(__sp)); \
__sp; \
})
#ifdef __x86_64__
#define __mov_sp "mov\t%%rsp,%0"
#elif defined(__aarch64__)
#define __mov_sp "mov\t%0,sp"
#endif
COSMOPOLITAN_C_END_
#endif /* GNU ELF */
#endif /* _COSMO_SOURCE */

7
libc/runtime/sysconf.c
View file

@ -30,6 +30,7 @@
#include "libc/sysv/consts/_posix.h"
#include "libc/sysv/consts/limits.h"
#include "libc/sysv/consts/rlimit.h"
#include "libc/sysv/consts/ss.h"
#include "libc/sysv/errfuns.h"
#include "libc/thread/thread.h"
@ -40,6 +41,8 @@
*
* - `_SC_CLK_TCK` returns number of clock ticks per second
* - `_SC_ARG_MAX` will perform expensive rlimit calculations
* - `_SC_SIGSTKSZ` returns host platform's preferred SIGSTKSZ
* - `_SC_MINSIGSTKSZ` returns host platform's required MINSIGSTKSZ
* - `_SC_PAGESIZE` currently always returns 65536 due to Windows
* - `_SC_AVPHYS_PAGES` returns average physical memory pages
* - `_SC_PHYS_PAGES` returns physical memory pages available
@ -58,6 +61,10 @@ long sysconf(int name) {
return FRAMESIZE;
case _SC_ARG_MAX:
return __get_arg_max();
case _SC_SIGSTKSZ:
return _SIGSTKSZ;
case _SC_MINSIGSTKSZ:
return __get_minsigstksz();
case _SC_CHILD_MAX:
return __get_rlimit(RLIMIT_NPROC);
case _SC_OPEN_MAX:

2
libc/runtime/sysconf.h
View file

@ -142,6 +142,8 @@
#define _SC_XOPEN_STREAMS 246
#define _SC_THREAD_ROBUST_PRIO_INHERIT 247
#define _SC_THREAD_ROBUST_PRIO_PROTECT 248
#define _SC_SIGSTKSZ 249
#define _SC_MINSIGSTKSZ 250
#if !(__ASSEMBLER__ + __LINKER__ + 0)
COSMOPOLITAN_C_START_

58
libc/runtime/syslib.internal.h
View file

@ -1,9 +1,5 @@
#ifndef COSMOPOLITAN_LIBC_RUNTIME_SYSLIB_H_
#define COSMOPOLITAN_LIBC_RUNTIME_SYSLIB_H_
#include "libc/calls/struct/sigaction.h"
#include "libc/calls/struct/sigset.h"
#include "libc/calls/struct/timespec.h"
#include "libc/thread/thread.h"
#if !(__ASSEMBLER__ + __LINKER__ + 0)
COSMOPOLITAN_C_START_
@ -16,35 +12,43 @@ COSMOPOLITAN_C_START_
*/
#define SYSLIB_MAGIC ('s' | 'l' << 8 | 'i' << 16 | 'b' << 24)
#define SYSLIB_VERSION 2
#define SYSLIB_VERSION 3
typedef uint64_t dispatch_time_t;
typedef uint64_t dispatch_semaphore_t;
struct Syslib {
int magic;
int version;
long (*fork)(void);
long (*pipe)(int[2]);
long (*clock_gettime)(int, struct timespec *);
long (*nanosleep)(const struct timespec *, struct timespec *);
long (*mmap)(void *, size_t, int, int, int, int64_t);
int (*pthread_jit_write_protect_supported_np)(void);
void (*pthread_jit_write_protect_np)(int);
void (*sys_icache_invalidate)(void *, size_t);
int (*pthread_create)(pthread_t *, const pthread_attr_t *, void *(*)(void *),
void *);
void (*pthread_exit)(void *);
int (*pthread_kill)(pthread_t, int);
int (*pthread_sigmask)(int, const sigset_t *restrict, sigset_t *restrict);
int (*pthread_setname_np)(const char *);
dispatch_semaphore_t (*dispatch_semaphore_create)(long);
long (*dispatch_semaphore_signal)(dispatch_semaphore_t);
long (*dispatch_semaphore_wait)(dispatch_semaphore_t, dispatch_time_t);
dispatch_time_t (*dispatch_walltime)(const struct timespec *, int64_t);
int __magic;
int __version;
long (*__fork)(void);
long (*__pipe)(int[2]);
long (*__clock_gettime)(int, void *);
long (*__nanosleep)(const void *, void *);
long (*__mmap)(void *, size_t, int, int, int, int64_t);
int (*__pthread_jit_write_protect_supported_np)(void);
void (*__pthread_jit_write_protect_np)(int);
void (*__sys_icache_invalidate)(void *, size_t);
int (*__pthread_create)(void *, const void *, void *(*)(void *), void *);
void (*__pthread_exit)(void *);
int (*__pthread_kill)(long, int);
int (*__pthread_sigmask)(int, const void *restrict, void *restrict);
int (*__pthread_setname_np)(const char *);
dispatch_semaphore_t (*__dispatch_semaphore_create)(long);
long (*__dispatch_semaphore_signal)(dispatch_semaphore_t);
long (*__dispatch_semaphore_wait)(dispatch_semaphore_t, dispatch_time_t);
dispatch_time_t (*__dispatch_walltime)(const void *, int64_t);
/* v2 (2023-09-10) */
long (*pthread_self)(void);
void (*dispatch_release)(dispatch_semaphore_t);
long (*__pthread_self)(void);
void (*__dispatch_release)(dispatch_semaphore_t);
int (*__raise)(int);
int (*__pthread_join)(long, void **);
void (*__pthread_yield_np)(void);
int __pthread_stack_min;
int __sizeof_pthread_attr_t;
int (*__pthread_attr_init)(void *);
int (*__pthread_attr_destroy)(void *);
int (*__pthread_attr_setstacksize)(void *, size_t);
int (*__pthread_attr_setguardsize)(void *, size_t);
};
extern struct Syslib *__syslib;

144
libc/runtime/vfork.S
View file

@ -1,144 +0,0 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/dce.h"
#include "libc/intrin/strace.internal.h"
#include "libc/thread/tls.h"
#include "libc/macros.internal.h"
// Forks process without copying page tables.
//
// This is the same as fork() except it's optimized for the case
// where the caller invokes execve() immediately afterwards. You
// can also call functions like close(), dup2(), etc. Call _exit
// but don't call exit. Look for vforksafe function annotations,
// For example pthread mutexes are @vforksafe because they don't
// do anything in a vfork()'d child process. TLS memory must not
// be disabled (it's enabled by default) since vfork() needs it.
//
// What makes vfork() dangerous is that any changes to memory in
// the child process can happen in the parent too. The exception
// to this rule is `errno` which is saved/restored in a register
// by this implementation. However, despite its dangers, vfork's
// performance is irresistible and wonderous to behold. If safer
// code is desired, consider posix_spawn() which uses vfork().
//
// Do not make the assumption that the parent is suspended until
// the child terminates since this uses the raw fork system call
// on Windows, OpenBSD, and MacOS. In that case the process will
// proceed without blocking the parent; however, the `__vforked`
// variable is still set to true in the child, so lock functions
// won't do anything, and other functions shall change behavior.
// This ensures that, even if the operating system does not give
// us the performance of vfork(), we'll still be able to cut out
// the libc overhead, e.g. pthread_atfork().
//
// @return pid of child process or 0 if forked process
// @returnstwice
// @threadsafe
// @vforksafe
.ftrace1
vfork:
.ftrace2
#ifdef __SANITIZE_ADDRESS__
jmp fork
#endif
#ifdef __x86_64__
#if SupportsWindows()
// these platforms disagree with vfork
testb $_HOSTXNU|_HOSTOPENBSD|_HOSTWINDOWS,__hostos(%rip)
jnz fork
#endif
#if !IsTiny()
push %rbp
mov %rsp,%rbp
call __require_tls
#ifdef SYSDEBUG
ezlea .Llog,di
call __stracef
#endif
pop %rbp
#endif
mov %fs:0,%r9 // get thread information block
mov 0x3c(%r9),%r8d // avoid question of @vforksafe errno
pop %rsi // saves return address in a register
mov __NR_vfork(%rip),%eax
#if SupportsBsd()
clc
#endif
syscall
#if SupportsBsd()
jnc 0f
neg %rax
0:
#endif
push %rsi // note it happens twice in same page
cmp $-4095,%eax
jae systemfive_error
mov %r8d,0x3c(%r9) // restore errno
1: test %eax,%eax
jnz .Lpar
.Lchi: orb $TIB_FLAG_VFORKED,0x40(%r9)
ret
.Lpar: andb $~TIB_FLAG_VFORKED,0x40(%r9)
ret
#elif defined(__aarch64__)
adrp x0,__hostos
ldr w0,[x0,#:lo12:__hostos]
tbz x0,3,1f // bit 3 is xnu
b fork // which doesn't support vfork()
1: mov x8,#220 // __NR_clone
mov x0,#0x4111 // SIGCHLD | CLONE_VM | CLONE_VFORK
mov x1,#0
svc 0
// if (!rc) {
// __get_tls()->tib_flags |= TIB_FLAG_VFORKED;
// } else {
// __get_tls()->tib_flags &= ~TIB_FLAG_VFORKED;
// }
sub x1,x28,#1152
ldr x2,[x1,0x40]
cbnz x0,2f
orr x2,x2,#TIB_FLAG_VFORKED
1: str x2,[x1,0x40]
b 3f
2: and x2,x2,#~TIB_FLAG_VFORKED
b 1b
// if (rc < 0) errno = -rc, rc = -1;
3: .hidden _sysret
b _sysret
#else
#error "architecture unsupported"
#endif
.endfn vfork,globl
#ifdef SYSDEBUG
.rodata.str1.1
.Llog: .ascii STRACE_PROLOGUE
.asciz "vfork()\n"
.previous
#endif /* DEBUGSYS */

25
libc/runtime/warnifpowersave.c
View file

@ -16,9 +16,14 @@
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/calls/blockcancel.internal.h"
#include "libc/calls/calls.h"
#include "libc/calls/syscall-sysv.internal.h"
#include "libc/dce.h"
#include "libc/errno.h"
#include "libc/runtime/runtime.h"
#include "libc/str/str.h"
#include "libc/sysv/consts/at.h"
#include "libc/sysv/consts/o.h"
// RDTSC on Linux has so much jitter when the CPU is in powersave mode.
@ -31,12 +36,18 @@
"/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor\n"
void __warn_if_powersave(void) {
int fd;
int e, fd;
char buf[16] = {0};
if (!fileexists(FILE)) return;
if ((fd = open(FILE, O_RDONLY)) == -1) return;
read(fd, buf, 15);
close(fd);
if (!startswith(buf, "powersave")) return;
write(2, WARN, sizeof(WARN) - 1);
if (IsLinux()) {
e = errno;
BLOCK_CANCELLATIONS;
if ((fd = __sys_openat(AT_FDCWD, FILE, O_RDONLY, 0)) != -1) {
sys_read(fd, buf, 15);
sys_close(fd);
if (!startswith(buf, "powersave")) return;
sys_write(2, WARN, sizeof(WARN) - 1);
}
ALLOW_CANCELLATIONS;
errno = e;
}
}

80
libc/runtime/winmain.greg.c
View file

@ -18,36 +18,22 @@
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/assert.h"
#include "libc/calls/internal.h"
#include "libc/calls/state.internal.h"
#include "libc/calls/syscall_support-nt.internal.h"
#include "libc/dce.h"
#include "libc/intrin/asan.internal.h"
#include "libc/intrin/asancodes.h"
#include "libc/intrin/describeflags.internal.h"
#include "libc/intrin/getenv.internal.h"
#include "libc/intrin/weaken.h"
#include "libc/limits.h"
#include "libc/log/libfatal.internal.h"
#include "libc/macros.internal.h"
#include "libc/nexgen32e/rdtsc.h"
#include "libc/nt/console.h"
#include "libc/nt/enum/accessmask.h"
#include "libc/nt/enum/consolemodeflags.h"
#include "libc/nt/enum/creationdisposition.h"
#include "libc/nt/enum/fileflagandattributes.h"
#include "libc/nt/enum/filemapflags.h"
#include "libc/nt/enum/filesharemode.h"
#include "libc/nt/enum/pageflags.h"
#include "libc/nt/files.h"
#include "libc/nt/ipc.h"
#include "libc/nt/memory.h"
#include "libc/nt/pedef.internal.h"
#include "libc/nt/process.h"
#include "libc/nt/runtime.h"
#include "libc/nt/signals.h"
#include "libc/nt/struct/ntexceptionpointers.h"
#include "libc/nt/struct/teb.h"
#include "libc/nt/synchronization.h"
#include "libc/nt/thread.h"
#include "libc/nt/thunk/msabi.h"
#include "libc/runtime/internal.h"
#include "libc/runtime/memtrack.internal.h"
@ -60,13 +46,13 @@
#ifdef __x86_64__
#define abi __msabi textwindows dontinstrument
// clang-format off
__msabi extern typeof(CreateFileMapping) *const __imp_CreateFileMappingW;
__msabi extern typeof(DuplicateHandle) *const __imp_DuplicateHandle;
__msabi extern typeof(ExitProcess) *const __imp_ExitProcess;
__msabi extern typeof(FreeEnvironmentStrings) *const __imp_FreeEnvironmentStringsW;
__msabi extern typeof(GetConsoleMode) *const __imp_GetConsoleMode;
__msabi extern typeof(GetCurrentProcess) *const __imp_GetCurrentProcess;
__msabi extern typeof(GetCurrentProcessId) *const __imp_GetCurrentProcessId;
__msabi extern typeof(GetEnvironmentStrings) *const __imp_GetEnvironmentStringsW;
__msabi extern typeof(GetFileAttributes) *const __imp_GetFileAttributesW;
@ -80,9 +66,9 @@ __msabi extern typeof(VirtualProtect) *const __imp_VirtualProtect;
// clang-format on
void cosmo(int, char **, char **, long (*)[2]) wontreturn;
void __switch_stacks(int, char **, char **, long (*)[2],
void (*)(int, char **, char **, long (*)[2]),
intptr_t) wontreturn;
void __stack_call(int, char **, char **, long (*)[2],
void (*)(int, char **, char **, long (*)[2]),
intptr_t) wontreturn;
static const signed char kNtStdio[3] = {
(signed char)kNtStdInputHandle,
@ -90,18 +76,12 @@ static const signed char kNtStdio[3] = {
(signed char)kNtStdErrorHandle,
};
forceinline int IsAlpha(int c) {
__funline int IsAlpha(int c) {
return ('A' <= c && c <= 'Z') || ('a' <= c && c <= 'z');
}
// implements all win32 apis on non-windows hosts
__msabi long __oops_win32(void) {
assert(!"win32 api called on non-windows host");
return 0;
}
// https://nullprogram.com/blog/2022/02/18/
__msabi static inline char16_t *MyCommandLine(void) {
__funline char16_t *MyCommandLine(void) {
void *cmd;
asm("mov\t%%gs:(0x60),%0\n"
"mov\t0x20(%0),%0\n"
@ -110,8 +90,14 @@ __msabi static inline char16_t *MyCommandLine(void) {
return cmd;
}
// implements all win32 apis on non-windows hosts
static abi long __oops_win32(void) {
assert(!"win32 api called on non-windows host");
return 0;
}
// returns true if utf-8 path is a win32-style path that exists
__msabi static textwindows bool32 WinFileExists(const char *path) {
static abi bool32 WinFileExists(const char *path) {
uint16_t path16[PATH_MAX];
size_t z = ARRAYLEN(path16);
size_t n = tprecode8to16(path16, z, path).ax;
@ -120,14 +106,14 @@ __msabi static textwindows bool32 WinFileExists(const char *path) {
}
// this ensures close(1) won't accidentally close(2) for example
__msabi static textwindows void DeduplicateStdioHandles(void) {
static abi void DeduplicateStdioHandles(void) {
for (long i = 0; i < 3; ++i) {
int64_t h1 = __imp_GetStdHandle(kNtStdio[i]);
for (long j = i + 1; j < 3; ++j) {
int64_t h2 = __imp_GetStdHandle(kNtStdio[j]);
if (h1 == h2) {
int64_t h3, proc = __imp_GetCurrentProcess();
__imp_DuplicateHandle(proc, h2, proc, &h3, 0, false,
int64_t h3;
__imp_DuplicateHandle(-1, h2, -1, &h3, 0, false,
kNtDuplicateSameAccess);
__imp_SetStdHandle(kNtStdio[j], h3);
}
@ -137,7 +123,7 @@ __msabi static textwindows void DeduplicateStdioHandles(void) {
// main function of windows init process
// i.e. first process spawned that isn't forked
__msabi static textwindows wontreturn void WinInit(const char16_t *cmdline) {
static abi wontreturn void WinInit(const char16_t *cmdline) {
__oldstack = (intptr_t)__builtin_frame_address(0);
// make console into utf-8 ansi/xterm style tty
@ -177,26 +163,6 @@ __msabi static textwindows wontreturn void WinInit(const char16_t *cmdline) {
struct WinArgs *wa =
(struct WinArgs *)(stackaddr + (stacksize - sizeof(struct WinArgs)));
// allocate asan memory if needed
if (IsAsan()) {
uintptr_t shadowaddr = 0x7fff8000 + (stackaddr >> 3);
uintptr_t shadowend = 0x7fff8000 + ((stackaddr + stacksize) >> 3);
uintptr_t shallocaddr = ROUNDDOWN(shadowaddr, FRAMESIZE);
uintptr_t shallocend = ROUNDUP(shadowend, FRAMESIZE);
uintptr_t shallocsize = shallocend - shallocaddr;
__imp_MapViewOfFileEx(
(_mmi.p[1].h = __imp_CreateFileMappingW(
-1, 0, kNtPageReadwrite, shallocsize >> 32, shallocsize, NULL)),
kNtFileMapWrite, 0, 0, shallocsize, (void *)shallocaddr);
_mmi.p[1].x = shallocaddr >> 16;
_mmi.p[1].y = (shallocaddr >> 16) + ((shallocsize - 1) >> 16);
_mmi.p[1].prot = PROT_READ | PROT_WRITE;
_mmi.p[1].flags = 0x00000022; // private+anonymous
_mmi.p[1].size = shallocsize;
_mmi.i = 2;
__asan_poison((void *)stackaddr, GetGuardSize(), kAsanStackOverflow);
}
// parse utf-16 command into utf-8 argv array in argument block
int count = GetDosArgv(cmdline, wa->argblock, ARRAYLEN(wa->argblock),
wa->argv, ARRAYLEN(wa->argv));
@ -235,12 +201,12 @@ __msabi static textwindows wontreturn void WinInit(const char16_t *cmdline) {
__envp = &wa->envp[0];
// handover control to cosmopolitan runtime
__switch_stacks(count, wa->argv, wa->envp, wa->auxv, cosmo,
stackaddr + (stacksize - sizeof(struct WinArgs)));
__stack_call(count, wa->argv, wa->envp, wa->auxv, cosmo,
stackaddr + (stacksize - sizeof(struct WinArgs)));
}
__msabi textwindows int64_t WinMain(int64_t hInstance, int64_t hPrevInstance,
const char *lpCmdLine, int64_t nCmdShow) {
abi int64_t WinMain(int64_t hInstance, int64_t hPrevInstance,
const char *lpCmdLine, int64_t nCmdShow) {
const char16_t *cmdline;
extern char os asm("__hostos");
os = _HOSTWINDOWS; // madness https://news.ycombinator.com/item?id=21019722

2
libc/runtime/zipos-mmap.c
View file

@ -49,7 +49,7 @@
* it does not need to be 64kb aligned.
* @return virtual base address of new mapping, or MAP_FAILED w/ errno
*/
dontasan void *__zipos_mmap(void *addr, size_t size, int prot, int flags,
void *__zipos_mmap(void *addr, size_t size, int prot, int flags,
struct ZiposHandle *h, int64_t off) {
if (off < 0) {

11
libc/runtime/zipos-open.c
View file

@ -52,6 +52,10 @@ static char *__zipos_mapend;
static size_t __zipos_maptotal;
static pthread_mutex_t __zipos_lock_obj;
static void __zipos_wipe(void) {
pthread_mutex_init(&__zipos_lock_obj, 0);
}
static void __zipos_lock(void) {
pthread_mutex_lock(&__zipos_lock_obj);
}
@ -60,10 +64,6 @@ static void __zipos_unlock(void) {
pthread_mutex_unlock(&__zipos_lock_obj);
}
static void __zipos_funlock(void) {
pthread_mutex_init(&__zipos_lock_obj, 0);
}
static void *__zipos_mmap_space(size_t mapsize) {
char *start;
size_t offset;
@ -268,5 +268,6 @@ int __zipos_open(struct ZiposUri *name, int flags) {
}
__attribute__((__constructor__)) static void __zipos_ctor(void) {
pthread_atfork(__zipos_lock, __zipos_unlock, __zipos_funlock);
__zipos_wipe();
pthread_atfork(__zipos_lock, __zipos_unlock, __zipos_wipe);
}

53
libc/runtime/zipos-seek.c
View file

@ -19,51 +19,34 @@
#include "libc/calls/calls.h"
#include "libc/runtime/zipos.internal.h"
#include "libc/stdckdint.h"
#include "libc/sysv/consts/s.h"
#include "libc/sysv/errfuns.h"
#include "libc/thread/thread.h"
#include "libc/thread/tls.h"
#include "libc/zip.internal.h"
static int64_t __zipos_seek_impl(struct ZiposHandle *h, int64_t offset,
unsigned whence) {
int64_t pos;
if (h->cfile == ZIPOS_SYNTHETIC_DIRECTORY ||
S_ISDIR(GetZipCfileMode(h->zipos->map + h->cfile))) {
return eisdir();
}
static int64_t GetPosition(struct ZiposHandle *h, int whence) {
switch (whence) {
case SEEK_SET:
if (offset >= 0) {
return offset;
} else {
return einval();
}
return 0;
case SEEK_CUR:
if (!ckd_add(&pos, h->pos, offset)) {
if (pos >= 0) {
return pos;
} else {
return einval();
}
} else {
return eoverflow();
}
return h->pos;
case SEEK_END:
if (!ckd_sub(&pos, h->size, offset)) {
if (pos >= 0) {
return pos;
} else {
return einval();
}
} else {
return eoverflow();
}
return h->size;
default:
return einval();
}
}
static int64_t Seek(struct ZiposHandle *h, int64_t offset, int whence) {
int64_t pos;
if (!ckd_add(&pos, GetPosition(h, whence), offset)) {
if (pos >= 0) {
return pos;
} else {
return einval();
}
} else {
return eoverflow();
}
}
/**
* Changes current position of zip file handle.
*
@ -74,7 +57,7 @@ static int64_t __zipos_seek_impl(struct ZiposHandle *h, int64_t offset,
*/
int64_t __zipos_seek(struct ZiposHandle *h, int64_t offset, unsigned whence) {
int64_t pos;
if ((pos = __zipos_seek_impl(h, offset, whence)) != -1) {
if ((pos = Seek(h, offset, whence)) != -1) {
h->pos = pos;
}
return pos;

2
libc/runtime/zipos.internal.h
View file

@ -56,7 +56,7 @@ int64_t __zipos_seek(struct ZiposHandle *, int64_t, unsigned);
int __zipos_fcntl(int, int, uintptr_t);
int __zipos_notat(int, const char *);
void *__zipos_mmap(void *, uint64_t, int32_t, int32_t, struct ZiposHandle *,
int64_t) dontasan;
int64_t);
COSMOPOLITAN_C_END_
#endif /* !(__ASSEMBLER__ + __LINKER__ + 0) */