mirrors/cosmopolitan
1
0
Fork 0
mirror of https://github.com/jart/cosmopolitan.git synced 2025-07-27 04:50:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Make improvements

Browse source 
- Every unit test now passes on Apple Silicon. The final piece of this
  puzzle was porting our POSIX threads cancelation support, since that
  works differently on ARM64 XNU vs. AMD64. Our semaphore support on
  Apple Silicon is also superior now compared to AMD64, thanks to the
  grand central dispatch library which lets *NSYNC locks go faster.

- The Cosmopolitan runtime is now more stable, particularly on Windows.
  To do this, thread local storage is mandatory at all runtime levels,
  and the innermost packages of the C library is no longer being built
  using ASAN. TLS is being bootstrapped with a 128-byte TIB during the
  process startup phase, and then later on the runtime re-allocates it
  either statically or dynamically to support code using _Thread_local.
  fork() and execve() now do a better job cooperating with threads. We
  can now check how much stack memory is left in the process or thread
  when functions like kprintf() / execve() etc. call alloca(), so that
  ENOMEM can be raised, reduce a buffer size, or just print a warning.

- POSIX signal emulation is now implemented the same way kernels do it
  with pthread_kill() and raise(). Any thread can interrupt any other
  thread, regardless of what it's doing. If it's blocked on read/write
  then the killer thread will cancel its i/o operation so that EINTR can
  be returned in the mark thread immediately. If it's doing a tight CPU
  bound operation, then that's also interrupted by the signal delivery.
  Signal delivery works now by suspending a thread and pushing context
  data structures onto its stack, and redirecting its execution to a
  trampoline function, which calls SetThreadContext(GetCurrentThread())
  when it's done.

- We're now doing a better job managing locks and handles. On NetBSD we
  now close semaphore file descriptors in forked children. Semaphores on
  Windows can now be canceled immediately, which means mutexes/condition
  variables will now go faster. Apple Silicon semaphores can be canceled
  too. We're now using Apple's pthread_yield() funciton. Apple _nocancel
  syscalls are now used on XNU when appropriate to ensure pthread_cancel
  requests aren't lost. The MbedTLS library has been updated to support
  POSIX thread cancelations. See tool/build/runitd.c for an example of
  how it can be used for production multi-threaded tls servers. Handles
  on Windows now leak less often across processes. All i/o operations on
  Windows are now overlapped, which means file pointers can no longer be
  inherited across dup() and fork() for the time being.

- We now spawn a thread on Windows to deliver SIGCHLD and wakeup wait4()
  which means, for example, that posix_spawn() now goes 3x faster. POSIX
  spawn is also now more correct. Like Musl, it's now able to report the
  failure code of execve() via a pipe although our approach favors using
  shared memory to do that on systems that have a true vfork() function.

- We now spawn a thread to deliver SIGALRM to threads when setitimer()
  is used. This enables the most precise wakeups the OS makes possible.

- The Cosmopolitan runtime now uses less memory. On NetBSD for example,
  it turned out the kernel would actually commit the PT_GNU_STACK size
  which caused RSS to be 6mb for every process. Now it's down to ~4kb.
  On Apple Silicon, we reduce the mandatory upstream thread size to the
  smallest possible size to reduce the memory overhead of Cosmo threads.
  The examples directory has a program called greenbean which can spawn
  a web server on Linux with 10,000 worker threads and have the memory
  usage of the process be ~77mb. The 1024 byte overhead of POSIX-style
  thread-local storage is now optional; it won't be allocated until the
  pthread_setspecific/getspecific functions are called. On Windows, the
  threads that get spawned which are internal to the libc implementation
  use reserve rather than commit memory, which shaves a few hundred kb.

- sigaltstack() is now supported on Windows, however it's currently not
  able to be used to handle stack overflows, since crash signals are
  still generated by WIN32. However the crash handler will still switch
  to the alt stack, which is helpful in environments with tiny threads.

- Test binaries are now smaller. Many of the mandatory dependencies of
  the test runner have been removed. This ensures many programs can do a
  better job only linking the the thing they're testing. This caused the
  test binaries for LIBC_FMT for example, to decrease from 200kb to 50kb

- long double is no longer used in the implementation details of libc,
  except in the APIs that define it. The old code that used long double
  for time (instead of struct timespec) has now been thoroughly removed.

- ShowCrashReports() is now much tinier in MODE=tiny. Instead of doing
  backtraces itself, it'll just print a command you can run on the shell
  using our new `cosmoaddr2line` program to view the backtrace.

- Crash report signal handling now works in a much better way. Instead
  of terminating the process, it now relies on SA_RESETHAND so that the
  default SIG_IGN behavior can terminate the process if necessary.

- Our pledge() functionality has now been fully ported to AARCH64 Linux.
This commit is contained in:
Justine Tunney 2023-09-18 20:44:45 -07:00
parent c4eb838516
commit ec480f5aa0
No known key found for this signature in database
GPG key ID: BE714B4575D6E328
638 changed files with 7925 additions and 8282 deletions
Download patch file Download diff file Expand all files Collapse all files

1
tool/build/build.mk
View file

@ -36,6 +36,7 @@ TOOL_BUILD_DIRECTDEPS = \
LIBC_NT_KERNEL32 \
LIBC_NT_USER32 \
LIBC_NT_WS2_32 \
LIBC_PROC \
LIBC_RUNTIME \
LIBC_SOCK \
LIBC_STDIO \

4
tool/build/compile.c
View file

@ -49,7 +49,7 @@
#include "libc/nexgen32e/x86info.h"
#include "libc/runtime/runtime.h"
#include "libc/stdio/append.h"
#include "libc/stdio/posix_spawn.h"
#include "libc/proc/posix_spawn.h"
#include "libc/str/str.h"
#include "libc/sysv/consts/auxv.h"
#include "libc/sysv/consts/clock.h"
@ -811,7 +811,7 @@ char *MakeTmpOut(const char *path) {
char *p = tmpout;
char *e = tmpout + sizeof(tmpout) - 1;
g_tmpout_original = path;
p = stpcpy(p, kTmpPath);
p = stpcpy(p, __get_tmpdir());
while ((c = *path++)) {
if (c == '/') c = '_';
if (p == e) {

104
tool/build/deltaify.c
View file

@ -1,104 +0,0 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
│vi: set net ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/calls/calls.h"
#include "libc/calls/struct/sigaction.h"
#include "libc/calls/struct/siginfo.h"
#include "libc/calls/ucontext.h"
#include "libc/errno.h"
#include "libc/log/check.h"
#include "libc/mem/mem.h"
#include "libc/runtime/runtime.h"
#include "libc/stdio/stdio.h"
#include "libc/sysv/consts/sa.h"
#include "libc/sysv/consts/sig.h"
#include "libc/time/time.h"
#include "libc/x/xgetline.h"
static int pid;
static void RelaySig(int sig, struct siginfo *si, void *uc) {
kill(pid, sig);
}
int main(int argc, char *argv[]) {
FILE *f;
bool ok;
char *s, *p;
int64_t micros;
long double t1, t2;
int ws, pipefds[2];
setvbuf(stdout, malloc(BUFSIZ), _IOLBF, BUFSIZ);
setvbuf(stderr, malloc(BUFSIZ), _IOLBF, BUFSIZ);
if (argc < 2) {
f = stdin;
t1 = nowl();
} else {
sigset_t block, mask;
struct sigaction saignore = {.sa_handler = SIG_IGN};
struct sigaction sadefault = {.sa_handler = SIG_DFL};
struct sigaction sarelay = {.sa_sigaction = RelaySig,
.sa_flags = SA_RESTART};
sigemptyset(&block);
sigaddset(&mask, SIGINT);
sigaddset(&mask, SIGQUIT);
sigaddset(&mask, SIGCHLD);
sigprocmask(SIG_BLOCK, &block, &mask);
sigaction(SIGHUP, &sarelay, 0);
sigaction(SIGTERM, &sarelay, 0);
sigaction(SIGINT, &saignore, 0);
sigaction(SIGQUIT, &saignore, 0);
CHECK_NE(-1, pipe(pipefds));
CHECK_NE(-1, (pid = vfork()));
if (!pid) {
close(pipefds[0]);
dup2(pipefds[1], 1);
sigaction(SIGHUP, &sadefault, NULL);
sigaction(SIGTERM, &sadefault, NULL);
sigaction(SIGINT, &sadefault, NULL);
sigaction(SIGQUIT, &sadefault, NULL);
sigprocmask(SIG_SETMASK, &mask, NULL);
execvp(argv[1], argv + 1);
_exit(127);
}
t1 = nowl();
close(pipefds[1]);
sigprocmask(SIG_SETMASK, &mask, NULL);
CHECK((f = fdopen(pipefds[0], "r")));
CHECK_NE(-1, setvbuf(f, malloc(4096), _IOLBF, 4096));
}
if ((p = xgetline(f))) {
t2 = nowl();
micros = (t2 - t1) * 1e6;
t1 = t2;
printf("%16ld\n", micros);
do {
s = xgetline(f);
t2 = nowl();
micros = (t2 - t1) * 1e6;
t1 = t2;
printf("%16ld %s", micros, p);
free(p);
} while ((p = s));
}
ok = !ferror(f);
if (argc < 2) return ok ? 0 : 1;
fclose(f);
while (waitpid(pid, &ws, 0) == -1) CHECK_EQ(EINTR, errno);
return !WIFEXITED(ws) ? 128 + WTERMSIG(ws) : ok ? WEXITSTATUS(ws) : 1;
}

42
tool/build/elf2pe.c
View file

@ -23,6 +23,7 @@
#include "libc/elf/scalar.h"
#include "libc/elf/struct/rela.h"
#include "libc/elf/struct/shdr.h"
#include "libc/fmt/conv.h"
#include "libc/fmt/itoa.h"
#include "libc/intrin/bits.h"
#include "libc/intrin/describeflags.internal.h"
@ -56,18 +57,20 @@
"copyright 2023 justine tunney\n" \
"https://github.com/jart/cosmopolitan\n"
#define MANUAL \
" -o OUTPUT INPUT\n" \
"\n" \
"DESCRIPTION\n" \
"\n" \
" Converts ELF executables to PE\n" \
"\n" \
"FLAGS\n" \
"\n" \
" -h show usage\n" \
" -o OUTPUT set output path\n" \
" -D PATH embed dos/bios stub\n" \
#define MANUAL \
" -o OUTPUT INPUT\n" \
"\n" \
"DESCRIPTION\n" \
"\n" \
" Converts ELF executables to PE\n" \
"\n" \
"FLAGS\n" \
"\n" \
" -h show usage\n" \
" -o OUTPUT set output path\n" \
" -D PATH embed dos/bios stub\n" \
" -S SIZE size of stack commit\n" \
" -R SIZE size of stack reserve\n" \
"\n"
#define MAX_ALIGN 65536
@ -152,6 +155,8 @@ struct Elf {
static const char *prog;
static const char *outpath;
static const char *stubpath;
static long FLAG_SizeOfStackCommit = 64 * 1024;
static long FLAG_SizeOfStackReserve = 8 * 1024 * 1024;
static wontreturn void Die(const char *thing, const char *reason) {
tinyprint(2, thing, ": ", reason, "\n", NULL);
@ -846,8 +851,9 @@ static struct ImagePointer GeneratePe(struct Elf *elf, char *fp, int64_t vp) {
opthdr->Subsystem = kNtImageSubsystemWindowsCui;
opthdr->DllCharacteristics = kNtImageDllcharacteristicsNxCompat |
kNtImageDllcharacteristicsHighEntropyVa;
opthdr->SizeOfStackReserve = 8 * 1024 * 1024;
opthdr->SizeOfStackCommit = 64 * 1024;
opthdr->SizeOfStackReserve =
MAX(FLAG_SizeOfStackReserve, FLAG_SizeOfStackCommit);
opthdr->SizeOfStackCommit = FLAG_SizeOfStackCommit;
// output data directory entries
if (elf->imports) {
@ -1033,7 +1039,7 @@ static struct ImagePointer GeneratePe(struct Elf *elf, char *fp, int64_t vp) {
static void GetOpts(int argc, char *argv[]) {
int opt;
while ((opt = getopt(argc, argv, "ho:D:")) != -1) {
while ((opt = getopt(argc, argv, "ho:D:R:S:")) != -1) {
switch (opt) {
case 'o':
outpath = optarg;
@ -1041,6 +1047,12 @@ static void GetOpts(int argc, char *argv[]) {
case 'D':
stubpath = optarg;
break;
case 'S':
FLAG_SizeOfStackCommit = sizetol(optarg, 1024);
break;
case 'R':
FLAG_SizeOfStackReserve = sizetol(optarg, 1024);
break;
case 'h':
ShowUsage(0, 1);
default:

1
tool/build/lib/buildlib.mk
View file

@ -44,6 +44,7 @@ TOOL_BUILD_LIB_A_DIRECTDEPS = \
LIBC_STR \
LIBC_SYSV \
LIBC_SYSV_CALLS \
LIBC_PROC \
LIBC_THREAD \
LIBC_TIME \
LIBC_TINYMATH \

2
tool/build/lib/eztls.c
View file

@ -24,8 +24,8 @@
#include "libc/fmt/itoa.h"
#include "libc/intrin/kprintf.h"
#include "libc/macros.internal.h"
#include "libc/runtime/syslib.internal.h"
#include "libc/sysv/consts/sig.h"
#include "libc/thread/thread.h"
#include "libc/x/x.h"
#include "libc/x/xsigaction.h"
#include "net/https/https.h"

24
tool/build/runit.c
View file

@ -22,6 +22,7 @@
#include "libc/calls/struct/itimerval.h"
#include "libc/calls/struct/sigaction.h"
#include "libc/calls/struct/stat.h"
#include "libc/calls/struct/timespec.h"
#include "libc/dns/dns.h"
#include "libc/errno.h"
#include "libc/fmt/conv.h"
@ -55,6 +56,8 @@
#include "libc/x/xasprintf.h"
#include "libc/x/xsigaction.h"
#include "net/https/https.h"
#include "third_party/mbedtls/debug.h"
#include "third_party/mbedtls/net_sockets.h"
#include "third_party/mbedtls/ssl.h"
#include "third_party/zlib/zlib.h"
#include "tool/build/lib/eztls.h"
@ -147,8 +150,8 @@ void CheckExists(const char *path) {
void Connect(void) {
const char *ip4;
int rc, err, expo;
long double t1, t2;
struct addrinfo *ai;
struct timespec deadline;
if ((rc = getaddrinfo(g_hostname, _gc(xasprintf("%hu", g_runitdport)),
&kResolvHints, &ai)) != 0) {
FATALF("%s:%hu: EAI_%s %m", g_hostname, g_runitdport, gai_strerror(rc));
@ -166,7 +169,8 @@ void Connect(void) {
CHECK_NE(-1,
(g_sock = socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol)));
expo = INITIAL_CONNECT_TIMEOUT;
t1 = nowl();
deadline = timespec_add(timespec_real(),
timespec_fromseconds(MAX_WAIT_CONNECT_SECONDS));
LOGIFNEG1(sigaction(SIGALRM, &(struct sigaction){.sa_handler = OnAlarm}, 0));
DEBUGF("connecting to %s (%hhu.%hhu.%hhu.%hhu) to run %s", g_hostname, ip4[0],
ip4[1], ip4[2], ip4[3], g_prog);
@ -178,11 +182,10 @@ TryAgain:
NULL));
rc = connect(g_sock, ai->ai_addr, ai->ai_addrlen);
err = errno;
t2 = nowl();
if (rc == -1) {
if (err == EINTR) {
expo *= 1.5;
if (t2 > t1 + MAX_WAIT_CONNECT_SECONDS) {
if (timespec_cmp(timespec_real(), deadline) >= 0) {
FATALF("timeout connecting to %s (%hhu.%hhu.%hhu.%hhu:%d)", g_hostname,
ip4[0], ip4[1], ip4[2], ip4[3], ntohs(ai->ai_addr4->sin_port));
__builtin_unreachable();
@ -293,10 +296,17 @@ bool Recv(char *p, int n) {
do rc = mbedtls_ssl_read(&ezssl, p + i, n - i);
while (rc == MBEDTLS_ERR_SSL_WANT_READ);
if (!rc) return false;
if (rc < 0) EzTlsDie("read response failed", rc);
if (rc < 0) {
if (rc == MBEDTLS_ERR_NET_CONN_RESET) {
EzTlsDie("connection reset", rc);
} else {
EzTlsDie("read response failed", rc);
}
}
}
return true;
}
int ReadResponse(void) {
int exitcode;
for (;;) {
@ -319,7 +329,7 @@ int ReadResponse(void) {
exitcode = 202;
break;
}
exitcode = *msg;
exitcode = *msg & 255;
if (exitcode) {
WARNF("%s says %s exited with %d", g_hostname, g_prog, exitcode);
} else {
@ -355,6 +365,7 @@ int RunOnHost(char *spec) {
sscanf(spec, "%100s %hu %hu", g_hostname, &g_runitdport, &g_sshport);
if (got < 1) {
kprintf("what on earth %#s -> %d\n", spec, got);
fprintf(stderr, "what on earth %#s -> %d\n", spec, got);
exit(1);
}
if (!strchr(g_hostname, '.')) strcat(g_hostname, ".test.");
@ -466,6 +477,7 @@ int SpawnSubprocesses(int argc, char *argv[]) {
int main(int argc, char *argv[]) {
ShowCrashReports();
signal(SIGPIPE, SIG_IGN);
// mbedtls_debug_threshold = 3;
if (getenv("DEBUG")) {
__log_level = kLogDebug;
}

73
tool/build/runitd.c
View file

@ -39,13 +39,13 @@
#include "libc/mem/gc.internal.h"
#include "libc/mem/mem.h"
#include "libc/nexgen32e/crc32.h"
#include "libc/proc/posix_spawn.h"
#include "libc/runtime/runtime.h"
#include "libc/runtime/syslib.internal.h"
#include "libc/sock/sock.h"
#include "libc/sock/struct/pollfd.h"
#include "libc/sock/struct/sockaddr.h"
#include "libc/stdio/append.h"
#include "libc/stdio/posix_spawn.h"
#include "libc/stdio/rand.h"
#include "libc/stdio/stdio.h"
#include "libc/str/str.h"
@ -249,8 +249,8 @@ void StartTcpServer(void) {
uint32_t asize;
g_servfd = socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, IPPROTO_TCP);
if (g_servfd == -1) {
fprintf(stderr, program_invocation_short_name,
": socket failed: ", strerror(errno), "\n", NULL);
fprintf(stderr, "%s: socket failed: %s\n", program_invocation_short_name,
strerror(errno));
exit(1);
}
struct timeval timeo = {DEATH_CLOCK_SECONDS / 10};
@ -259,8 +259,8 @@ void StartTcpServer(void) {
setsockopt(g_servfd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));
if (bind(g_servfd, (struct sockaddr *)&g_servaddr, sizeof(g_servaddr)) ==
-1) {
fprintf(stderr, program_invocation_short_name,
": bind failed: ", strerror(errno), "\n", NULL);
fprintf(stderr, "%s: bind failed: %s\n", program_invocation_short_name,
strerror(errno));
exit(1);
}
unassert(!listen(g_servfd, 10));
@ -474,7 +474,7 @@ void *ClientWorker(void *arg) {
crc = READ32BE(msg + 13);
origname = gc(calloc(1, namesize + 1));
Recv(client, origname, namesize);
INFOF("%s sent %#s (%'u bytes @ %#s)", addrstr, origname, filesize,
VERBF("%s sent %#s (%'u bytes @ %#s)", addrstr, origname, filesize,
client->tmpexepath);
char *exedata = gc(malloc(filesize));
Recv(client, exedata, filesize);
@ -490,14 +490,23 @@ void *ClientWorker(void *arg) {
// thus we use an optimistic approach to avoid expensive locks
sprintf(client->tmpexepath, "o/%s.XXXXXX.com", basename(origname));
int exefd = openatemp(AT_FDCWD, client->tmpexepath, 4, O_CLOEXEC, 0700);
ftruncate(exefd, filesize);
if (exefd == -1) {
WARNF("%s failed to open temporary file %#s due to %m", addrstr,
client->tmpexepath);
pthread_exit(0);
}
if (ftruncate(exefd, filesize)) {
WARNF("%s failed to write %#s due to %m", addrstr, origname);
close(exefd);
pthread_exit(0);
}
if (write(exefd, exedata, filesize) != filesize) {
WARNF("%s failed to write %#s", addrstr, origname);
WARNF("%s failed to write %#s due to %m", addrstr, origname);
close(exefd);
pthread_exit(0);
}
if (close(exefd)) {
WARNF("%s failed to close %#s", addrstr, origname);
WARNF("%s failed to close %#s due to %m", addrstr, origname);
pthread_exit(0);
}
@ -534,9 +543,11 @@ RetryOnEtxtbsyRaceCondition:
}
}
errno_t err;
struct timespec started;
posix_spawnattr_t spawnattr;
posix_spawn_file_actions_t spawnfila;
sigemptyset(&sigmask);
started = timespec_real();
pipe2(client->pipe, O_CLOEXEC);
posix_spawnattr_init(&spawnattr);
posix_spawnattr_setflags(&spawnattr, POSIX_SPAWN_SETPGROUP);
@ -591,8 +602,8 @@ RetryOnEtxtbsyRaceCondition:
INFOF("poll interrupted");
continue;
} else {
WARNF("killing %d %s and hanging up %d because poll failed", client->fd,
origname, client->pid);
WARNF("killing %d %s and hanging up %d because poll failed with %m",
client->fd, origname, client->pid);
goto HangupClientAndTerminateJob;
}
}
@ -616,6 +627,10 @@ RetryOnEtxtbsyRaceCondition:
client->pid);
continue;
}
if (received == MBEDTLS_ERR_SSL_CANCELED) { // EAGAIN SO_RCVTIMEO
WARNF("%s (pid %d) is is canceling job", origname, client->pid);
goto HangupClientAndTerminateJob;
}
WARNF("client ssl read failed with -0x%04x (%s) so killing %s",
-received, GetTlsError(received), origname);
goto TerminateJob;
@ -647,21 +662,28 @@ WaitAgain:
kill(client->pid, SIGINT);
goto WaitAgain;
}
if (errno == ECANCELED) {
WARNF("thread is canceled; killing %s pid %d", origname, client->pid);
kill(client->pid, SIGKILL);
goto WaitAgain;
}
WARNF("waitpid failed %m");
client->pid = 0;
goto HangupClientAndTerminateJob;
}
client->pid = 0;
int exitcode;
struct timespec ended = timespec_real();
int64_t micros = timespec_tomicros(timespec_sub(ended, started));
if (WIFEXITED(wstatus)) {
if (WEXITSTATUS(wstatus)) {
WARNF("%s on %s exited with %d", origname, g_hostname,
WEXITSTATUS(wstatus));
appendf(&client->output, "------ %s %s $?=%d (0x%08x) ------\n",
g_hostname, origname, WEXITSTATUS(wstatus), wstatus);
WARNF("%s on %s exited with $?=%d after %'ldµs", origname, g_hostname,
WEXITSTATUS(wstatus), micros);
appendf(&client->output, "------ %s %s $?=%d (0x%08x) %,ldµs ------\n",
g_hostname, origname, WEXITSTATUS(wstatus), wstatus, micros);
} else {
VERBF("%s on %s exited with %d", origname, g_hostname,
WEXITSTATUS(wstatus));
INFOF("%s on %s exited with $?=%d after %'ldµs", origname, g_hostname,
WEXITSTATUS(wstatus), micros);
}
exitcode = WEXITSTATUS(wstatus);
} else if (WIFSIGNALED(wstatus)) {
@ -670,14 +692,16 @@ WaitAgain:
client->output = 0;
goto RetryOnEtxtbsyRaceCondition;
}
WARNF("%s on %s terminated with %s", origname, g_hostname,
strsignal(WTERMSIG(wstatus)));
char sigbuf[21];
WARNF("%s on %s terminated after %'ldµs with %s", origname, g_hostname,
micros, strsignal_r(WTERMSIG(wstatus), sigbuf));
exitcode = 128 + WTERMSIG(wstatus);
appendf(&client->output, "------ %s %s $?=%s (0x%08x) ------\n", g_hostname,
origname, strsignal(WTERMSIG(wstatus)), wstatus);
appendf(&client->output, "------ %s %s $?=%s (0x%08x) %,ldµs ------\n",
g_hostname, origname, strsignal(WTERMSIG(wstatus)), wstatus,
micros);
} else {
WARNF("%s on %s died with wait status 0x%08x", origname, g_hostname,
wstatus);
WARNF("%s on %s died after %'ldµs with wait status 0x%08x", origname,
g_hostname, micros, wstatus);
exitcode = 127;
}
if (wstatus) {
@ -701,6 +725,7 @@ void HandleClient(void) {
client->addrsize = sizeof(client->addr);
for (;;) {
if (g_interrupted) {
pthread_cancel(0);
free(client);
return;
}
@ -765,7 +790,7 @@ void Daemonize(void) {
}
int main(int argc, char *argv[]) {
#if IsModeDbg()
#ifndef NDEBUG
ShowCrashReports();
#endif
GetOpts(argc, argv);

1
tool/build/zipobj.c
View file

@ -211,6 +211,7 @@ void zipobj(int argc, char **argv) {
}
int main(int argc, char **argv) {
ShowCrashReports();
timestamp.tv_sec = 1647414000; /* determinism */
/* clock_gettime(CLOCK_REALTIME, &timestamp); */
zipobj(argc, argv);

18
tool/hello/hello.mk
View file

@ -76,4 +76,22 @@ o/$(MODE)/tool/hello/hello-pe.com: \
o/$(MODE)/tool/build/elf2pe.com
@$(COMPILE) -AELF2PE o/$(MODE)/tool/build/elf2pe.com -o $@ $<
# elf2pe can generate binaries that don't have dll imports
o/$(MODE)/tool/hello/life-pe.com.dbg: \
o/$(MODE)/tool/hello/life-pe.o
@$(COMPILE) -ALINK.elf $(LINK) $(LINKARGS) $(OUTPUT_OPTION) -q -e WinMain
o/$(MODE)/tool/hello/life-pe.com: \
o/$(MODE)/tool/hello/life-pe.com.dbg \
o/$(MODE)/tool/build/elf2pe.com
@$(COMPILE) -AELF2PE o/$(MODE)/tool/build/elf2pe.com -o $@ $<
# demonstrates in process monitor the lowest resource usage a win32 app can have
o/$(MODE)/tool/hello/wait-pe.com.dbg: \
o/$(MODE)/tool/hello/wait-pe.o
@$(COMPILE) -ALINK.elf $(LINK) $(LINKARGS) $(OUTPUT_OPTION) -q -e WinMain
o/$(MODE)/tool/hello/wait-pe.com: \
o/$(MODE)/tool/hello/wait-pe.com.dbg \
o/$(MODE)/tool/build/elf2pe.com
@$(COMPILE) -AELF2PE o/$(MODE)/tool/build/elf2pe.com -R 64kb -S 4kb -o $@ $<
$(TOOL_HELLO_OBJS): tool/hello/hello.mk

13
tool/hello/life-pe.c Normal file
View file

@ -0,0 +1,13 @@
#if 0
/*─────────────────────────────────────────────────────────────────╗
│ To the extent possible under law, Justine Tunney has waived │
│ all copyright and related or neighboring rights to this file, │
│ as it is written in the following disclaimers: │
│ • http://unlicense.org/ │
│ • http://creativecommons.org/publicdomain/zero/1.0/ │
╚─────────────────────────────────────────────────────────────────*/
#endif
__attribute__((__ms_abi__)) long WinMain(void) {
return 42 << 8;
}

19
tool/hello/wait-pe.c Normal file
View file

@ -0,0 +1,19 @@
#if 0
/*─────────────────────────────────────────────────────────────────╗
│ To the extent possible under law, Justine Tunney has waived │
│ all copyright and related or neighboring rights to this file, │
│ as it is written in the following disclaimers: │
│ • http://unlicense.org/ │
│ • http://creativecommons.org/publicdomain/zero/1.0/ │
╚─────────────────────────────────────────────────────────────────*/
#endif
#include "tool/build/elf2pe.h"
#define STD_OUTPUT_HANDLE -11u
__dll_import("kernel32.dll", uint32_t, SleepEx, (uint32_t, bool32));
__attribute__((__ms_abi__)) long WinMain(void) {
SleepEx(-1, true);
return 0;
}

3
tool/net/definitions.lua
View file

@ -7266,8 +7266,7 @@ function unix.tiocgwinsz(fd) end
---
--- This creates a secure temporary file inside `$TMPDIR`. If it isn't
--- defined, then `/tmp` is used on UNIX and GetTempPath() is used on
--- the New Technology. This resolution of `$TMPDIR` happens once in a
--- ctor, which is copied to the `kTmpPath` global.
--- the New Technology. This resolution of `$TMPDIR` happens once.
---
--- Once close() is called, the returned file is guaranteed to be
--- deleted automatically. On UNIX the file is unlink()'d before this

3
tool/net/help.txt
View file

@ -4710,8 +4710,7 @@ UNIX MODULE
This creates a secure temporary file inside `$TMPDIR`. If it isn't
defined, then `/tmp` is used on UNIX and GetTempPath() is used on
the New Technology. This resolution of `$TMPDIR` happens once in a
ctor, which is copied to the `kTmpPath` global.
the New Technology. This resolution of `$TMPDIR` happens once.
Once close() is called, the returned file is guaranteed to be
deleted automatically. On UNIX the file is unlink()'d before this

11
tool/net/lfuncs.c
View file

@ -21,6 +21,7 @@
#include "libc/calls/calls.h"
#include "libc/calls/struct/rusage.h"
#include "libc/calls/struct/stat.h"
#include "libc/calls/struct/timespec.h"
#include "libc/dns/dns.h"
#include "libc/errno.h"
#include "libc/fmt/itoa.h"
@ -51,6 +52,7 @@
#include "libc/sysv/consts/o.h"
#include "libc/sysv/consts/rusage.h"
#include "libc/sysv/consts/sock.h"
#include "libc/thread/thread.h"
#include "libc/time/time.h"
#include "libc/x/x.h"
#include "net/http/escape.h"
@ -106,7 +108,8 @@ int LuaBin(lua_State *L) {
}
int LuaGetTime(lua_State *L) {
lua_pushnumber(L, nowl());
struct timespec now = timespec_real();
lua_pushnumber(L, now.tv_sec + now.tv_nsec * 1e-9);
return 1;
}
@ -894,7 +897,7 @@ int LuaBenchmark(lua_State *L) {
for (attempts = 0;;) {
lua_gc(L, LUA_GCCOLLECT);
sched_yield();
pthread_yield();
core = TSC_AUX_CORE(Rdpid());
interrupts = GetInterrupts();
for (avgticks = iter = 1; iter < count; ++iter) {
@ -916,7 +919,7 @@ int LuaBenchmark(lua_State *L) {
for (attempts = 0;;) {
lua_gc(L, LUA_GCCOLLECT);
sched_yield();
pthread_yield();
core = TSC_AUX_CORE(Rdpid());
interrupts = GetInterrupts();
for (avgticks = iter = 1; iter < count; ++iter) {
@ -937,7 +940,7 @@ int LuaBenchmark(lua_State *L) {
avgticks = MAX(avgticks - overhead, 0);
lua_gc(L, LUA_GCRESTART);
lua_pushinteger(L, ConvertTicksToNanos(round(avgticks)));
lua_pushinteger(L, avgticks / 3);
lua_pushinteger(L, round(avgticks));
lua_pushinteger(L, round(overhead));
lua_pushinteger(L, attempts);

4
tool/net/net.mk
View file

@ -20,8 +20,7 @@ TOOL_NET_COMS = \
o/$(MODE)/tool/net/redbean-demo.com \
o/$(MODE)/tool/net/redbean-static.com \
o/$(MODE)/tool/net/redbean-unsecure.com \
o/$(MODE)/tool/net/redbean-original.com \
o/$(MODE)/tool/net/wb.com
o/$(MODE)/tool/net/redbean-original.com
TOOL_NET_CHECKS = \
o/$(MODE)/tool/net/net.pkg \
@ -38,6 +37,7 @@ TOOL_NET_DIRECTDEPS = \
LIBC_NEXGEN32E \
LIBC_NT_IPHLPAPI \
LIBC_NT_KERNEL32 \
LIBC_PROC \
LIBC_RUNTIME \
LIBC_SOCK \
LIBC_STDIO \

42
tool/net/redbean.c
View file

@ -54,12 +54,10 @@
#include "libc/mem/gc.internal.h"
#include "libc/mem/mem.h"
#include "libc/nexgen32e/crc32.h"
#include "libc/nexgen32e/nt2sysv.h"
#include "libc/nexgen32e/rdtsc.h"
#include "libc/nexgen32e/vendor.internal.h"
#include "libc/nexgen32e/x86feature.h"
#include "libc/nt/enum/fileflagandattributes.h"
#include "libc/nt/thread.h"
#include "libc/runtime/clktck.h"
#include "libc/runtime/internal.h"
#include "libc/runtime/memtrack.internal.h"
@ -102,7 +100,6 @@
#include "libc/sysv/consts/timer.h"
#include "libc/sysv/consts/w.h"
#include "libc/sysv/errfuns.h"
#include "libc/thread/spawn.h"
#include "libc/thread/thread.h"
#include "libc/thread/tls.h"
#include "libc/x/x.h"
@ -151,12 +148,6 @@ __static_yoink("blink_linux_aarch64"); // for raspberry pi
__static_yoink("blink_xnu_aarch64"); // is apple silicon
#endif
#if !IsTiny()
#ifdef __x86_64__
__static_yoink("ShowCrashReportsEarly");
#endif
#endif
/**
* @fileoverview redbean - single-file distributable web server
*
@ -514,8 +505,8 @@ static struct Strings hidepaths;
static const char *launchbrowser;
static const char ctIdx = 'c'; // a pseudo variable to get address of
static struct spawn replth;
static struct spawn monitorth;
static pthread_t replth;
static pthread_t monitorth;
static struct Buffer inbuf_actual;
static struct Buffer inbuf;
static struct Buffer oldin;
@ -6549,7 +6540,10 @@ static int ExitWorker(void) {
}
if (monitortty) {
terminatemonitor = true;
_join(&monitorth);
if (monitorth) {
pthread_join(monitorth, 0);
monitorth = 0;
}
}
_Exit(0);
}
@ -6582,7 +6576,7 @@ static int EnableSandbox(void) {
}
}
static int MemoryMonitor(void *arg, int tid) {
static void *MemoryMonitor(void *arg) {
static struct termios oldterm;
static int tty;
sigset_t ss;
@ -6738,8 +6732,9 @@ static int MemoryMonitor(void *arg, int tid) {
}
static void MonitorMemory(void) {
if (_spawn(MemoryMonitor, 0, &monitorth) == -1) {
WARNF("(memv) failed to start memory monitor %m");
errno_t err;
if ((err = pthread_create(&monitorth, 0, MemoryMonitor, 0))) {
WARNF("(memv) failed to start memory monitor %s", strerror(err));
}
}
@ -7443,10 +7438,16 @@ void RedBean(int argc, char *argv[]) {
if (!isexitingworker) {
if (!IsTiny()) {
terminatemonitor = true;
_join(&monitorth);
if (monitorth) {
pthread_join(monitorth, 0);
monitorth = 0;
}
}
#ifndef STATIC
_join(&replth);
if (replth) {
pthread_join(replth, 0);
replth = 0;
}
#endif
HandleShutdown();
CallSimpleHookIfDefined("OnServerStop");
@ -7459,7 +7460,7 @@ void RedBean(int argc, char *argv[]) {
}
int main(int argc, char *argv[]) {
#if !IsTiny() && !defined(__x86_64__)
#if !IsTiny()
ShowCrashReports();
#endif
LoadZipArgs(&argc, &argv);
@ -7470,7 +7471,10 @@ int main(int argc, char *argv[]) {
// 2. unwound worker exit
if (IsModeDbg()) {
if (isexitingworker) {
_join(&replth);
if (replth) {
pthread_join(replth, 0);
replth = 0;
}
linenoiseDisableRawMode();
linenoiseHistoryFree();
}

506
tool/net/wb.c
View file

@ -1,506 +0,0 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
│vi: set net ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/assert.h"
#include "libc/calls/calls.h"
#include "libc/dns/dns.h"
#include "libc/errno.h"
#include "libc/fmt/conv.h"
#include "libc/log/check.h"
#include "libc/log/log.h"
#include "libc/macros.internal.h"
#include "libc/math.h"
#include "libc/mem/gc.h"
#include "libc/mem/mem.h"
#include "libc/sock/goodsocket.internal.h"
#include "libc/sock/sock.h"
#include "libc/stdio/append.h"
#include "libc/stdio/rand.h"
#include "libc/stdio/stdio.h"
#include "libc/str/slice.h"
#include "libc/str/str.h"
#include "libc/sysv/consts/af.h"
#include "libc/sysv/consts/ex.h"
#include "libc/sysv/consts/exit.h"
#include "libc/sysv/consts/ipproto.h"
#include "libc/sysv/consts/sig.h"
#include "libc/sysv/consts/so.h"
#include "libc/sysv/consts/sock.h"
#include "libc/sysv/consts/sol.h"
#include "libc/sysv/consts/tcp.h"
#include "libc/time/time.h"
#include "libc/x/x.h"
#include "libc/x/xsigaction.h"
#include "net/http/http.h"
#include "net/http/url.h"
#include "net/https/https.h"
#include "third_party/getopt/getopt.internal.h"
#include "third_party/mbedtls/ctr_drbg.h"
#include "third_party/mbedtls/debug.h"
#include "third_party/mbedtls/error.h"
#include "third_party/mbedtls/net_sockets.h"
#include "third_party/mbedtls/ssl.h"
#define OPTS "BIqksvzX:H:C:m:"
#define Micros(t) ((int64_t)((t)*1e6))
#define HasHeader(H) (!!msg.headers[H].a)
#define HeaderData(H) (inbuf.p + msg.headers[H].a)
#define HeaderLength(H) (msg.headers[H].b - msg.headers[H].a)
#define HeaderEqualCase(H, S) \
SlicesEqualCase(S, strlen(S), HeaderData(H), HeaderLength(H))
struct Buffer {
size_t n, c;
char *p;
};
struct Headers {
size_t n;
char **p;
} headers;
bool suiteb;
char *request;
bool isdone;
char *urlarg;
int method = kHttpGet;
bool authmode = MBEDTLS_SSL_VERIFY_NONE;
char *host;
char *port;
char *flags;
bool usessl;
uint32_t ip;
struct Url url;
struct addrinfo *addr;
struct Buffer inbuf;
long error_count;
long failure_count;
long message_count;
long connect_count;
double *latencies;
size_t latencies_n;
size_t latencies_c;
long double start_run;
long double end_run;
long double start_fetch;
long double end_fetch;
long connectionstobemade = 100;
long messagesperconnection = 100;
mbedtls_x509_crt *cachain;
mbedtls_ssl_config conf;
mbedtls_ssl_context ssl;
mbedtls_ctr_drbg_context drbg;
struct addrinfo hints = {.ai_family = AF_INET,
.ai_socktype = SOCK_STREAM,
.ai_protocol = IPPROTO_TCP,
.ai_flags = AI_NUMERICSERV};
void OnInt(int sig) {
isdone = true;
}
static int TlsSend(void *c, const unsigned char *p, size_t n) {
int rc;
if ((rc = write(*(int *)c, p, n)) == -1) {
if (errno == EINTR) {
return MBEDTLS_ERR_SSL_WANT_WRITE;
} else if (errno == EAGAIN) {
return MBEDTLS_ERR_SSL_TIMEOUT;
} else if (errno == EPIPE || errno == ECONNRESET || errno == ENETRESET) {
return MBEDTLS_ERR_NET_CONN_RESET;
} else {
VERBOSEF("tls write() error %s", strerror(errno));
return MBEDTLS_ERR_NET_RECV_FAILED;
}
}
return rc;
}
static int TlsRecv(void *c, unsigned char *p, size_t n, uint32_t o) {
int r;
if ((r = read(*(int *)c, p, n)) == -1) {
if (errno == EINTR) {
return MBEDTLS_ERR_SSL_WANT_READ;
} else if (errno == EAGAIN) {
return MBEDTLS_ERR_SSL_TIMEOUT;
} else if (errno == EPIPE || errno == ECONNRESET || errno == ENETRESET) {
return MBEDTLS_ERR_NET_CONN_RESET;
} else {
VERBOSEF("tls read() error %s", strerror(errno));
return MBEDTLS_ERR_NET_RECV_FAILED;
}
}
return r;
}
static wontreturn void PrintUsage(FILE *f, int rc) {
fprintf(f, "usage: %s [-%s] URL\n", OPTS, program_invocation_name);
fprintf(f, "wb - cosmopolitan http/https benchmark tool\n");
fprintf(f, " -C INT connections to be made\n");
fprintf(f, " -m INT messages per connection\n");
fprintf(f, " -B use suite b ciphersuites\n");
fprintf(f, " -v increase verbosity\n");
fprintf(f, " -H K:V append http header\n");
fprintf(f, " -X NAME specify http method\n");
fprintf(f, " -k verify ssl certs\n");
fprintf(f, " -I same as -X HEAD\n");
fprintf(f, " -z same as -H Accept-Encoding:gzip\n");
fprintf(f, " -h show this help\n");
exit(rc);
}
int fetch(void) {
int status;
ssize_t rc;
int t, ret, sock;
long messagesremaining;
struct HttpMessage msg;
struct HttpUnchunker u;
size_t g, n, hdrsize, paylen;
messagesremaining = messagesperconnection;
/*
* Setup crypto.
*/
if (usessl) {
mbedtls_ssl_session_reset(&ssl);
CHECK_EQ(0, mbedtls_ssl_set_hostname(&ssl, host));
}
/*
* Connect to server.
*/
InitHttpMessage(&msg, kHttpResponse);
ip = ntohl(((struct sockaddr_in *)addr->ai_addr)->sin_addr.s_addr);
CHECK_NE(-1, (sock = GoodSocket(addr->ai_family, addr->ai_socktype,
addr->ai_protocol, false, 0)));
if (connect(sock, addr->ai_addr, addr->ai_addrlen) == -1) {
goto TransportError;
}
if (usessl) {
mbedtls_ssl_set_bio(&ssl, &sock, TlsSend, 0, TlsRecv);
if ((ret = mbedtls_ssl_handshake(&ssl))) {
goto TransportError;
}
}
SendAnother:
/*
* Send HTTP Message.
*/
n = appendz(request).i;
if (usessl) {
ret = mbedtls_ssl_write(&ssl, request, n);
if (ret != n) goto TransportError;
} else if (write(sock, request, n) != n) {
goto TransportError;
}
/*
* Handle response.
*/
InitHttpMessage(&msg, kHttpResponse);
for (hdrsize = paylen = t = 0;;) {
if (inbuf.n == inbuf.c) {
inbuf.c += 1000;
inbuf.c += inbuf.c >> 1;
inbuf.p = realloc(inbuf.p, inbuf.c);
}
if (usessl) {
if ((rc = mbedtls_ssl_read(&ssl, inbuf.p + inbuf.n, inbuf.c - inbuf.n)) <
0) {
if (rc == MBEDTLS_ERR_SSL_PEER_CLOSE_NOTIFY) {
rc = 0;
} else {
goto TransportError;
}
}
} else if ((rc = read(sock, inbuf.p + inbuf.n, inbuf.c - inbuf.n)) == -1) {
goto TransportError;
}
g = rc;
inbuf.n += g;
switch (t) {
case kHttpClientStateHeaders:
if (!g) goto TransportError;
rc = ParseHttpMessage(&msg, inbuf.p, inbuf.n);
if (rc == -1) goto TransportError;
if (rc) {
hdrsize = rc;
if (100 <= msg.status && msg.status <= 199) {
if ((HasHeader(kHttpContentLength) &&
!HeaderEqualCase(kHttpContentLength, "0")) ||
(HasHeader(kHttpTransferEncoding) &&
!HeaderEqualCase(kHttpTransferEncoding, "identity"))) {
goto TransportError;
}
DestroyHttpMessage(&msg);
InitHttpMessage(&msg, kHttpResponse);
memmove(inbuf.p, inbuf.p + hdrsize, inbuf.n - hdrsize);
inbuf.n -= hdrsize;
break;
}
if (msg.status == 204 || msg.status == 304) {
goto Finished;
}
if (HasHeader(kHttpTransferEncoding) &&
!HeaderEqualCase(kHttpTransferEncoding, "identity")) {
if (HeaderEqualCase(kHttpTransferEncoding, "chunked")) {
t = kHttpClientStateBodyChunked;
bzero(&u, sizeof(u));
goto Chunked;
} else {
goto TransportError;
}
} else if (HasHeader(kHttpContentLength)) {
rc = ParseContentLength(HeaderData(kHttpContentLength),
HeaderLength(kHttpContentLength));
if (rc == -1) goto TransportError;
if ((paylen = rc) <= inbuf.n - hdrsize) {
goto Finished;
} else {
t = kHttpClientStateBodyLengthed;
}
} else {
t = kHttpClientStateBody;
}
}
break;
case kHttpClientStateBody:
if (!g) {
paylen = inbuf.n;
goto Finished;
}
break;
case kHttpClientStateBodyLengthed:
if (!g) goto TransportError;
if (inbuf.n - hdrsize >= paylen) goto Finished;
break;
case kHttpClientStateBodyChunked:
Chunked:
rc = Unchunk(&u, inbuf.p + hdrsize, inbuf.n - hdrsize, &paylen);
if (rc == -1) goto TransportError;
if (rc) goto Finished;
break;
default:
__builtin_unreachable();
}
}
Finished:
status = msg.status;
DestroyHttpMessage(&msg);
if (!isdone && status == 200 && --messagesremaining > 0) {
long double now = nowl();
end_fetch = now;
++message_count;
latencies = realloc(latencies, ++latencies_n * sizeof(*latencies));
latencies[latencies_n - 1] = end_fetch - start_fetch;
start_fetch = now;
goto SendAnother;
}
close(sock);
return status;
TransportError:
close(sock);
DestroyHttpMessage(&msg);
return 900;
}
int main(int argc, char *argv[]) {
xsigaction(SIGPIPE, SIG_IGN, 0, 0, 0);
xsigaction(SIGINT, OnInt, 0, 0, 0);
/*
* Read flags.
*/
int opt;
__log_level = kLogWarn;
while ((opt = getopt(argc, argv, OPTS)) != -1) {
switch (opt) {
case 's':
case 'q':
break;
case 'B':
suiteb = true;
appendf(&flags, " -B");
break;
case 'v':
++__log_level;
break;
case 'I':
method = kHttpHead;
appendf(&flags, " -I");
break;
case 'H':
headers.p = realloc(headers.p, ++headers.n * sizeof(*headers.p));
headers.p[headers.n - 1] = optarg;
appendf(&flags, " -H '%s'", optarg);
break;
case 'z':
headers.p = realloc(headers.p, ++headers.n * sizeof(*headers.p));
headers.p[headers.n - 1] = "Accept-Encoding: gzip";
appendf(&flags, " -z");
break;
case 'X':
CHECK((method = GetHttpMethod(optarg, strlen(optarg))));
appendf(&flags, " -X %s", optarg);
break;
case 'k':
authmode = MBEDTLS_SSL_VERIFY_REQUIRED;
appendf(&flags, " -k");
break;
case 'm':
messagesperconnection = strtol(optarg, 0, 0);
break;
case 'C':
connectionstobemade = strtol(optarg, 0, 0);
break;
case 'h':
PrintUsage(stdout, EXIT_SUCCESS);
default:
PrintUsage(stderr, EX_USAGE);
}
}
appendf(&flags, " -m %ld", messagesperconnection);
appendf(&flags, " -C %ld", connectionstobemade);
if (optind == argc) PrintUsage(stdout, EXIT_SUCCESS);
urlarg = argv[optind];
cachain = GetSslRoots();
long connectsremaining = connectionstobemade;
/*
* Parse URL.
*/
_gc(ParseUrl(urlarg, -1, &url, kUrlPlus));
_gc(url.params.p);
usessl = false;
if (url.scheme.n) {
if (url.scheme.n == 5 && !memcasecmp(url.scheme.p, "https", 5)) {
usessl = true;
} else if (!(url.scheme.n == 4 && !memcasecmp(url.scheme.p, "http", 4))) {
FATALF("bad scheme");
}
}
if (url.host.n) {
host = _gc(strndup(url.host.p, url.host.n));
if (url.port.n) {
port = _gc(strndup(url.port.p, url.port.n));
} else {
port = usessl ? "443" : "80";
}
} else {
host = "127.0.0.1";
port = "80";
}
CHECK(IsAcceptableHost(host, -1));
url.fragment.p = 0, url.fragment.n = 0;
url.scheme.p = 0, url.scheme.n = 0;
url.user.p = 0, url.user.n = 0;
url.pass.p = 0, url.pass.n = 0;
url.host.p = 0, url.host.n = 0;
url.port.p = 0, url.port.n = 0;
if (!url.path.n || url.path.p[0] != '/') {
char *p = _gc(xmalloc(1 + url.path.n));
mempcpy(mempcpy(p, "/", 1), url.path.p, url.path.n);
url.path.p = p;
++url.path.n;
}
/*
* Create HTTP message.
*/
appendf(&request,
"%s %s HTTP/1.1\r\n"
"Host: %s:%s\r\n",
kHttpMethod[method], _gc(EncodeUrl(&url, 0)), host, port);
for (int i = 0; i < headers.n; ++i) {
appendf(&request, "%s\r\n", headers.p[i]);
}
appendf(&request, "\r\n");
/*
* Perform DNS lookup.
*/
int rc;
if ((rc = getaddrinfo(host, port, &hints, &addr)) != EAI_SUCCESS) {
FATALF("getaddrinfo(%s:%s) failed", host, port);
}
/*
* Setup SSL crypto.
*/
mbedtls_ssl_init(&ssl);
mbedtls_ctr_drbg_init(&drbg);
mbedtls_ssl_config_init(&conf);
CHECK_EQ(0, mbedtls_ctr_drbg_seed(&drbg, GetEntropy, 0, "justine", 7));
CHECK_EQ(0,
mbedtls_ssl_config_defaults(
&conf, MBEDTLS_SSL_IS_CLIENT, MBEDTLS_SSL_TRANSPORT_STREAM,
suiteb ? MBEDTLS_SSL_PRESET_SUITEB : MBEDTLS_SSL_PRESET_SUITEC));
mbedtls_ssl_conf_authmode(&conf, authmode);
mbedtls_ssl_conf_ca_chain(&conf, cachain, 0);
mbedtls_ssl_conf_rng(&conf, mbedtls_ctr_drbg_random, &drbg);
CHECK_EQ(0, mbedtls_ssl_setup(&ssl, &conf));
int status;
latencies_c = 1024;
latencies = malloc(latencies_c * sizeof(*latencies));
start_run = nowl();
while (!isdone && --connectsremaining >= 0) {
start_fetch = nowl();
status = fetch();
end_fetch = nowl();
if (status == 200) {
++connect_count;
++message_count;
latencies = realloc(latencies, ++latencies_n * sizeof(*latencies));
latencies[latencies_n - 1] = end_fetch - start_fetch;
} else if (status == 900) {
++failure_count;
} else {
++error_count;
}
}
end_run = nowl();
double latencies_sum = fsum(latencies, latencies_n);
double avg_latency = latencies_sum / message_count;
printf("wb%s\n", flags);
printf("msgs / second: %,ld qps\n",
(int64_t)(message_count / (end_run - start_run)));
printf("run time: %,ldµs\n", Micros(end_run - start_run));
printf("latency / msgs: %,ldµs\n", Micros(avg_latency));
printf("message count: %,ld\n", message_count);
printf("connect count: %,ld\n", connect_count);
printf("error count: %,ld (non-200 responses)\n", error_count);
printf("failure count: %,ld (transport error)\n", failure_count);
return 0;
}

18
tool/viz/lib/ycbcr2rgb3.c
View file

@ -25,8 +25,10 @@
#include "dsp/core/illumination.h"
#include "dsp/core/q.h"
#include "dsp/scale/scale.h"
#include "libc/assert.h"
#include "libc/calls/calls.h"
#include "libc/calls/struct/sigset.h"
#include "libc/calls/struct/timespec.h"
#include "libc/intrin/bsr.h"
#include "libc/intrin/pmulhrsw.h"
#include "libc/log/check.h"
@ -64,7 +66,7 @@ const double kSrgbToXyz[3][3] = {
long magikarp_latency_;
long gyarados_latency_;
long ycbcr2rgb_latency_;
long double magikarp_start_;
struct timespec magikarp_start_;
struct YCbCr {
bool yonly;
@ -261,15 +263,14 @@ void YCbCrConvert(struct YCbCr *me, long yn, long xn,
const unsigned char Y[restrict yys][yxs], long cys, long cxs,
unsigned char Cb[restrict cys][cxs],
unsigned char Cr[restrict cys][cxs]) {
long double ts;
ts = nowl();
struct timespec ts = timespec_real();
if (!me->yonly) {
YCbCr2Rgb(yn, xn, RGB, yys, yxs, Y, cys, cxs, Cb, Cr, me->magnums,
me->lighting, me->transfer[pf10_]);
} else {
Y2Rgb(yn, xn, RGB, yys, yxs, Y, me->magnums, me->transfer[pf10_]);
}
ycbcr2rgb_latency_ = lroundl((nowl() - ts) * 1e6l);
ycbcr2rgb_latency_ = timespec_tomicros(timespec_sub(timespec_real(), ts));
}
void YCbCr2RgbScaler(struct YCbCr *me, long dyn, long dxn,
@ -279,7 +280,6 @@ void YCbCr2RgbScaler(struct YCbCr *me, long dyn, long dxn,
unsigned char Cr[restrict cys][cxs], long yyn, long yxn,
long cyn, long cxn, double syn, double sxn, double pry,
double prx) {
long double ts;
long scyn, scxn;
double yry, yrx, cry, crx, yoy, yox, coy, cox;
scyn = syn * cyn / yyn;
@ -297,8 +297,8 @@ void YCbCr2RgbScaler(struct YCbCr *me, long dyn, long dxn,
Magkern2xY(cys, cxs, Cr, scyn, scxn), HALF(yyn), yxn,
HALF(cyn), scxn, syn / 2, sxn, pry, prx);
} else {
magikarp_latency_ = lroundl((nowl() - magikarp_start_) * 1e6l);
ts = nowl();
struct timespec ts = timespec_real();
magikarp_latency_ = timespec_tomicros(timespec_sub(ts, magikarp_start_));
yry = syn / dyn;
yrx = sxn / dxn;
cry = syn * cyn / yyn / dyn;
@ -326,7 +326,7 @@ void YCbCr2RgbScaler(struct YCbCr *me, long dyn, long dxn,
me->chroma.cy, me->chroma.cx, false);
GyaradosUint8(cys, cxs, Cr, cys, cxs, Cr, dyn, dxn, scyn, scxn, 0, 255,
me->chroma.cy, me->chroma.cx, false);
gyarados_latency_ = lround((nowl() - ts) * 1e6l);
gyarados_latency_ = timespec_tomicros(timespec_sub(timespec_real(), ts));
YCbCrConvert(me, dyn, dxn, RGB, yys, yxs, Y, cys, cxs, Cb, Cr);
INFOF("done");
}
@ -381,7 +381,7 @@ void *YCbCr2RgbScale(long dyn, long dxn,
CHECK_LE(cyn, cys);
CHECK_LE(cxn, cxs);
INFOF("magikarp2x");
magikarp_start_ = nowl();
magikarp_start_ = timespec_real();
minyys = MAX(ceil(syn), MAX(yyn, ceil(dyn * pry)));
minyxs = MAX(ceil(sxn), MAX(yxn, ceil(dxn * prx)));
mincys = MAX(cyn, ceil(dyn * pry));

11
tool/viz/life.c
View file

@ -1094,13 +1094,12 @@ static bool HasPendingInput(void) {
}
static bool ShouldDraw(void) {
long double now, rate;
static long double next;
struct timespec now;
static struct timespec next;
if (!isdragging) return true;
now = nowl();
rate = 1. / 24;
if (now > next && !HasPendingInput()) {
next = now + rate;
now = timespec_real();
if (timespec_cmp(now, next) > 0 && !HasPendingInput()) {
next = timespec_add(now, timespec_frommicros(1. / 24 * 1e6));
return true;
} else {
return false;

12
tool/viz/memzoom.c
View file

@ -331,12 +331,12 @@ static long Index(long y, long x) {
}
static void PreventBufferbloat(void) {
long double now, rate;
static long double last;
now = nowl();
rate = 1. / fps;
if (now - last < rate) {
dsleep(rate - (now - last));
struct timespec now, rate;
static struct timespec last;
now = timespec_real();
rate = timespec_frommicros(1. / fps * 1e6);
if (timespec_cmp(timespec_sub(now, last), rate) < 0) {
timespec_sleep(timespec_sub(rate, timespec_sub(now, last)));
}
last = now;
}

101
tool/viz/printvideo.c
View file

@ -33,6 +33,7 @@
#include "libc/calls/struct/sigaction.h"
#include "libc/calls/struct/siginfo.h"
#include "libc/calls/struct/sigset.h"
#include "libc/calls/struct/timespec.h"
#include "libc/calls/struct/winsize.h"
#include "libc/calls/termios.h"
#include "libc/calls/ucontext.h"
@ -42,6 +43,7 @@
#include "libc/fmt/fmt.h"
#include "libc/fmt/itoa.h"
#include "libc/intrin/bits.h"
#include "libc/intrin/kprintf.h"
#include "libc/intrin/safemacros.internal.h"
#include "libc/intrin/xchg.internal.h"
#include "libc/log/check.h"
@ -88,6 +90,7 @@
#include "libc/sysv/consts/termios.h"
#include "libc/sysv/consts/w.h"
#include "libc/sysv/errfuns.h"
#include "libc/thread/thread.h"
#include "libc/time/time.h"
#include "libc/x/xsigaction.h"
#include "third_party/getopt/getopt.internal.h"
@ -174,11 +177,11 @@ mode.\n\
*(B) = pvalloc(N); \
})
#define TIMEIT(OUT_NANOS, FORM) \
do { \
long double Start = nowl(); \
FORM; \
(OUT_NANOS) = (uint64_t)((nowl() - Start) * 1e9L); \
#define TIMEIT(OUT_NANOS, FORM) \
do { \
struct timespec Start = timespec_real(); \
FORM; \
(OUT_NANOS) = timespec_tonanos(timespec_sub(timespec_real(), Start)); \
} while (0)
typedef bool (*openspeaker_f)(void);
@ -278,9 +281,9 @@ static uint64_t t1, t2, t3, t4, t5, t6, t8;
static const char *sox_, *ffplay_, *patharg_;
static struct VtFrame vtframe_[2], *f1_, *f2_;
static struct Graphic graphic_[2], *g1_, *g2_;
static long double deadline_, dura_, starttime_;
static struct timespec deadline_, dura_, starttime_;
static bool yes_, stats_, dither_, ttymode_, istango_;
static long double decode_start_, f1_start_, f2_start_;
static struct timespec decode_start_, f1_start_, f2_start_;
static int16_t pcm_[PLM_AUDIO_SAMPLES_PER_FRAME * 2 / 8][8];
static int16_t pcmscale_[PLM_AUDIO_SAMPLES_PER_FRAME * 2 / 8][8];
static bool fullclear_, historyclear_, tuned_, yonly_, gotvideo_;
@ -308,23 +311,18 @@ static void StrikeDownCrapware(int sig) {
kill(playpid_, SIGKILL);
}
static long AsMilliseconds(long double ts) {
return rintl(ts * 1e3);
}
static long AsNanoseconds(long double ts) {
return rintl(ts * 1e9);
}
static long double GetGraceTime(void) {
return deadline_ - nowl();
static struct timespec GetGraceTime(void) {
return timespec_sub(deadline_, timespec_real());
}
static int GetNamedVector(const struct NamedVector *choices, size_t n,
const char *s) {
int i;
char name[sizeof(choices->name)];
strlcpy(name, s, sizeof(name));
#pragma GCC push_options
#pragma GCC diagnostic ignored "-Wstringop-truncation"
strncpy(name, s, sizeof(name));
#pragma GCC pop_options
strntoupper(name, sizeof(name));
for (i = 0; i < n; ++i) {
if (memcmp(choices[i].name, name, sizeof(name)) == 0) {
@ -345,7 +343,7 @@ static int GetLighting(const char *s) {
static bool CloseSpeaker(void) {
int rc, wstatus;
rc = 0;
sched_yield();
pthread_yield();
if (playfd_) {
rc |= close(playfd_);
playfd_ = -1;
@ -375,8 +373,13 @@ static void ResizeVtFrame(struct VtFrame *f, size_t yn, size_t xn) {
f->i = f->n = 0;
}
static float timespec_tofloat(struct timespec ts) {
return ts.tv_sec + ts.tv_nsec * 1e-9;
}
static void RecordFactThatFrameWasFullyRendered(void) {
fcring_.p[fcring_.i] = nowl() - starttime_;
fcring_.p[fcring_.i] =
timespec_tofloat(timespec_sub(timespec_real(), starttime_));
fcring_.n += 1;
fcring_.i += 1;
fcring_.i &= ARRAYLEN(fcring_.p) - 1;
@ -521,7 +524,7 @@ static bool OpenSpeaker(void) {
if (ffplay_) tryspeakerfns_[i++] = TryFfplay;
if (sox_) tryspeakerfns_[i++] = TrySox;
}
snprintf(fifopath_, sizeof(fifopath_), "%s%s.%d.%d.wav", kTmpPath,
snprintf(fifopath_, sizeof(fifopath_), "%s%s.%d.%d.wav", __get_tmpdir(),
firstnonnull(program_invocation_short_name, "unknown"), getpid(),
count);
for (i = 0; i < ARRAYLEN(tryspeakerfns_); ++i) {
@ -539,7 +542,7 @@ static bool OpenSpeaker(void) {
static void OnAudio(plm_t *mpeg, plm_samples_t *samples, void *user) {
if (playfd_ != -1) {
DEBUGF("OnAudio() [grace=%,ldns]", AsNanoseconds(GetGraceTime()));
DEBUGF("OnAudio() [grace=%,ldns]", timespec_tonanos(GetGraceTime()));
CHECK_EQ(2, chans_);
CHECK_EQ(ARRAYLEN(pcm_) * 8, samples->count * chans_);
float2short(ARRAYLEN(pcm_), pcm_, (void *)samples->interleaved);
@ -549,7 +552,7 @@ static void OnAudio(plm_t *mpeg, plm_samples_t *samples, void *user) {
TryAgain:
if (WriteAudio(playfd_, pcm_, sizeof(pcm_), 1000) != -1) {
DEBUGF("WriteAudio(%d, %zu) ok [grace=%,ldns]", playfd_,
samples->count * 2, AsNanoseconds(GetGraceTime()));
samples->count * 2, timespec_tonanos(GetGraceTime()));
} else {
WARNF("WriteAudio(%d, %zu) failed: %s", playfd_, samples->count * 2,
strerror(errno));
@ -765,7 +768,7 @@ static void RasterIt(void) {
static void TranscodeVideo(plm_frame_t *pf) {
CHECK_EQ(pf->cb.width, pf->cr.width);
CHECK_EQ(pf->cb.height, pf->cr.height);
DEBUGF("TranscodeVideo() [grace=%,ldns]", AsNanoseconds(GetGraceTime()));
DEBUGF("TranscodeVideo() [grace=%,ldns]", timespec_tonanos(GetGraceTime()));
g2_ = &graphic_[1];
t5 = 0;
@ -879,8 +882,9 @@ static ssize_t WriteVideoCall(void) {
if (plm_get_audio_enabled(plm_)) {
plm_set_audio_lead_time(
plm_,
MAX(0, MIN(nowl() - f1_start_, plm_get_samplerate(plm_) /
PLM_AUDIO_SAMPLES_PER_FRAME)));
max(0,
min(timespec_tofloat(timespec_sub(timespec_real(), f1_start_)),
plm_get_samplerate(plm_) / PLM_AUDIO_SAMPLES_PER_FRAME)));
}
f1_start_ = f2_start_;
f1_->i = f1_->n = 0;
@ -904,10 +908,10 @@ static void DrainVideo(void) {
static void WriteVideo(void) {
ssize_t rc;
DEBUGF("write(tty) grace=%,ldns", AsNanoseconds(GetGraceTime()));
DEBUGF("write(tty) grace=%,ldns", timespec_tonanos(GetGraceTime()));
if ((rc = WriteVideoCall()) != -1) {
DEBUGF("write(tty) → %zd [grace=%,ldns]", rc,
AsNanoseconds(GetGraceTime()));
timespec_tonanos(GetGraceTime()));
} else if (errno == EAGAIN || errno == EINTR) {
DEBUGF("write(tty) → EINTR");
longjmp(jbi_, 1);
@ -1269,11 +1273,11 @@ static void PerformBestEffortIo(void) {
{infd_, POLLIN},
{outfd_, f1_ && f1_->n ? POLLOUT : 0},
};
pollms = MAX(0, AsMilliseconds(GetGraceTime()));
pollms = MAX(0, timespec_tomillis(GetGraceTime()));
DEBUGF("poll() ms=%,d", pollms);
if ((toto = poll(fds, ARRAYLEN(fds), pollms)) != -1) {
DEBUGF("poll() toto=%d [grace=%,ldns]", toto,
AsNanoseconds(GetGraceTime()));
timespec_tonanos(GetGraceTime()));
if (toto) {
if (fds[0].revents & (POLLIN | POLLERR)) ReadKeyboard();
if (fds[1].revents & (POLLOUT | POLLERR)) WriteVideo();
@ -1305,32 +1309,36 @@ static void HandleSignals(void) {
}
static void PrintVideo(void) {
long double decode_last, decode_end, next_tick, lag;
dura_ = MIN(MAX_FRAMERATE, 1 / plm_get_framerate(plm_));
struct timespec decode_last, decode_end, next_tick, lag;
dura_ = timespec_frommicros(min(MAX_FRAMERATE, 1 / plm_get_framerate(plm_)) *
1e6);
INFOF("framerate=%f dura=%f", plm_get_framerate(plm_), dura_);
next_tick = deadline_ = decode_last = nowl();
next_tick += dura_;
deadline_ += dura_;
next_tick = deadline_ = decode_last = timespec_real();
next_tick = timespec_add(next_tick, dura_);
deadline_ = timespec_add(deadline_, dura_);
do {
DEBUGF("plm_decode [grace=%,ldns]", AsNanoseconds(GetGraceTime()));
decode_start_ = nowl();
plm_decode(plm_, decode_start_ - decode_last);
DEBUGF("plm_decode [grace=%,ldns]", timespec_tonanos(GetGraceTime()));
decode_start_ = timespec_real();
plm_decode(plm_,
timespec_tofloat(timespec_sub(decode_start_, decode_last)));
decode_last = decode_start_;
decode_end = nowl();
lag = decode_end - decode_start_;
while (decode_end + lag > next_tick) next_tick += dura_;
deadline_ = next_tick - lag;
decode_end = timespec_real();
lag = timespec_sub(decode_end, decode_start_);
while (timespec_cmp(timespec_add(decode_end, lag), next_tick) > 0) {
next_tick = timespec_add(next_tick, dura_);
}
deadline_ = timespec_sub(next_tick, lag);
if (gotvideo_ || !plm_get_video_enabled(plm_)) {
gotvideo_ = false;
INFOF("entering printvideo event loop (lag=%,ldns, grace=%,ldns)",
AsNanoseconds(lag), AsNanoseconds(GetGraceTime()));
timespec_tonanos(lag), timespec_tonanos(GetGraceTime()));
}
do {
if (!setjmp(jbi_)) {
PerformBestEffortIo();
}
HandleSignals();
} while (AsMilliseconds(GetGraceTime()) > 0);
} while (timespec_tomillis(GetGraceTime()) > 0);
} while (plm_ && !plm_has_ended(plm_));
}
@ -1366,7 +1374,7 @@ static void PrintUsage(int rc, FILE *f) {
static void GetOpts(int argc, char *argv[]) {
int opt;
snprintf(logpath_, sizeof(logpath_), "%s%s.log", kTmpPath,
snprintf(logpath_, sizeof(logpath_), "%s%s.log", __get_tmpdir(),
firstnonnull(program_invocation_short_name, "unknown"));
while ((opt = getopt(argc, argv, "?34AGSTVYabdfhnpstxyzvL:")) != -1) {
switch (opt) {
@ -1533,6 +1541,7 @@ static void TryToOpenFrameBuffer(void) {
int main(int argc, char *argv[]) {
sigset_t wut;
const char *s;
ShowCrashReports();
gamma_ = 2.4;
volscale_ -= 2;
dither_ = true;
@ -1581,7 +1590,7 @@ int main(int argc, char *argv[]) {
if (t2 > t1) longjmp(jb_, 1);
OpenVideo();
DimensionDisplay();
starttime_ = nowl();
starttime_ = timespec_real();
PrintVideo();
}
INFOF("jb_ triggered");

2
tool/viz/tailf.c
View file

@ -93,7 +93,7 @@ int main(int argc, char *argv[]) {
}
chopped = j != n;
}
dsleep(.01);
usleep(10000);
}
close(fd);
WriteString("\r\n");

2
tool/viz/viz.mk
View file

@ -31,12 +31,14 @@ TOOL_VIZ_DIRECTDEPS = \
LIBC_NT_GDI32 \
LIBC_NT_KERNEL32 \
LIBC_NT_USER32 \
LIBC_PROC \
LIBC_RUNTIME \
LIBC_SOCK \
LIBC_STDIO \
LIBC_STR \
LIBC_SYSV \
LIBC_SYSV_CALLS \
LIBC_THREAD \
LIBC_TIME \
LIBC_TINYMATH \
LIBC_X \