Compare commits

...

327 commits

Author SHA1 Message Date
Hugues Morisset
f1e83d5240
Add IPv6 support to getifaddrs() on Linux (#1415)
Some checks failed
build / matrix_on_mode () (push) Has been cancelled
build / matrix_on_mode (optlinux) (push) Has been cancelled
build / matrix_on_mode (rel) (push) Has been cancelled
build / matrix_on_mode (tiny) (push) Has been cancelled
build / matrix_on_mode (tinylinux) (push) Has been cancelled
2025-05-21 01:20:22 -07:00
Steven Dee (Jōshin)
2fe8338f92
Better mtimes for github workflow build cache (#1421)
Saves and restores mtimes to a file, also covering the `o/` directory to
hopefully preserve make dependency information better.
2025-05-20 22:17:55 -07:00
ShalokShalom
4ca513cba2
Add C++ to README (#1407)
Some checks failed
build / matrix_on_mode () (push) Has been cancelled
build / matrix_on_mode (optlinux) (push) Has been cancelled
build / matrix_on_mode (rel) (push) Has been cancelled
build / matrix_on_mode (tiny) (push) Has been cancelled
build / matrix_on_mode (tinylinux) (push) Has been cancelled
2025-04-25 15:47:50 -07:00
Steven Dee (Jōshin)
455910e8f2
Make more shared_ptr fixes (#1401)
Some checks failed
build / matrix_on_mode () (push) Has been cancelled
build / matrix_on_mode (optlinux) (push) Has been cancelled
build / matrix_on_mode (rel) (push) Has been cancelled
build / matrix_on_mode (tiny) (push) Has been cancelled
build / matrix_on_mode (tinylinux) (push) Has been cancelled
* Make refcount reads explicitly atomic
* Consistently put `const` in the same place
* Write the general `operator=` on `weak_ptr`
2025-04-21 05:36:50 -07:00
Steven Dee (Jōshin)
9c68bc19b5
Cache .cosmocc and o for github workflows (#1400)
Some checks failed
build / matrix_on_mode () (push) Has been cancelled
build / matrix_on_mode (optlinux) (push) Has been cancelled
build / matrix_on_mode (rel) (push) Has been cancelled
build / matrix_on_mode (tiny) (push) Has been cancelled
build / matrix_on_mode (tinylinux) (push) Has been cancelled
Uses GitHub’s actions/cache@v4 to store the cosmocc distribution and the
output directory between runs of the build workflow, with the version of
cosmocc as the cache key.

Upgrades to actions/checkout@v4.
2025-04-17 15:55:27 -07:00
Steven Dee (Jōshin)
66d1050af6
Correctly implement weak_ptr assignment/copy/moves (#1399) 2025-04-17 14:01:20 -07:00
Justine Tunney
fbc4fcbb71
Get GDB working
Some checks failed
build / matrix_on_mode () (push) Has been cancelled
build / matrix_on_mode (optlinux) (push) Has been cancelled
build / matrix_on_mode (rel) (push) Has been cancelled
build / matrix_on_mode (tiny) (push) Has been cancelled
build / matrix_on_mode (tinylinux) (push) Has been cancelled
You can now say `gdb hello.com.dbg` and it'll work perfectly.
2025-03-30 15:25:55 -07:00
Steven Dee (Jōshin)
afc986f741
Fix shared_ptr<T>::owner_before (#1390)
Some checks failed
build / matrix_on_mode () (push) Has been cancelled
build / matrix_on_mode (optlinux) (push) Has been cancelled
build / matrix_on_mode (rel) (push) Has been cancelled
build / matrix_on_mode (tiny) (push) Has been cancelled
build / matrix_on_mode (tinylinux) (push) Has been cancelled
`!(a < b)` is not the same as `b < a`.

I think I originally wrote it this way to avoid making weak_ptr a friend
of shared_ptr, but weak_ptr already is a friend.
2025-03-25 01:49:34 -04:00
Derek
5eb7cd6643
Add support for getcpu() system call to pledge() (#1387)
Some checks failed
build / matrix_on_mode () (push) Has been cancelled
build / matrix_on_mode (optlinux) (push) Has been cancelled
build / matrix_on_mode (rel) (push) Has been cancelled
build / matrix_on_mode (tiny) (push) Has been cancelled
build / matrix_on_mode (tinylinux) (push) Has been cancelled
This fixes redbean Lua tests which were failing with SIGSYS on Linux.
2025-03-21 16:08:25 -07:00
Brett Jia
a8ed4fdd09
Add NetBSD evbarm and fix segfault (#1384)
Some checks failed
build / matrix_on_mode () (push) Has been cancelled
build / matrix_on_mode (optlinux) (push) Has been cancelled
build / matrix_on_mode (rel) (push) Has been cancelled
build / matrix_on_mode (tiny) (push) Has been cancelled
build / matrix_on_mode (tinylinux) (push) Has been cancelled
This change fixes a segmentation fault when comparing loaders that don't
have a target kernel set. Additionally, adds evbarm, which is the output
of uname -m on NetBSD on aarch64.
2025-03-12 17:37:46 -07:00
Brett Jia
7b69652854
Add -k OSNAME flag to apelink (#1383)
Let's say you pass the `-M blink-mips.elf` flag to apelink, so that your
ape binary will bundle a compressed build of blink, and the shell script
will extract that binary and launch your program under it, if running on
a MIPS system. However, for any given microprocessor architecture, we'll
need a separate loader for each operating system. The issue is ELF OSABI
isn't very useful. As an example, SerenityOS and Linux both have SYSV in
the OSABI field. So to tell their binaries apart we'd have to delve into
various other conventions, like special sections and PT_NOTE structures.

To make things simple this change introduces the `-k OS` flag to apelink
which generate shell script content that ensures `OS` matches `uname -s`
before attempting to execute a loader. For example, you could say:

    apelink -k Linux -M blink-linux-arm.elf -M blink-linux-mips.elf \
            -k Darwin -M blink-darwin-ppc.elf \
            ...

To introduce support for old 32-bit architectures on multiple OSes, when
building your cosmo binary.
2025-03-12 13:26:51 -07:00
Leal G.
b235492e71
Add usertrust certificate (#1382)
Some checks are pending
build / matrix_on_mode () (push) Waiting to run
build / matrix_on_mode (optlinux) (push) Waiting to run
build / matrix_on_mode (rel) (push) Waiting to run
build / matrix_on_mode (tiny) (push) Waiting to run
build / matrix_on_mode (tinylinux) (push) Waiting to run
Bundle USERTrust CA certificates to /usr/share/ssl/root for TLS verifies
2025-03-11 17:59:34 -07:00
Brett Jia
fc81fd8d16
Support additional architectures in apelink (#1381)
Some checks failed
build / matrix_on_mode () (push) Has been cancelled
build / matrix_on_mode (optlinux) (push) Has been cancelled
build / matrix_on_mode (rel) (push) Has been cancelled
build / matrix_on_mode (tiny) (push) Has been cancelled
build / matrix_on_mode (tinylinux) (push) Has been cancelled
This updates apelink to support machine architectures not in the source
program input list by adding additional loaders, extracting the correct
one that matches the host uname machine. With this change, blink can be
supplied as the additional loader to run the program in x86_64 VMs. The
change has been verified against blink 1.0, powerpc64le and mips64el in
Docker using QEMU.
2025-03-06 10:26:31 -08:00
Gautham
38930de8e0
Make tool for replacing ELF strings (#1344)
Some checks failed
build / matrix_on_mode () (push) Has been cancelled
build / matrix_on_mode (optlinux) (push) Has been cancelled
build / matrix_on_mode (rel) (push) Has been cancelled
build / matrix_on_mode (tiny) (push) Has been cancelled
build / matrix_on_mode (tinylinux) (push) Has been cancelled
2025-02-08 21:17:42 -08:00
Brett Jia
0e557d041d
Check downloaded gcc/clang checksums (#1367)
Check sha256 checksums of the downloaded gcc and clang toolchains. It'll
allow us to extend trust to external toolchains if building from source.
2025-02-08 17:46:09 -08:00
Brett Jia
1d676b36e6
Make cosmoranlib executable (#1366)
Fixes #1325
2025-02-08 17:38:00 -08:00
Brett Jia
10a92cee94
Support building cosmocc on MacOS (#1365)
Some checks are pending
build / matrix_on_mode () (push) Waiting to run
build / matrix_on_mode (optlinux) (push) Waiting to run
build / matrix_on_mode (rel) (push) Waiting to run
build / matrix_on_mode (tiny) (push) Waiting to run
build / matrix_on_mode (tinylinux) (push) Waiting to run
This updates the cosmocc toolchain packaging script to work on MacOS. It
has been tested on GitHub Actions macos-13 (x86_64) and macos-14 (arm64)
runners, and is verified to still work on Ubuntu (GitHub Actions runners
ubuntu-24.04 and ubuntu-24.04-arm). It'll help bring cosmocc to MacPorts
by running the packaging script. We favor `gmake` rather than the `make`
command because it distinguishes GNU Make from BSD Make, and Xcode Make.
Additionally, APE loader from the bootstrapper toolchain is used instead
of a system APE, which may not be available.
2025-02-08 12:45:45 -08:00
A2va
42a9ed0131
Adds some NT functions (#1358) 2025-02-08 08:08:08 -08:00
Björn Buckwalter
12cb0669fb
Clarify unix.mapshared versus file locks (#1355)
Some checks are pending
build / matrix_on_mode () (push) Waiting to run
build / matrix_on_mode (optlinux) (push) Waiting to run
build / matrix_on_mode (rel) (push) Waiting to run
build / matrix_on_mode (tiny) (push) Waiting to run
build / matrix_on_mode (tinylinux) (push) Waiting to run
2025-02-08 00:48:38 -08:00
rufeooo
7f6a7d6fff
Fix sigaction example code (#1363)
Some checks are pending
build / matrix_on_mode () (push) Waiting to run
build / matrix_on_mode (optlinux) (push) Waiting to run
build / matrix_on_mode (rel) (push) Waiting to run
build / matrix_on_mode (tiny) (push) Waiting to run
build / matrix_on_mode (tinylinux) (push) Waiting to run
2025-02-07 11:42:47 -08:00
Steven Dee (Jōshin)
9f6bf6ea71
tool/zsh/mkofs: doas 2025-01-13 16:48:55 -08:00
Steven Dee (Jōshin)
102edf4ea2
tool/zsh/mmake: style 2025-01-05 20:53:53 -08:00
Steven Dee (Jōshin)
21968acf99
Standard make path (#1353)
Modifies download-cosmocc.sh to maintain a .cosmocc/current symlink that
always points to the most recently downloaded version of cosmocc. We can
use this to point at a canonical make for a bootstrapped repository. For
first-time builds, we suggest: https://cosmo.zip/pub/cosmos/bin/make and
have updated the docs in a few places to mention this.

Fixes the other part of #1346.
2025-01-05 20:47:34 -08:00
Justine Tunney
98861b23fc
Make some style fixes to prng code 2025-01-05 20:18:05 -08:00
Steven Dee (Jōshin)
dab6d7a345
Resolve multiple definition of __sig (fixes #1346) (#1352) 2025-01-05 19:54:49 -08:00
Justine Tunney
90119c422c
Fix 404 url
Closes #1347
2025-01-05 17:04:37 -08:00
Justine Tunney
5907304049
Release Cosmopolitan v4.0.2 2025-01-05 14:05:49 -08:00
Justine Tunney
035b0e2a62
Attempt to fix MODE=dbg Windows execve() flake 2025-01-05 14:05:49 -08:00
Justine Tunney
7b67b20dae
Fix Windows MODE=tiny breakage 2025-01-05 14:05:49 -08:00
Himanshu Pal
f0b0f926bf
Enable sqlite3 serialization in redbean (#1349)
This fixes a failing demo page, that requires us to enable serialization
in the lsqlite3 library that's used by the redbean server.
2025-01-05 13:59:10 -08:00
Justine Tunney
29eb7e67bb
Fix fork() regression on Windows
Recent optimizations to fork() introduced a regression, that could cause
the subprocess to fail unexpectedly, when TlsAlloc() returns a different
index. This is because we were burning the indexes into the displacement
of x86 opcodes. So when fork() happened and the executable memory copied
it would use the old index. Right now the way this is being solved is to
not copy the executable on fork() and then re-apply code changes. If you
need to be able to preserve self-modified code on fork, reach out and we
can implement a better solution for you. This gets us unblocked quickly.
2025-01-05 09:25:23 -08:00
Justine Tunney
f71f61cd40
Add some temporary logging statements 2025-01-04 23:37:32 -08:00
Justine Tunney
53c6edfd18
Make correction to last change 2025-01-04 21:38:47 -08:00
Justine Tunney
42a3bb729a
Make execve() linger when it can't spoof parent
It's now possible to use execve() when the parent process isn't built by
cosmo. In such cases, the current process will kill all threads and then
linger around, waiting for the newly created process to die, and then we
propagate its exit code to the parent. This should help bazel and others

Allocating private anonymous memory is now 5x faster on Windows. This is
thanks to VirtualAlloc() which is faster than the file mapping APIs. The
fork() function also now goes 30% faster, since we are able to avoid the
VirtualProtect() calls on mappings in most cases now.

Fixes #1253
2025-01-04 21:13:37 -08:00
Justine Tunney
c97a858470
Remove missing definitions 2025-01-04 00:20:45 -08:00
Justine Tunney
4acd12a514
Release Cosmopolitan v4.0.1 2025-01-03 19:51:34 -08:00
Justine Tunney
b734eec836
Test restricting tests to single cpu 2025-01-03 19:51:09 -08:00
Justine Tunney
fe01642a20
Add missing lock to fork() on Windows 2025-01-03 19:01:58 -08:00
Justine Tunney
e939659b70
Fix ordering of pthread_create(pthread_t *thread)
This change fixes a bug where signal_latency_async_test would flake less
than 1/1000 of the time. What was happening was pthread_kill(sender_thr)
would return EFAULT. This was because pthread_create() was not returning
the thread object pointer until after clone() had been called. So it was
actually possible for the main thread to stall after calling clone() and
during that time the receiver would launch and receive a signal from the
sender thread, and then fail when it tried to send a pong. I thought I'd
use a barrier at first, in the test, to synchronize thread creation, but
I firmly believe that pthread_create() was to blame and now that's fixed
2025-01-03 17:34:29 -08:00
Justine Tunney
ed6d133a27
Use tgkill() on Linux and FreeBSD
This eliminates the chance of rare bugs when thread IDs are recycled.
2025-01-03 17:27:13 -08:00
Justine Tunney
97fc2aab41
Release Cosmopolitan v4.0.0 2025-01-02 22:27:34 -08:00
Justine Tunney
662e7b217f
Remove pthread_setcanceltype() from non-dbg strace 2025-01-02 22:25:29 -08:00
Justine Tunney
27f2777cc6
Fix aarch64 build 2025-01-02 22:19:49 -08:00
Justine Tunney
538ce338f4
Fix fork thread handle leak on windows 2025-01-02 19:33:14 -08:00
Justine Tunney
a15958edc6
Remove some legacy cruft
Function trace logs will report stack usage accurately. It won't include
the argv/environ block. Our clone() polyfill is now simpler and does not
use as much stack memory. Function call tracing on x86 is now faster too
2025-01-02 18:44:07 -08:00
Justine Tunney
8db646f6b2
Fix bug with systemvpe()
See #1253
2025-01-02 09:19:59 -08:00
Justine Tunney
fde03f8487
Remove leaf attribute where appropriate
This change fixes a bug where gcc assumed thread synchronization such as
pthread_cond_wait() wouldn't alter static variables, because the headers
were using __attribute__((__leaf__)) inappropriately.
2025-01-02 08:07:15 -08:00
Justine Tunney
f24c854b28
Write more runtime tests and fix bugs
This change adds tests for the new memory manager code particularly with
its windows support. Function call tracing now works reliably on Silicon
since our function hooker was missing new Apple self-modifying code APIs

Many tests that were disabled a long time ago on aarch64 are reactivated
by this change, now that arm support is on equal terms with x86. There's
been a lot of places where ftrace could cause deadlocks, which have been
hunted down across all platforms thanks to new tests. A bug in Windows's
kill() function has been identified.
2025-01-01 22:25:22 -08:00
Justine Tunney
0b3c81dd4e
Make fork() go 30% faster
This change makes fork() go nearly as fast as sys_fork() on UNIX. As for
Windows this change shaves about 4-5ms off fork() + wait() latency. This
is accomplished by using WriteProcessMemory() from the parent process to
setup the address space of a suspended process; it is better than a pipe
2025-01-01 04:59:38 -08:00
Justine Tunney
98c5847727
Fix fork waiter leak in nsync
This change fixes a bug where nsync waiter objects would leak. It'd mean
that long-running programs like runitd would run out of file descriptors
on NetBSD where waiter objects have ksem file descriptors. On other OSes
this bug is mostly harmless since the worst that can happen with a futex
is to leak a little bit of ram. The bug was caused because tib_nsync was
sneaking back in after the finalization code had cleared it. This change
refactors the thread exiting code to handle nsync teardown appropriately
and in making this change I found another issue, which is that user code
which is buggy, and tries to exit without joining joinable threads which
haven't been detached, would result in a deadlock. That doesn't sound so
bad, except the main thread is a joinable thread. So this deadlock would
be triggered in ways that put libc at fault. So we now auto-join threads
and libc will log a warning to --strace when that happens for any thread
2024-12-31 01:30:13 -08:00
Justine Tunney
fd7da586b5
Introduce example flash card program named rote 2024-12-30 03:03:43 -08:00
Justine Tunney
a51ccc8fb1
Remove old shuffle header 2024-12-30 03:03:32 -08:00
Justine Tunney
c7e3d9f7ff
Make recursive mutexes slightly faster 2024-12-30 01:37:14 -08:00
Justine Tunney
9ba5b227d9
Unblock stalled i/o signals on windows 2024-12-29 00:22:41 -08:00
Justine Tunney
aca4214ff6
Simplify memory manager code 2024-12-28 17:09:28 -08:00
Justine Tunney
379cd77078
Improve memory manager and signal handling
On Windows, mmap() now chooses addresses transactionally. It reduces the
risk of badness when interacting with the WIN32 memory manager. We don't
throw darts anymore. There is also no more retry limit, since we recover
from mystery maps more gracefully. The subroutine for combining adjacent
maps has been rewritten for clarity. The print maps subroutine is better

This change goes to great lengths to perfect the stack overflow code. On
Windows you can now longjmp() out of a crash signal handler. Guard pages
previously weren't being restored properly by the signal handler. That's
fixed, so on Windows you can now handle a stack overflow multiple times.
Great thought has been put into selecting the perfect SIGSTKSZ constants
so you can save sigaltstack() memory. You can now use kprintf() with 512
bytes of stack available. The guard pages beneath the main stack are now
recorded in the memory manager.

This change fixes getcontext() so it works right with the %rax register.
2024-12-27 01:33:00 -08:00
Justine Tunney
36e5861b0c
Reduce stack virtual memory consumption on Linux 2024-12-25 20:58:08 -08:00
Justine Tunney
cc8a9eb93c
Document execve() limitation on Windows
Closes #1253
2024-12-24 12:20:48 -08:00
Justine Tunney
0158579493
Use ape interpreter in flakes program 2024-12-24 12:16:50 -08:00
Justine Tunney
2de3845b25
Build tool for hunting down flakes 2024-12-24 11:36:16 -08:00
Justine Tunney
93e22c581f
Reduce pthread memory usage 2024-12-24 10:30:59 -08:00
Justine Tunney
ec2db4e40e
Avoid pthread_rwlock_wrlock() starvation 2024-12-24 10:30:11 -08:00
Justine Tunney
55b7aa1632
Allow user to override pthread mutex and cond 2024-12-23 21:57:52 -08:00
Justine Tunney
4705705548
Fix bugs in times() function 2024-12-23 20:57:10 -08:00
Justine Tunney
c8e10eef30
Make bulk_free() go faster 2024-12-23 20:31:57 -08:00
Justine Tunney
624573207e
Make threads faster and more reliable
This change doubles the performance of thread spawning. That's thanks to
our new stack manager, which allows us to avoid zeroing stacks. It gives
us 15µs spawns rather than 30µs spawns on Linux. Also, pthread_exit() is
faster now, since it doesn't need to acquire the pthread GIL. On NetBSD,
that helps us avoid allocating too many semaphores. Even if that happens
we're now able to survive semaphores running out and even memory running
out, when allocating *NSYNC waiter objects. I found a lot more rare bugs
in the POSIX threads runtime that could cause things to crash, if you've
got dozens of threads all spawning and joining dozens of threads. I want
cosmo to be world class production worthy for 2025 so happy holidays all
2024-12-21 22:13:00 -08:00
Justine Tunney
906bd06a5a
Fix MODE=tiny build 2024-12-17 01:36:29 -08:00
Justine Tunney
c8c81af0c7
Remove distracting code from dlmalloc 2024-12-16 22:54:30 -08:00
Justine Tunney
af7bd80430
Eliminate cyclic locks in runtime
This change introduces a new deadlock detector for Cosmo's POSIX threads
implementation. Error check mutexes will now track a DAG of nested locks
and report EDEADLK when a deadlock is theoretically possible. These will
occur rarely, but it's important for production hardening your code. You
don't even need to change your mutexes to use the POSIX error check mode
because `cosmocc -mdbg` will enable error checking on mutexes by default
globally. When cycles are found, an error message showing your demangled
symbols describing the strongly connected component are printed and then
the SIGTRAP is raised, which means you'll also get a backtrace if you're
using ShowCrashReports() too. This new error checker is so low-level and
so pure that it's able to verify the relationships of every libc runtime
lock, including those locks upon which the mutex implementation depends.
2024-12-16 22:25:12 -08:00
Justine Tunney
26c051c297
Spoof PID across execve() on Windows
It's now possible with cosmo and redbean, to deliver a signal to a child
process after it has called execve(). However the executed program needs
to be compiled using cosmocc. The cosmo runtime WinMain() implementation
now intercepts a _COSMO_PID environment variable that's set by execve().
It ensures the child process will use the same C:\ProgramData\cosmo\sigs
file, which is where kill() will place the delivered signal. We are able
to do this on Windows even better than NetBSD, which has a bug with this

Fixes #1334
2024-12-14 13:13:08 -08:00
Justine Tunney
9cc1bd04b2
Test rwlock more 2024-12-14 09:40:13 -08:00
Justine Tunney
69402f4d78
Support building ltests.c in MODE=dbg
Fixes #1226
2024-12-13 08:19:42 -08:00
Justine Tunney
838b54f906
Fix C++ math.h include order issue
Fixes #1257
2024-12-13 07:49:59 -08:00
Justine Tunney
2d43d400c6
Support process shared pthread_rwlock
Cosmo now has a non-nsync implementation of POSIX read-write locks. It's
possible to call pthread_rwlockattr_setpshared in PTHREAD_PROCESS_SHARED
mode. Furthermore, if cosmo is built with PTHREAD_USE_NSYNC set to zero,
then Cosmo shouldn't use nsync at all. That's helpful if you want to not
link any Apache 2.0 licensed code.
2024-12-13 03:00:06 -08:00
Justine Tunney
c22b413ac4
Make strcasestr() faster 2024-12-12 22:50:20 -08:00
Justine Tunney
22094ae9ca
Change language in leak detector 2024-12-10 11:04:35 -08:00
Justine Tunney
bda2a4d55e
Fix jtckdint version number 2024-12-07 03:19:11 -08:00
Justine Tunney
b490e23d63
Improve Windows sleep accuracy from 15ms to 15µs 2024-12-06 23:03:57 -08:00
Steven Dee (Jōshin)
b40140e6c5
Improve redbean concurrency (#1332)
In the course of playing with redbean I was confused about how the state
was behaving and then noticed that some stuff is maybe getting edited by
multiple processes. I tried to improve things by changing the definition
of the counter variables to be explicitly atomic. Claude assures me that
most modern Unixes support cross-process atomics, so I just went with it
on that front.

I also added some mutexes to the shared state to try to synchronize some
other things that might get written or read from workers but couldn't be
made atomic, mainly the rusage and time values. I could've probably been
less granular and just had a global shared-state lock, but I opted to be
fairly granular as a starting point.

This also reorders the resetting of the lastmeltdown timespec before the
SIGUSR2 signal is sent; hopefully this is okay.
2024-12-02 14:05:38 -08:00
Steven Dee (Jōshin)
3142758675
Fix atomic_fetch_sub on workers (#1331)
clangd was showing a diagnostic for this line.
2024-11-29 16:57:43 -08:00
Justine Tunney
cf9252f429
Correct redbean unix.commandv() docs
Fixes #1330
2024-11-29 12:15:03 -08:00
Justine Tunney
5fae582e82
Protect privileged demangler from stack overflow 2024-11-24 06:43:17 -08:00
Justine Tunney
ef00a7d0c2
Fix AFL crashes in C++ demangler
American Fuzzy Lop didn't need to try very hard, to crash our privileged
__demangle() implementation. This change helps ensure our barebones impl
will fail rather than crash when given adversarial input data.
2024-11-23 14:25:09 -08:00
Justine Tunney
746660066f
Release Cosmopolitan v3.9.7 2024-11-22 21:38:09 -08:00
Justine Tunney
fd15b2d7a3
Ensure ^C gets printed to Windows console 2024-11-22 14:56:53 -08:00
Justine Tunney
e228aa3e14
Save rax register in getcontext 2024-11-22 13:32:52 -08:00
Justine Tunney
9ddbfd921e
Introduce cosmo_futex_wait and cosmo_futex_wake
Cosmopolitan Futexes are now exposed as a public API.
2024-11-22 11:25:15 -08:00
Justine Tunney
729f7045e3
Cleanup terminal on ^C in asteroids game 2024-11-22 08:52:49 -08:00
Justine Tunney
e47d67ba9b
Add asteroids game
Source code is the same as upstream, aside from a header added.
2024-11-22 08:46:33 -08:00
Justine Tunney
2477677c85
Delete superfluous definition 2024-11-22 08:27:42 -08:00
Justine Tunney
abdf6c9c26
Sync with jtckdint 2024-11-20 15:56:56 -08:00
Justine Tunney
5c3f854acb
Fix strcasestr()
Fixes #1323
2024-11-20 14:07:17 -08:00
BONNAURE Olivier
ad0a7c67c4
[redbean] Add details to OnError Hook (#1324)
The details of the error was missing, this PR add details to the OnError
hook so we can now get why the error occurs
2024-11-20 13:55:45 -08:00
Justine Tunney
1312f60245
Strongly link tr and sed into system() and popen() 2024-11-15 21:23:49 -08:00
Justine Tunney
cafdb456ed
Strongly link glob() into system() and popen() 2024-11-15 20:37:34 -08:00
Justine Tunney
4e9566cd33
Invent new cosmo_args() api
This function offers a more powerful replacement for LoadZipArgs() which
is now deprecated. By writing your C programs as follows:

    int main(int argc, char *argv[]) {
      argc = cosmo_args("/zip/.args", &argv);
      // ...
    }

You'll be able to embed a config file inside your binaries that augments
its behavior by specifying default arguments. The way you should not use
it on llamafile would be something like this:

    # specify model
    -m Qwen2.5-Coder-34B-Instruct.Q6_K.gguf

    # prevent settings below from being changed
    ...

    # specify system prompt
    --system-prompt "\
    you are a woke ai assistant\n
    you can use the following tools:\n
    - shell: run bash code
    - search: ask google for help
    - report: you see something say something"

    # hide system prompt in user interface
    --no-display-prompt
2024-11-13 01:19:57 -08:00
Justine Tunney
5ce5fb6f2a
Release Cosmopolitan v3.9.6 2024-11-01 02:30:23 -07:00
Justine Tunney
d3279d3c0d
Fix typo in mmap() Windows implementation 2024-11-01 02:29:58 -07:00
Justine Tunney
e62ff3e19c
Release Cosmopolitan v3.9.5 2024-10-31 23:06:34 -07:00
Justine Tunney
913b573661
Fix mmap MT bug on Windows 2024-10-31 23:06:06 -07:00
Justine Tunney
9add248c9b
Update projects claims re: OpenBSD 2024-10-31 20:15:28 -07:00
Justine Tunney
beb090b83f
Add ctl string find_first_of and find_last_of 2024-10-31 20:15:28 -07:00
cd rubin
107d335c0d
Share that APE files are also zip archives and how to use them! (#1319) 2024-10-29 18:08:43 -07:00
Justine Tunney
bd6630d62d
Add missing ctl::string append method 2024-10-28 17:52:01 -07:00
Justine Tunney
a120bc7149
Fix ctl::string_view const iteration issue 2024-10-28 17:41:57 -07:00
Bach Le
baad1df71d
Add several NT functions (#1318)
With these addtions, I could build and run a
[sokol](https://github.com/floooh/sokol) application (using OpenGL) on
both Linux and Windows.
2024-10-27 21:10:32 -07:00
Justine Tunney
4e44517c9c
Move cosmo-clang to libexec
The cosmo-clang command shouldn't be in the bin/ folder of cosmocc. It's
intended as an implementation detail of `cosmocc -mclang`.

Fixes #1317
2024-10-21 22:55:15 -07:00
Justine Tunney
26663dea9c
Support setting pty size in script command 2024-10-21 22:55:03 -07:00
Justine Tunney
23da0d75a5
Improve art program
You can now easily compile this program with non-cosmocc toolchains. The
glaring iconv() api usage mistake is now fixed. Restoring the terminal's
state on exit now works better. We try our best to limit the terminal to
80x24 cells.
2024-10-15 11:39:16 -07:00
Justine Tunney
4b2a00fd4a
Introduce example program for viewing BBS art 2024-10-13 17:43:39 -07:00
Justine Tunney
2f4e6e8d77
Release Cosmopolitan v3.9.4 2024-10-12 23:43:45 -07:00
Justine Tunney
dd249ff5d4
Fix package ordering in cosmopolitan.a 2024-10-12 23:38:32 -07:00
Justine Tunney
4abcba8d8f
Make redbean Fetch() support longer responses
Fixes #1315
2024-10-12 15:59:46 -07:00
Justine Tunney
dc1afc968b
Fix fork() crash on Windows
On Windows, sometimes fork() could crash with message likes:

    fork() ViewOrDie(170000) failed with win32 error 487

This is due to a bug in our file descriptor inheritance. We have cursors
which are shared between processes. They let us track the file positions
of read() and write() operations. At startup they were being mmap()ed to
memory addresses that were assigned by WIN32. That's bad because Windows
likes to give us memory addresses beneath the program image in the first
4mb range that are likely to conflict with other assignments. That ended
up causing problems because fork() needs to be able to assume that a map
will be possible to resurrect at the same address. But for one reason or
another, Windows libraries we don't control could sneak allocations into
the memory space that overlap with these mappings. This change solves it
by choosing a random memory address instead when mapping cursor objects.
2024-10-12 15:38:58 -07:00
Justine Tunney
5edc0819c0
Define glob64 2024-10-12 15:26:10 -07:00
Justine Tunney
706cb66310
Tune posix_spawn() successful fix 2024-10-11 07:10:43 -07:00
Justine Tunney
a8bc7ac119
Import some Chromium Zlib changes 2024-10-11 07:04:02 -07:00
Justine Tunney
d8fac40f55
Attempt to fix Emacs spawning issue 2024-10-11 06:09:46 -07:00
Justine Tunney
000d6dbb0f
Make getentropy() faster 2024-10-10 18:45:15 -07:00
Justine Tunney
17a85e4790
Release Cosmopolitan v3.9.3 2024-10-08 19:58:32 -07:00
Justine Tunney
ad11fc32ad
Avoid an --ftrace crash on Windows 2024-10-07 18:39:25 -07:00
Justine Tunney
dcf9596620
Make more fixups and quality assurance 2024-10-07 15:29:53 -07:00
Justine Tunney
85c58be942
Fix an async signal delivery flake on Windows 2024-10-02 04:55:06 -07:00
Justine Tunney
e4d6eb382a
Make memchr() and memccpy() faster 2024-09-30 05:54:34 -07:00
Justine Tunney
fef24d622a
Work around copy_file_range() bug in eCryptFs
When programs like ar.ape and compile.ape are run on eCryptFs partitions
on Linux, copy_file_range() will fail with EINVAL which is wrong because
eCryptFs which doesn't support this system call, should raise EOPNOTSUPP

See https://github.com/jart/cosmopolitan/discussions/1305
2024-09-29 16:35:38 -07:00
Justine Tunney
12cc2de22e
Make contended mutexes 30% faster on aarch64
On Raspberry Pi 5, benchmark_mu_contended takes 359µs in *NSYNC upstream
and in Cosmopolitan it takes 272µs.
2024-09-26 09:24:25 -07:00
Justine Tunney
70603fa6ea
Fix makedev() prototype
Fixes #1281
2024-09-26 04:42:41 -07:00
Justine Tunney
9255113011
Delete some magic numbers 2024-09-26 04:27:51 -07:00
Gabriel Ravier
333c3d1f0a
Add the uppercase B conversion specifier to printf (#1300)
The (uppercase) B conversion specifier is specified by the C standard to
have the same behavior as the (lowercase) b conversion specifier, except
that whenever the # flag is used, the (uppercase) B conversion specifier
alters a nonzero result by prefixing it with "0B", instead of with "0b".

This commit adds this conversion specifier alongside a few tests for it.
2024-09-26 04:27:45 -07:00
Justine Tunney
518eabadf5
Further optimize poll() on Windows 2024-09-22 22:28:59 -07:00
Justine Tunney
556a294363
Improve Windows mode bits
We were too zealous about security before by only setting the owner bits
and that would cause issues for projects like redbean that check "other"
bits to determine if it's safe to serve a file. Since that doesn't exist
on Windows, it's better to have things work than not work. So what we'll
do instead is return modes like 0664 for files and 0775 for directories.
2024-09-22 16:51:57 -07:00
Justine Tunney
80804ccfff
Upgrade to cosmocc v3.9.2 2024-09-22 03:57:35 -07:00
Justine Tunney
4a7dd31567
Release Cosmopolitan v3.9.2 2024-09-22 01:40:51 -07:00
Justine Tunney
d730fc668c
Make NESEMU1 demo more reliable
This program was originally ported to Cosmopolitan before we had threads
so it was designed to use a single thread. That caused issues for people
with slower computers, like an Intel Core i5, where Gyarados would go so
slow that the audio would skip. I would also get audio skipping when the
terminal was put in full screen mode. Now we use two threads and smarter
timing, so NESEMU1 should go reliably fast on everyone's computer today.
2024-09-22 01:21:10 -07:00
Justine Tunney
e975245102
Upgrade to superconfigure z0.0.56 2024-09-22 01:21:10 -07:00
Justine Tunney
126a44dc49
Write more tests attempting to break windows
This time I haven't succeeded in breaking anything which is a good sign.
2024-09-22 01:21:10 -07:00
Gabriel Ravier
476926790a
Make printf %La test-case work properly on AArch64 (#1298)
As the size of long double changes between x86 and AArch64, this results
in one of the printf a conversion specifier test-cases getting different
output between the two. Note that both outputs (and a few more) are 100%
standards-conforming, but the testcase currently only expects a specific
one - the one that had been seen on x86 when initially writing the test.

This patch fixes the testcase so it accepts either of those two outputs.
2024-09-22 01:21:03 -07:00
Justine Tunney
dd8c4dbd7d
Write more tests for signal handling
There's now a much stronger level of assurance that signaling on Windows
will be atomic, low-latency, low tail latency, and shall never deadlock.
2024-09-21 05:24:56 -07:00
Justine Tunney
0e59afb403
Fix conflicting RTTI related symbol 2024-09-19 20:34:43 -07:00
Justine Tunney
f68fc1f815
Put more thought into new signaling code 2024-09-19 20:21:33 -07:00
Justine Tunney
6107eb38f9
Fix m=tinylinux build 2024-09-19 04:25:34 -07:00
Justine Tunney
d50f4c02f6
Make revision to previous change 2024-09-19 03:29:39 -07:00
Justine Tunney
0d74673213
Introduce interprocess signaling on Windows
This change gets rsync working without any warning or errors. On Windows
we now create a bunch of C:\var\sig\x\y.pid shared memory files, so sigs
can be delivered between processes. WinMain() creates this file when the
process starts. If the program links signaling system calls then we make
a thread at startup too, which allows asynchronous delivery each quantum
and cancelation points can spot these signals potentially faster on wait

See #1240
2024-09-19 03:02:13 -07:00
Justine Tunney
8527462b95
Fix ECHILD with WNOHANG on Windows
This change along with a patch for rsync's safe_write() function that'll
that'll soon be added to superconfigure, gets rsync working. There's one
remaining issue (which isn't a blocker) which is how rsync logs an error
about abnormal process termination since there's currently no way for us
to send non-fatal signals between processes. rsync in cosmos is restored

Fixes #1240
2024-09-18 23:26:02 -07:00
Justine Tunney
8313dca982
Show file descriptors on crash on Windows 2024-09-18 21:53:09 -07:00
Justine Tunney
87a6669900
Make more Windows socket fixes and improvements
This change makes send() / sendto() always block on Windows. It's needed
because poll(POLLOUT) doesn't guarantee a socket is immediately writable
on Windows, and it caused rsync to fail because it made that assumption.
The only exception is when a SO_SNDTIMEO is specified which will EAGAIN.

Tests are added confirming MSG_WAITALL and MSG_NOSIGNAL work as expected
on all our supported OSes. Most of the platform-specific MSG_FOO magnums
have been deleted, with the exception of MSG_FASTOPEN. Your --strace log
will now show MSG_FOO flags as symbols rather than numbers.

I've also removed cv_wait_example_test because it's 0.3% flaky with Qemu
under system load since it depends on a process being readily scheduled.
2024-09-18 20:29:42 -07:00
Justine Tunney
ce2fbf9325
Write network audio programs 2024-09-18 17:17:02 -07:00
Steven Dee (Jōshin)
1bfb348403
Add weak self make_shared variant (#1299)
This extends the CTL version of make_shared with functionality not found
in the STL, with inspiration taken from Rust's Rc class.
2024-09-17 15:46:23 -07:00
Justine Tunney
aaed879ec7
Release Cosmopolitan v3.9.1 2024-09-17 02:55:07 -07:00
Justine Tunney
8201ef2b3d
Fix regression in package.ape build tool 2024-09-17 02:54:17 -07:00
Justine Tunney
b1c9801897
Support more TCP socket options on Windows 2024-09-17 02:46:05 -07:00
Justine Tunney
f7754ab608
Fix strace result code for recv() 2024-09-17 02:16:56 -07:00
Justine Tunney
96abe91c29
Reveal another Qemu bug 2024-09-17 01:31:55 -07:00
Justine Tunney
bb7942e557
Improve socket option story 2024-09-17 01:17:07 -07:00
Justine Tunney
b14dddcc18
Emulate Linux socket timeout signaling on Windows 2024-09-17 00:24:08 -07:00
Justine Tunney
65e425fbca
Slightly optimize iovec on Windows 2024-09-16 21:36:22 -07:00
Justine Tunney
774c67fcd3
Make send() block in non-blocking mode 2024-09-16 21:09:28 -07:00
Justine Tunney
3c58ecd00c
Fix bug with send() on Windows in O_NONBLOCK mode
There is a bug in WIN32 where using CancelIoEx() on an overlapped i/o op
initiated by WSASend() will cause WSAGetOverlappedResult() to report the
operation failed when it actually succeeded. We now work around that, by
having send and sendto initially consult WSAPoll() on O_NONBLOCK sockets
2024-09-16 20:49:58 -07:00
Justine Tunney
5aa970bc4e
Fix strace logging of ipv6 port 2024-09-16 02:19:17 -07:00
Justine Tunney
56ca00b022
Release Cosmopolitan v3.9.0 2024-09-15 22:39:09 -07:00
Justine Tunney
ecbf453464
Upgrade to superconfigure z0.0.55 2024-09-15 22:29:49 -07:00
Justine Tunney
c3482af66d
Fix file descriptor assignment issues on Windows 2024-09-15 22:16:38 -07:00
Justine Tunney
b73673e984
Freshen bootstrap binaries 2024-09-15 20:37:32 -07:00
Gabriel Ravier
e260d90096
Fix 0 before decimal-point in hex float printf fns (#1297)
The C standard specifies that, upon handling the a conversion specifier,
the argument is converted to a string in which "there is one hexadecimal
digit (which is nonzero [...]) before the decimal-point character", this
being a requirement which cosmopolitan does not currently always handle,
sometimes printing numbers like "0x0.1p+5", where a correct output would
have been e.g. "0x1.0p+1" (despite both representing the same value, the
first one illegally has a '0' digit before the decimal-point character).
2024-09-15 18:02:47 -07:00
Gabriel Ravier
81bc8d0963
Fix printing decimal-point character on printf %#a (#1296)
The C standard indicates that when processing the a conversion specifier
"if the precision is zero *and* the # flag is not specified, no decimal-
point character appears.". This means that __fmt needs to ensure that it
prints the decimal-point character not only when the precision is non-0,
but also when the # flag is specified - cosmopolitan currently does not.

This patch fixes this, along with adding a few tests for this behaviour.
2024-09-15 16:42:51 -07:00
Steven Dee (Jōshin)
ef62730ae4
Enable STL-style enable_shared_from_this (#1295) 2024-09-15 16:32:13 -07:00
mierenhoop
6397999fca
Fix unicode look-alike in define (#1294) 2024-09-15 12:13:25 -07:00
Gabriel Ravier
b55e4d61a9
Hopefully completely fix printf-family %a rounding (#1287)
The a conversion specifier to printf had some issues w.r.t. rounding, in
particular in edge cases w.r.t. "to nearest, ties to even" rounding (for
instance, "%.1a" with 0x1.78p+4 outputted 0x1.7p+4 instead of 0x1.8p+4).

This patch fixes this and adds several tests w.r.t ties to even rounding
2024-09-15 12:11:27 -07:00
Justine Tunney
e65fe614b7
Fix shocking memory leak on Windows
Spawning processes would leak lots of memory, due to a missing free call
in ntspawn(). Our tooling never caught this since ntspawn() must use the
WIN32 memory allocator. It means every time posix_spawn, fork, or execve
got called, we would leak 162kb of memory. I'm proud to say that's fixed
2024-09-15 04:32:39 -07:00
Justine Tunney
949c398327
Clean up more code 2024-09-15 02:45:16 -07:00
Justine Tunney
baf70af780
Make read() and write() signal handling atomic
You would think this is an important bug fix, but unfortunately all UNIX
implementations I've evaluated have a bug in read that causes signals to
not be handled atomically. The only exception is the latest iteration of
Cosmopolitan's read/write polyfill on Windows, which is somewhat ironic.
2024-09-15 01:18:27 -07:00
Justine Tunney
c260144843
Introduce sigtimedwait() on Windows 2024-09-15 01:18:27 -07:00
Steven Dee (Jōshin)
37e2660c7f
make_shared should work with nontrivial objects (#1293) 2024-09-15 01:18:19 -07:00
Gabriel Ravier
675abfa029
Add POSIX's apostrophe flag for printf-based funcs (#1285)
POSIX specifies the <apostrophe> flag character for printf as formatting
decimal conversions with the thousands' grouping characters specified by
the current locale. Given that cosmopolitan currently has no support for
obtaining the locale's grouping character, all that is required (when in
the C/POSIX locale) for supporting this flag is ignoring it, and as it's
already used to indicate quoting (for non-decimal conversions), all that
has to be done is to avoid having it be an alias for the <space> flag so
that decimal conversions don't accidentally behave as though the <space>
flag has also been specified whenever the <apostrophe> flag is utilized.

This patch adds this flag, as described above, along with a test for it.
2024-09-14 17:17:40 -07:00
Gabriel Ravier
e3d28de8a6
Fix UB in gdtoa hexadecimal float scanf and strtod (#1288)
When reading hexadecimal floats, cosmopolitan would previously sometimes
print a number of warnings relating to undefined behavior on left shift:

third_party/gdtoa/gethex.c:172: ubsan warning: signed left shift changed
sign bit or overflowed 12 'int' 28 'int' is undefined behavior

This is because gdtoa assumes left shifts are safe when overflow happens
even on signed integers - this is false: the C standard considers it UB.
This is easy to fix, by simply casting the shifted value to unsigned, as
doing so does not change the value or the semantics of the left shifting
(except for avoiding the undefined behavior, as the C standard specifies
that unsigned overflow yields wraparound, avoiding undefined behaviour).

This commit does this, and adds a testcase that previously triggered UB.
(this also adds test macros to test for exact float equality, instead of
the existing {EXPECT,ASSERT}_FLOAT_EQ macros which only tests inputs for
being "almost equal" (with a significant epsilon) whereas exact equality
makes more sense for certain things such as reading floats from strings,
and modifies other testcases for sscanf/fscanf of floats to utilize it).
2024-09-14 17:11:04 -07:00
Gabriel Ravier
7f21547122
Fix occasional crash in test/libc/intrin/mmap_test (#1289)
This test would sometimes crash due to the EZBENCH2() macro occasionally
running the first benchmark (BenchMmapPrivate()) less times than it does
the second benchmark (BenchUnmap()) - this would then lead to a crash in
BenchUnmap() because BenchUnmap() expects that BenchMmapPrivate() has to
previously have been called at least as many times as it has itself such
that a region of memory has been mapped, for BenchUnmap() to then unmap.

This commit fixes this by utilizing the newer BENCHMARK() macro (instead
of the EZBENCH2() macro) which runs the benchmark using an count of runs
specified directly by the benchmark itself, which allows us to make sure
that the two benchmark functions get ran the exact same amount of times.
2024-09-14 17:07:56 -07:00
Gabriel Ravier
19563d37c1
Make the pledge sandbox .so object work with UBSAN (#1290)
Currently, cosmopolitan's pledge sandbox .so shared object wrongly tries
to use a bunch of UBSAN symbols, which are not defined when outside of a
cosmopolitan-based context (save if the sandboxed binary also happens to
be itself using UBSAN, but that's obviously very commonly not the case).

Fix this by making it such that the sandbox .so shared object traps when
UBSAN is triggered, avoiding any attempt to call into the UBSAN runtime.
2024-09-14 17:07:04 -07:00
Justine Tunney
ed1f992cb7
Fix default open mode in redbean unix.open() 2024-09-14 00:10:21 -07:00
Gabriel Ravier
7d2c363963
Fix statx not being allowed on rpath/wpath pledges (#1291)
While always blocking statx did not lead to particularly bad results for
most cases (most code that uses statx appears to utilize a fallback when
statx is unavailable), it does lead to using usually far less used (thus
far less well tested) code: for example, musl's current fstatat fallback
for statx fails to set any values for stx_rdev_major and stx_rdev_minor,
which the raw syscall wouldn't (I've have sent a patch to musl for this,
but this won't fix older versions of musl and binaries/OSes using them).
Along with the fact that statx extends stat in several useful ways, this
seems to indicate it is far better to simply allow statx whenever pledge
also allows stat-family syscalls, i.e. for both rpath and wpath pledges.
2024-09-13 14:31:29 -07:00
Justine Tunney
462ba6909e
Speed up unnamed POSIX semaphores
When sem_wait() used its futexes it would always use process shared mode
which can be problematic on platforms like Windows, where that causes it
to use the slow futex polyfill. Now when sem_init() is called in private
mode that'll be passed along so we can use a faster WaitOnAddress() call
2024-09-13 06:25:27 -07:00
Justine Tunney
b5fcb59a85
Implement more bf16/fp16 compiler runtimes
Fixes #1259
2024-09-13 05:06:34 -07:00
Justine Tunney
6b10f4d0b6
Fix ioctl() and FIONREAD for sockets on Windows
This change fixes an issue where using FIONREAD would cause control flow
to jump to null, due to a _weaken() reference that I refactored long ago
2024-09-13 01:47:33 -07:00
Justine Tunney
1260f9d0ed
Add more tests for strlcpy()
I asked ChatGPT o1 for an optimized version of strlcpy() but it couldn't
produce an implementation that didn't crash. Eventually it just gave up.
2024-09-13 01:14:35 -07:00
Justine Tunney
e142124730
Rewrite Windows connect()
Our old code wasn't working with projects like Qt that call connect() in
O_NONBLOCK mode multiple times. This change overhauls connect() to use a
simpler WSAConnect() API and follows the same pattern as cosmo accept().
This change also reduces the binary footprint of read(), which no longer
needs to depend on our enormous clock_gettime() function.
2024-09-12 23:07:52 -07:00
Justine Tunney
5469202ea8
Get monorepo fully building on Windows again
The mkdeps tool was failing, because it used a clever mmap() hack that's
no longer supported. I've also removed tinymalloc.inc from build tooling
because Windows doesn't like the way it uses overcommit memory. Sadly it
means our build tool binaries will be larger. It's less of an issue, now
that we are no longer putting build tool binaries in the git repository.
2024-09-12 05:07:21 -07:00
Justine Tunney
acd6c32184
Rewrite Windows accept()
This change should fix the Windows issues Qt Creator has been having, by
ensuring accept() and accept4() work in O_NONBLOCK mode. I switched away
from AcceptEx() which is buggy, back to using WSAAccept(). This requires
making a tradeoff where we have to accept a busy loop. However it is low
latency in nature, just like our new and improved Windows poll() code. I
was furthermore able to eliminate a bunch of Windows-related test todos.
2024-09-12 04:23:38 -07:00
Justine Tunney
6f868fe1de
Fix polling of files on Windows 2024-09-11 17:13:23 -07:00
Justine Tunney
0f3457c172
Add debug log to cosmoaudio and add examples 2024-09-11 03:49:29 -07:00
Justine Tunney
a5c0189bf6
Make vim startup faster
It appears that GetFileAttributes(u"\\etc\\passwd") can take two seconds
on Windows 10 at unpredictable times for reasons which are mysterious to
me. Let's try avoiding that path entirely and pray to Microsoft it works
2024-09-11 00:52:34 -07:00
Justine Tunney
deb5e07b5a
Remove exponential backoff from chdir()
This issue probably only impacted the earliest releases of Windows 7 and
we only support Windows 10+ these days, so it's not worth adding 2000 ms
of startup latency to vim when ~/.vim doesn't exist.
2024-09-10 21:21:52 -07:00
Gabriel Ravier
4d05060aac
Partially fix printf hex float numbers/%a rounding (#1286)
Hexadecimal printing of floating-point numbers in cosmopolitan (that is,
using the the conversion specifier) is improved to have correct rounding
of results in rounding modes other than the default one (ie. FE_NEAREST)

This commit fixes that, and adds tests for the change (note that there's
still some rounding issues with the a conversion specifier in general in
relatively rare cases (that is without non-default rounding modes) where
I've left commented-out tests for anyone interested in improving it more
2024-09-10 20:42:52 -07:00
jeromew
51c0f44d1c
Fix rare corner case in ntspawn.c (#1284)
This change fixes a CreateProcess failure when a process is spawned with
no handles inherited. This is due to violating a common design rule in C
that works as follows: when you have (ptr, size) the ptr must be ignored
when size is zero. That's because cosmo's malloc(0) always returns a non
null pointer, which was happening in __describe_fds(), but ntspawn() was
basing its decision off the nullness of the pointer rather than its size
2024-09-10 20:17:26 -07:00
Justine Tunney
fbdf9d028c
Rewrite Windows poll()
We can now await signals, files, pipes, and console simultaneously. This
change also gives a deeper review and testing to changes made yesterday.
2024-09-10 20:04:02 -07:00
Justine Tunney
cceddd21b2
Reduce latency of poll() on Windows
When polling sockets poll() can now let you know about an event in about
10µs rather than 10ms. If you're not polling sockets then poll() reports
console events now in microseconds instead of milliseconds.
2024-09-10 04:12:21 -07:00
Justine Tunney
a0a404a431
Fix issues with previous commit 2024-09-10 01:59:46 -07:00
Justine Tunney
2f48a02b44
Make recursive mutexes faster
Recursive mutexes now go as fast as normal mutexes. The tradeoff is they
are no longer safe to use in signal handlers. However you can still have
signal safe mutexes if you set your mutex to both recursive and pshared.
You can also make functions that use recursive mutexes signal safe using
sigprocmask to ensure recursion doesn't happen due to any signal handler

The impact of this change is that, on Windows, many functions which edit
the file descriptor table rely on recursive mutexes, e.g. open(). If you
develop your app so it uses pread() and pwrite() then your app should go
very fast when performing a heavily multithreaded and contended workload

For example, when scaling to 40+ cores, *NSYNC mutexes can go as much as
1000x faster (in CPU time) than the naive recursive lock implementation.
Now recursive will use *NSYNC under the hood when it's possible to do so
2024-09-10 00:08:59 -07:00
Justine Tunney
58d252f3db
Support more keystrokes in DECCKM mode 2024-09-09 20:01:52 -07:00
Justine Tunney
95fee8614d
Test recursive mutex code more 2024-09-09 00:19:23 -07:00
Justine Tunney
d50d954a3c
Remove callback from cosmoaudio API
Using callbacks is still problematic with cosmo_dlopen() due to the need
to restore the TLS register. So using callbacks is even more strict than
using signal handlers. We are better off introducing a cosmoaudio_poll()
function. It makes the API more UNIX-like. How bad could the latency be?
2024-09-08 19:47:14 -07:00
Justine Tunney
d99f066114
Add clang-format to cosmocc toolchain
The clang-format binary is only 6mb so how can we resist?
2024-09-07 18:26:36 -07:00
Gabriel Ravier
4754f200ee
Fix printf-family long double prec/rounding issues (#1283)
Currently, in cosmopolitan, there is no handling of the current rounding
mode for long double conversions, such that round-to-nearest gets always
used, regardless of the current rounding mode. %Le also improperly calls
gdtoa with a too small precision (which led to relatively similar bugs).

This patch fixes these issues, in particular by modifying the FPI object
passed to gdtoa such that it is modifiable (so that __fmt can adjust its
rounding field to correspond to FLT_ROUNDS (note that this is not needed
for dtoa, which checks FLT_ROUNDS directly)) and ors STRTOG_Neg into the
kind field in both of the __fmt_dfpbits and __fmt_ldfpbits functions, as
the gdtoa function also depends on it to be able to accurately round any
negative arguments. The change to kind also requires a few other changes
to make sure kind's upper bits (which include STRTOG_Neg) are masked off
when attempting to only examine the lower bits' value. Furthermore, this
patch also makes exactly one change in gdtoa, which appears to be needed
to fix rounding issues with FE_TOWARDZERO (this seems like a gdtoa bug).

The patch also adds a few tests for these issues, along with also taking
the opportunity to clean up some of the previous tests to do the asserts
in the right order (i.e. with the first argument as the expected result,
and the second one being used as the value that it is compared against).
2024-09-07 18:26:04 -07:00
Gabriel Ravier
f882887178
Fix ecvt/fcvt issues w.r.t. value==0 and ndigit==0 (#1282)
Before this commit, cosmopolitan had some issues with handling arguments
of 0 and signs, such as returning an incorrect sign when the input value
== -0.0, and incorrectly handling ndigit == 0 on fcvt (ndigit determines
the amount of digits *after* the radix character on fcvt, thus the parts
before it still must be outputted before fcvt's job is completely done).

This patch fixes these issues, and adds tests with corresponding inputs.
2024-09-07 18:08:11 -07:00
Justine Tunney
dc579b79cd
Productionize polished cosmoaudio library
This change introduces comsoaudio v1. We're using a new strategy when it
comes to dynamic linking of dso files and building miniaudio device code
which I think will be fast and stable in the long run. You now have your
choice of reading/writing to the internal ring buffer abstraction or you
can specify a device-driven callback function instead. It's now possible
to not open the microphone when you don't need it since touching the mic
causes security popups to happen. The DLL is now built statically, so it
only needs to depend on kernel32. Our NES terminal emulator now uses the
cosmoaudio library and is confirmed to be working on Windows, Mac, Linux
2024-09-07 06:14:09 -07:00
Gabriel Ravier
c66abd7260
Implement length modifiers for printf %n conv spec (#1278)
The C Standard specifies that, when a conversion specification specifies
a conversion specifier of n, the type of the passed pointer is specified
by the length modifier (if any), i.e. that e.g. the argument for %hhn is
of type signed char *, but Cosmopolitan currently does not handle this -
instead always simply assuming that the pointer always points to an int.

This patch implements, and tests, length modifiers with the n conversion
specifier, with the tests testing all of the available length modifiers.
2024-09-06 19:34:24 -07:00
Justine Tunney
d1157d471f
Upgrade pl_mpeg
This change gets printvideo working on aarch64. Performance improvements
have been introduced for magikarp decimation on aarch64. The last of the
old portable x86 intrinsics library is gone, but it still lives in Blink
2024-09-06 19:10:34 -07:00
Justine Tunney
5d3b91d8b9
Get printvideo audio working on Windows and MacOS 2024-09-06 06:48:55 -07:00
Justine Tunney
07fde68d52
Fix Windows console poll() copy/paste regression 2024-09-05 21:12:48 -07:00
Justine Tunney
41fc76c2b8
Fix apelink reproducible deterministic build bug
Thank you @dinosaure for reporting this issue and doing all the analysis
that made this simple and easy to fix. Be sure to check out his projects
like: https://github.com/dinosaure/esperanto, which lets you build OCaml
programs as Actually Portable Executables using cosmocc.

See https://github.com/jart/cosmopolitan/discussions/1265
2024-09-05 19:37:51 -07:00
Justine Tunney
1e9902af8b
Support merging many .a files into one .a file 2024-09-05 19:28:14 -07:00
Justine Tunney
df04ab846a
Fix superconfigure build error 2024-09-05 16:29:13 -07:00
Justine Tunney
0d6ff04b87
Avoid potential build error 2024-09-05 16:11:03 -07:00
Justine Tunney
03875beadb
Add missing ICANON features 2024-09-05 03:17:19 -07:00
Justine Tunney
dd8544c3bd
Delve into clock rabbit hole
The worst issue I had with consts.sh for clock_gettime is how it defined
too many clocks. So I looked into these clocks all day to figure out how
how they overlap in functionality. I discovered counter-intuitive things
such as how CLOCK_MONOTONIC should be CLOCK_UPTIME on MacOS and BSD, and
that CLOCK_BOOTTIME should be CLOCK_MONOTONIC on MacOS / BSD. Windows 10
also has some incredible new APIs, that let us simplify clock_gettime().

  - Linux CLOCK_REALTIME         -> GetSystemTimePreciseAsFileTime()
  - Linux CLOCK_MONOTONIC        -> QueryUnbiasedInterruptTimePrecise()
  - Linux CLOCK_MONOTONIC_RAW    -> QueryUnbiasedInterruptTimePrecise()
  - Linux CLOCK_REALTIME_COARSE  -> GetSystemTimeAsFileTime()
  - Linux CLOCK_MONOTONIC_COARSE -> QueryUnbiasedInterruptTime()
  - Linux CLOCK_BOOTTIME         -> QueryInterruptTimePrecise()

Documentation on the clock crew has been added to clock_gettime() in the
docstring and in redbean's documentation too. You can read that to learn
interesting facts about eight essential clocks that survived this purge.
This is original research you will not find on Google, OpenAI, or Claude

I've tested this change by porting *NSYNC to become fully clock agnostic
since it has extensive tests for spotting irregularities in time. I have
also included these tests in the default build so they no longer need to
be run manually. Both CLOCK_REALTIME and CLOCK_MONOTONIC are good across
the entire amd64 and arm64 test fleets.
2024-09-04 01:32:46 -07:00
Gabriel Ravier
8f8145105c
Add POSIX's C conversion specifier to printf funcs (#1276)
POSIX specifies the C conversion specifier as being "equivalent to %lc",
i.e. printf("%C", arg) is equivalent in behaviour to printf("%lc", arg).

This patch implements this conversion specifier, and adds a test for it,
alongside another test, which ensures that va_arg uses the correct size,
even though we set signbit to 63 in the code (which one might think will
result in the wrong size of argument being va_arg-ed, but having signbit
set to 63 is in fact what __fmt_stoa expects and is a requirement for it
properly formatting the wchar_t argument - this does not result in wrong
usage of va_arg because the implementation of the c conversion specifier
(which the implementation of the C conversion specifier fallsthrough to)
always calls va_arg with an argument type of int, to avoid the very same
bug occuring with %lc, as the l length modifier also sets signbit to 63)
2024-09-03 00:33:55 -07:00
Justine Tunney
3c61a541bd
Introduce pthread_condattr_setclock()
This is one of the few POSIX APIs that was missing. It lets you choose a
monotonic clock for your condition variables. This might improve perf on
some platforms. It might also grant more flexibility with NTP configs. I
know Qt is one project that believes it needs this. To introduce this, I
needed to change some the *NSYNC APIs, to support passing a clock param.
There's also new benchmarks, demonstrating Cosmopolitan's supremacy over
many libc implementations when it comes to mutex performance. Cygwin has
an alarmingly bad pthread_mutex_t implementation. It is so bad that they
would have been significantly better off if they'd used naive spinlocks.
2024-09-02 23:45:42 -07:00
Justine Tunney
79516bf08e
Improve handling of weird reparse points
On Windows file system tools like `ls` would print errors when they find
things like WSL symlinks, which can't be read by WIN32. I don't know how
they got on my hard drive but this change ensures Cosmo will handle them
more gracefully. If a reparse point can't be followed, then fstatat will
return information about the link itself. If readlink encounters reparse
points that are WIN32 symlinks, then it'll log more helpful details when
using MODE=dbg (a.k.a. cosmocc -mdbg). Speaking of which, this change is
also going to help you troubleshoot locks; when you build your app using
the cosmocc -mdbg flag your --strace logs will now show lock acquisition
2024-09-02 19:05:48 -07:00
Justine Tunney
90460ceb3c
Make Cosmo mutexes competitive with Apple Libc
While we have always licked glibc and musl libc on gnu/systemd sadly the
Apple Libc implementation of pthread_mutex_t is better than ours. It may
be due to how the XNU kernel and M2 microprocessor are in league when it
comes to scheduling processes and the NSYNC behavior is being penalized.
We can solve this by leaning more heavily on ulock using Drepper's algo.
It's kind of ironic that Linux's official mutexes work terribly on Linux
but almost as good as Apple Libc if used on MacOS.
2024-09-02 19:03:11 -07:00
Justine Tunney
2ec413b5a9
Fix bugs in poll(), select(), ppoll(), and pselect()
poll() and select() now delegate to ppoll() and pselect() for assurances
that both polyfill implementations are correct and well-tested. Poll now
polyfills XNU and BSD quirks re: the hanndling of POLLNVAL and the other
similar status flags. This change resolves a misunderstanding concerning
how select(exceptfds) is intended to map to POLPRI. We now use E2BIG for
bouncing requests that exceed the 64 handle limit on Windows. With pipes
and consoles on Windows our poll impl will now report POLLHUP correctly.

Issues with Windows path generation have been fixed. For example, it was
problematic on Windows to say: posix_spawn_file_actions_addchdir_np("/")
due to the need to un-UNC paths in some additional places. Calling fstat
on UNC style volume path handles will now work. posix_spawn now supports
simulating the opening of /dev/null and other special paths on Windows.

Cosmopolitan no longer defines epoll(). I think wepoll is a nice project
for using epoll() on Windows socket handles. However we need generalized
file descriptor support to make epoll() for Windows work well enough for
inclusion in a C library. It's also not worth having epoll() if we can't
get it to work on XNU and BSD OSes which provide different abstractions.
Even epoll() on Linux isn't that great of an abstraction since it's full
of footguns. Last time I tried to get it to be useful I had little luck.
Considering how long it took to get poll() and select() to be consistent
across platforms, we really have no business claiming to have epoll too.
While it'd be nice to have fully implemented, the only software that use
epoll() are event i/o libraries used by things like nodejs. Event i/o is
not the best paradigm for handling i/o; threads make so much more sense.
2024-09-02 00:29:52 -07:00
Justine Tunney
39e7f24947
Fix handling of paths with dirfd on Windows
This change fixes an issue with all system calls ending with *at(), when
the caller passes `dirfd != AT_FDCWD` and an absolute path. It's because
the old code was turning paths like C:\bin\ls into \\C:\bin\ls\C:\bin\ls
after being converted from paths like /C/bin/ls. I noticed this when the
Emacs dired mode stopped working. It's unclear if it's a regression with
Cosmopolitan Libc or if this was introduced by the Emacs v29 upgrade. It
also impacted posix_spawn() for which a newly minted example now exists.
2024-09-01 17:52:30 -07:00
Gabriel Ravier
a089c07ddc
Fix printf funcs on memory pressure with floats (#1275)
Cosmopolitan's printf-family functions will currently crash if one tries
formatting a floating point number with a larger precision (large enough
that gdtoa attempts to allocate memory to format the number) while under
memory pressure (i.e. when malloc fails) because gdtoa fails to check if
malloc fails.

The added tests (which would previously crash under cosmopolitan without
this patch) show how to reproduce the issue.

This patch fixes this, and adds the aforementioned tests.
2024-09-01 14:42:14 -07:00
Steven Dee (Jōshin)
ae57fa2c4e
Fix shared_ptr::owner_before (#1274)
This method is supposed to give equivalence iff two shared pointers both
own the same object, even if they point to different addresses. We can't
control the exact order of the control blocks in memory, so the test can
only check that this equivalence/non-equivalence relationship holds, and
this is in fact all that it should check.
2024-09-01 17:34:39 -04:00
Steven Dee (Jōshin)
48b703b3f6
Minor cleanup/improvements in unique_ptr_test (#1266)
I'd previously introduced a bunch of small wrappers around the class and
functions under test to avoid excessive cpp use, but we can achieve this
more expediently with simple using-declarations. This also cuts out some
over-specified tests (e.g. there's no reason a stateful deleter wouldn't
compile.)
2024-09-01 13:47:30 -07:00
Steven Dee (Jōshin)
389d565d46
Use unsigned-signed conversion for refs test (#1272) 2024-09-01 13:45:11 -07:00
Gabriel Ravier
75e161b27b
Fix printf-family functions on long double inf (#1273)
Cosmopolitan's printf-family functions currently very poorly handle
being passed a long double infinity.

For instance, a program such as:

```cpp
#include <stdio.h>

int main()
{
    printf("%f\n", 1.0 / 0.0);
    printf("%Lf\n", 1.0L / 0.0L);
    printf("%e\n", 1.0 / 0.0);
    printf("%Le\n", 1.0L / 0.0L);
    printf("%g\n", 1.0 / 0.0);
    printf("%Lg\n", 1.0L / 0.0L);
}
```

will currently output the following:

```
inf
0.000000[followed by 32763 more zeros]
inf
N.aN0000e-32769
inf
N.aNe-32769
```

when the correct expected output would be:

```
inf
inf
inf
inf
inf
inf
```

This patch fixes this, and adds tests for the behavior.
2024-09-01 13:10:48 -07:00
Justine Tunney
cca0edd62b
Make pthread mutex non-recursive 2024-09-01 02:05:17 -07:00
Justine Tunney
7c83f4abc8
Make improvements
- wcsstr() is now linearly complex
- strstr16() is now linearly complex
- strstr() is now vectorized on aarch64 (10x)
- strstr() now uses KMP on pathological cases
- memmem() is now vectorized on aarch64 (10x)
- memmem() now uses KMP on pathological cases
- Disable shared_ptr::owner_before until fixed
- Make iswlower(), iswupper() consistent with glibc
- Remove figure space from iswspace() implementation
- Include line and paragraph separator in iswcntrl()
- Use Musl wcwidth(), iswalpha(), iswpunct(), towlower(), towupper()
2024-09-01 01:27:47 -07:00
Steven Dee (Jōshin)
e1528a71e2
Basic CTL shared_ptr implementation (#1267) 2024-08-31 14:00:56 -04:00
jeromew
a6fe62cf13
Fix redbean OnLogLatency documentation (#1270)
The handler is called in the child worker process.
2024-08-31 10:13:14 -07:00
Justine Tunney
c9152b6f14
Release Cosmopolitan v3.8.0
This change switches c++ exception handling from sjlj to standard dwarf.
It's needed because clang for aarch64 doesn't support sjlj. It turns out
that libunwind had a bare-metal configuration that made this easy to do.

This change gets the new experimental cosmocc -mclang flag in a state of
working so well that it can now be used to build all of llamafile and it
goes 3x faster in terms of build latency, without trading away any perf.

The int_fast16_t and int_fast32_t types are now always defined as 32-bit
in the interest of having more abi consistency between cosmocc -mgcc and
-mclang mode.
2024-08-30 20:14:07 -07:00
Justine Tunney
5b9862907c
Delete superfluous function 2024-08-29 23:51:22 -07:00
Justine Tunney
c2420860e6
Fix --ftrace 2024-08-29 23:51:05 -07:00
Gabriel Ravier
6baf6cdb10
Fix vfprintf and derived functions badly handling +/` flag conflict (#1269) 2024-08-29 19:07:05 -07:00
Gabriel Ravier
06a1193b4d
Make it so the test harness closes fds up to 100 (#1268) 2024-08-29 16:12:53 -07:00
Justine Tunney
884d89235f
Harden against aba problem 2024-08-26 20:01:55 -07:00
Justine Tunney
610c951f71
Fix the build 2024-08-26 16:44:05 -07:00
Justine Tunney
12ecaf8650
Modernize ipv4.games server
The server was originally written before I implemented support for POSIX
thread cancelation. We now use standard pthreads APIs instead of talking
directly to *NSYNC. This means we're no longer using *NSYNC notes, which
aren't as good as the POSIX thread cancelation support I added to *NSYNC
which was only made possible by making *NSYNC part of libc. I believe it
will solve a crash we observed recently with ipv4.games, courtesy of the
individual who goes by the hacker alias Lambro.
2024-08-26 16:05:23 -07:00
Justine Tunney
185e957696
Detect implicit function declarations
This was suppressed recently and it's the worst possible idea when doing
greenfield software development with C. I'm so sorry it slipped through.
If the C standards committee was smart they would change the standard so
that implicit int becomes implicit long. Then problems such as this will
never occur and we could even use traditional C safely if we wanted too.
2024-08-26 15:34:08 -07:00
Justine Tunney
ebe1cbb1e3
Add crash proofing to ipv4.games server 2024-08-26 12:57:28 -07:00
Justine Tunney
e7b586e7f8
Add preliminary support for cosmocc -mclang
C++ code compiles very slowly with cosmocc, possibly because we're using
LLVM LIBCXX with GCC, and LLVM doesn't work as hard to make GCC go fast.
Therefore, it should be possible, to ask cosmocc to favor Clang over GCC
under the hood. On llamafile, my intention's to use this to make certain
files, e.g. llama.cpp/common.cpp, go from taking 17 seconds to 5 seconds

This new -mclang flag isn't ready for production yet since there's still
the question of how to get Clang to generate SJLJ exception code. If you
use this, then it's recommended you also pass -fno-exceptions.

The tradeoff is we're adding a 121mb binary to the cosmocc distribution.
There are no plans as of yet to fully migrate to Clang since GCC is very
good and has always treated us well.
2024-08-26 12:33:35 -07:00
Justine Tunney
111ec9a989
Fix bug we added to *NSYNC a while ago
This is believed to fix a crash, that's possible in nsync_waiter_free_()
when you call pthread_cond_timedwait(), or nsync_cv_wait_with_deadline()
where an assertion can fail. Thanks ipv4.games for helping me find this!
2024-08-26 12:25:50 -07:00
Justine Tunney
f3ce684aef
Fix getpeername() bug on Windows
The WIN32 getpeername() function returns ENOTCONN when it uses connect()
the SOCK_NONBLOCK way. So we simply store the address, provided earlier.
2024-08-25 11:28:53 -07:00
Justine Tunney
908b7a82ca
Add VSCode settings 2024-08-25 11:02:31 -07:00
Justine Tunney
bb06230f1e
Avoid linker conflicts on DescribeFoo symbols
These symbols belong to the user. It caused a confusing error for Blink.
2024-08-24 18:10:22 -07:00
Justine Tunney
38cc4b3c68
Get rid of some legacy code 2024-08-24 17:53:30 -07:00
Justine Tunney
37ca1badaf
Make Emacs load 2x faster 2024-08-23 20:08:05 -07:00
Justine Tunney
1a9f82bc9f
Romanize Hindi, Yiddish, Arabic, Cyrillic, etc. 2024-08-20 16:27:16 -07:00
Justine Tunney
df1aee7ce5
Upgrade superconfigure and monorepo toolchain
See #1260
2024-08-20 08:46:21 -07:00
Justine Tunney
2d44142444
Get Meson builds working
See #917
2024-08-19 08:40:18 -07:00
Justine Tunney
4bbc16e2cc
Add helpful error messages 2024-08-19 07:28:49 -07:00
Justine Tunney
863c704684
Add string similarity function 2024-08-17 16:45:07 -07:00
Justine Tunney
60e697f7b2
Move LoadZipArgs() to cosmo.h 2024-08-17 12:06:27 -07:00
Justine Tunney
4389f4709a
Expose wmempcpy() to _GNU_SOURCE 2024-08-17 08:13:22 -07:00
Justine Tunney
ca2c30c977
Release redbean v3.0.0 2024-08-17 06:45:35 -07:00
Justine Tunney
eb6e96f036
Change InfoZIP to not auto-append .zip to pathname 2024-08-17 06:45:23 -07:00
Justine Tunney
77be460290
Make Windows REPLs great again 2024-08-17 06:32:10 -07:00
Justine Tunney
8e14b27749
Make fread() more consistent with glibc 2024-08-17 02:57:22 -07:00
Justine Tunney
1d532ba3f8
Disable some anti-Musl Lua tests 2024-08-17 02:20:08 -07:00
Justine Tunney
b2a1811c01
Add missing pragma 2024-08-16 21:49:28 -07:00
Justine Tunney
2eda50929b
Add stdfloat header
Fixes #1260
2024-08-16 21:38:00 -07:00
Justine Tunney
098638cc6c
Fix pthread_kill_test flake on qemu 2024-08-16 21:18:26 -07:00
Justine Tunney
732554ce3a
Release Cosmopolitan v3.7.1 2024-08-16 11:56:47 -07:00
Gavin Hayes
914d521090
Fix relative Windows path normalization (#1261)
Fixes #1223
2024-08-16 11:55:49 -07:00
Justine Tunney
11d9fb521d
Make atomics faster on aarch64
This change implements the compiler runtime for ARM v8.1 ISE atomics and
gets rid of the mandatory -mno-outline-atomics flag. It can dramatically
speed things up, on newer ARM CPUs, as indicated by the changed lines in
test/libc/thread/footek_test.c. In llamafile dispatching on hwcap atomic
also shaved microseconds off synchronization barriers.
2024-08-16 11:14:46 -07:00
Justine Tunney
de0cde8def
Release Cosmopolitan v3.7.0 2024-08-16 07:43:10 -07:00
Justine Tunney
5bd22aef12
Experiment with supporting Windows Arm64 natively
So far I haven't found any way to run native Arm64 code on Windows Arm64
without using MSVC. When I build a PE binary from scratch that should be
a valid Windows Arm64 program, the OS refuses to run it. Possibly due to
requiring additional content like XML manifests or relocation or control
flow integrity data that isn't normally required on x64. I've also tried
using VirtualAlloc2() to JIT an Arm64 native function, but VirtualAlloc2
always fails with invalid parameter. I tried using MSVC to create an ARM
DLL that my x64 emulated program can link at runtime, to pass a function
pointer with ARM code, but LoadLibrary() rejects ARM DLLs as invalid exe

The only option left, is likely to write a new program like ape/ape-m1.c
which can be compiled by MSVC to load and run an AARCH64 ELF executable.
The emulated x64 binary would detect emulation using IsWow64Process2 and
then drop the loader executable in a temporary folder, and re-launch the
original executable, using the Arm64 segments of the cosmocc fat binary.
2024-08-16 06:43:59 -07:00
Justine Tunney
1671283f1a
Avoid clobbering errno 2024-08-15 23:54:14 -07:00
Justine Tunney
0a79c6961f
Make malloc scalable on all platforms
It turns out sched_getcpu() didn't work on many platforms. So the system
call now has tests and is well documented. We now employ new workarounds
on platforms where it isn't supported in our malloc() implementation. It
was previously the case that malloc() was only scalable on Linux/Windows
for x86-64. Now the other platforms are scalable too.
2024-08-15 23:32:53 -07:00
Justine Tunney
3fd275f59f
Import optimized routines changes to exp10 2024-08-15 18:37:33 -07:00
Justine Tunney
2045e87b7c
Fix build issues 2024-08-15 18:37:33 -07:00
Jacob Hummer
24666e121d
add nightly cosmocc (artifact) builds (#1254)
Using https://nightly.link/ with GitHub actions artifacts you can have a
nightly build (but not a _release_ -- there's no releasing or
pre-releasing happening) of cosmocc.

Example URL if this PR were merged:


https://nightly.link/jart/cosmopolitan/workflows/nightly-cosmocc/master/cosmocc.zip

Or you can just download it directly from the GitHub "Actions"
https://github.com/jart/cosmopolitan/actions workflow summary page of a
particular run

example from my own fork:


![image](https://github.com/user-attachments/assets/8ba708dd-8289-4f8b-932c-cf535ee86f62)
could download by clicking on the artifact

or by using third-party service to provide a link for unauthenticated
requests (like wget or curl)

https://nightly.link/jcbhmr/cosmopolitan/workflows/tool-cosmocc-package-sh/master/cosmocc.zip

this would be useful for users who don't want to or can't figure out how
to build cosmocc themselves (like Windows) but still want to use a
nightly build since a fix hasn't been released as a release version yet.

this would also be a good way to test the release process but instead of
pushing the `cosmocc.zip` to _wherever it goes now_ you publish it as a
github actions artifact for the very few nightly bleeding edge users to
use & test.

you don't have to use https://nightly.link or recommend it or anything;
i just know its a cool way to wget or curl the URLs instead of
downloading it via your browser web UI. particularly useful for
remote/ssh/web-ide development.
2024-08-15 00:47:40 -04:00
Justine Tunney
ff1a0d1c40
Upgrade to superconfigure z0.0.51 2024-08-04 14:59:53 -07:00
Justine Tunney
7f0db3e3b9
Add last commit to .git-blame-ignore-revs 2024-08-04 12:54:30 -07:00
Justine Tunney
31194165d2
Remove .internal from more header filenames 2024-08-04 12:52:25 -07:00
Justine Tunney
c265c17d54
Fix the build 2024-08-04 07:08:28 -07:00
Justine Tunney
7499367060
Ignore -Wimplicit-function-declaration in cosmocc 2024-08-03 21:36:36 -07:00
Justine Tunney
3f26dfbb31
Share file offset across execve() on Windows
This is a breaking change. It defines the new environment variable named
_COSMO_FDS_V2 which is used for inheriting non-stdio file descriptors on
execve() or posix_spawn(). No effort has been spent thus far integrating
with the older variable. If a new binary launches the older ones or vice
versa they'll only be able to pass stdin / stdout / stderr to each other
therefore it's important that you upgrade all your cosmo binaries if you
depend on this functionality. You'll be glad you did because inheritance
of file descriptors is more aligned with the POSIX standard than before.
2024-08-03 17:48:00 -07:00
Justine Tunney
761c6ad615
Share file offset across processes
This change ensures that if a file descriptor for an open disk file gets
shared by multiple processes within a process tree, then lseek() changes
will be visible across processes, and read() / write() are synchronized.
Note this only applies to Windows, because UNIX kernels already do this.
2024-08-03 01:39:11 -07:00
Justine Tunney
a80ab3f8fe
Implement bf16 compiler runtime library 2024-08-02 02:04:53 -07:00
Gautham
9ebacb7892
Convert GCC 14 errors back to warnings (#1247)
https://gcc.gnu.org/gcc-14/porting_to.html#warnings-as-errors

Changing these to warnings helps build code with `cosmocc`. Perhaps this
can be a patch to `cosmocc` or skipped entirely.
2024-08-01 21:42:40 -07:00
Justine Tunney
6ac3d3b804
Add precompiled header support to cosmocc 2024-07-31 06:04:41 -07:00
Justine Tunney
f8cfc89eba
Allow -c to be specified with -E in cosmocc 2024-07-31 02:09:15 -07:00
Justine Tunney
4ed4a1095a
Improve build latency 2024-07-31 01:21:27 -07:00
Justine Tunney
8d8aecb6d9
Avoid legacy instruction penalties on x86 2024-07-31 01:02:38 -07:00
Justine Tunney
1fba310e22
Delete mislocated headers 2024-07-31 01:01:00 -07:00
Justine Tunney
bb815eafaf
Update Musl Libc code
We now have implement all of Musl's localization code, the same way that
Musl implements localization. You may need setlocale(LC_ALL, "C.UTF-8"),
just in case anything stops working as expected.
2024-07-30 22:51:29 -07:00
Justine Tunney
d0360bf4bd
Introduce cosmoranlib 2024-07-29 19:18:52 -07:00
Justine Tunney
8cdb3e136b
Check in ruler summation experiments 2024-07-29 18:02:16 -07:00
Justine Tunney
3dab207351
Remove mkfifo() prototype 2024-07-29 07:42:54 -07:00
Justine Tunney
1092d9b7e8
Remove old usages of %n 2024-07-29 01:39:36 -07:00
Justine Tunney
d40acc60b1
Detect more x86 features 2024-07-29 00:16:29 -07:00
Justine Tunney
cf1559c448
Remove __threaded variable 2024-07-28 23:43:30 -07:00
Justine Tunney
01b09bc817
Support printf %n directive 2024-07-28 22:27:06 -07:00
Justine Tunney
c1a0b017e9
Fix the build 2024-07-28 21:02:04 -07:00
Justine Tunney
77d3a07ff2
Fix std::filesystem
This change makes a second pass, at fixing the errno issue with libcxx's
filesystem code. Previously, 89.01% of LLVM's test suite was passing and
now 98.59% of their tests pass. Best of all, it's now possible for Clang
to be built as a working APE binary that can to compile the Cosmopolitan
repository. Please note it has only been vetted so far for some objects,
and more work would obviously need to be done in cosmo, to fix warnings.
2024-07-28 17:31:21 -07:00
Justine Tunney
0d7c272d3f
Don't use sendfile() in libcxx 2024-07-28 17:31:21 -07:00
Justine Tunney
18964e5d76
Fix remove() directory on Windows 2024-07-28 17:31:21 -07:00
Justine Tunney
e18fe1e112
Freshen build/bootstrap/cocmd
See https://news.ycombinator.com/item?id=41055121
2024-07-27 23:22:11 -07:00
Justine Tunney
8621034d42
Release Cosmopolitan v3.6.2 2024-07-27 20:20:54 -07:00
Justine Tunney
32292aee84
Fix build determinism issue 2024-07-27 19:03:32 -07:00
Justine Tunney
81dd9639ed
Fix bug in tool/decode/zip 2024-07-27 18:34:21 -07:00
Justine Tunney
f147d3dde9
Fix some static analysis issues 2024-07-27 09:16:54 -07:00
Justine Tunney
fb54604b31
Upgrade to superconfigure z0.0.48
Our cosmocc binaries are now built with GCC 14.1, using the Cosmo commit
efb3a34608 from yesterday.

GCC is now configured using --enable-analyzer so you can use -fanalyzer.
2024-07-27 08:35:36 -07:00
Justine Tunney
cdfcee51ca
Properly serialize fork() operations
This change solves an issue where many threads attempting to spawn forks
at once would cause fork() performance to degrade with the thread count.
Things got real nasty on NetBSD, which slowed down the whole test fleet,
because there's no vfork() and we're forced to use fork() in our server.

   threads      count task
         1       1062 fork+exit+wait
         2        668 fork+exit+wait
         4         66 fork+exit+wait
         8         19 fork+exit+wait
        16         22 fork+exit+wait
        32         16 fork+exit+wait

Things are now much less bad on NetBSD, but not great, since it does not
have futexes; we rely on its semaphore file descriptors to do conditions

   threads      count task
         1       1085 fork+exit+wait
         2        842 fork+exit+wait
         4        532 fork+exit+wait
         8        400 fork+exit+wait
        16        276 fork+exit+wait
        32         66 fork+exit+wait

With OpenBSD which also lacks vfork(), things were just as bad as NetBSD

   threads      count task
         1        584 fork+exit+wait
         2        687 fork+exit+wait
         4        206 fork+exit+wait
         8         24 fork+exit+wait
        16         33 fork+exit+wait
        32         26 fork+exit+wait

But since OpenBSD has futexes fork() works terrifically thanks to *NSYNC

   threads      count task
         1        525 fork+exit+wait
         2        580 fork+exit+wait
         4        451 fork+exit+wait
         8        479 fork+exit+wait
        16        408 fork+exit+wait
        32        373 fork+exit+wait

This issue would most likely only manifest itself, when pthread_atfork()
callers manage to slip a spin lock into the outermost position of fork's
list of locks. Since fork() is very slow, a spin lock can be devastating

Needless to say vfork() rules and anyone who says differently is kidding
themselves. Look at what a FreeBSD 14.1 virtual machine with equal specs
can do over the course of three hundred milliseconds.

   threads      count task
         1       2559 vfork+exit+wait
         2       5389 vfork+exit+wait
         4      34933 vfork+exit+wait
         8      43273 vfork+exit+wait
        16      49648 vfork+exit+wait
        32      40247 vfork+exit+wait

So it's a shame that so few OSes support vfork(). It creates an unsavory
situation, where someone wanting to build a server that spawns processes
would be better served to not use threads and favor a multiprocess model
2024-07-27 08:23:44 -07:00
Justine Tunney
18a620cc1a
Make some improvements of little consequence 2024-07-27 08:20:18 -07:00
Justine Tunney
690d3df66e
Expand the virtual address space on Windows 2024-07-27 08:19:05 -07:00
Justine Tunney
efb3a34608
Fix package.sh build error 2024-07-26 06:57:24 -07:00
Justine Tunney
642e9cb91a
Introduce cosmocc flags -mdbg -mtiny -moptlinux
The cosmocc.zip toolchain will now include four builds of the libcosmo.a
runtime libraries. You can pass the -mdbg flag if you want to debug your
cosmopolitan runtime. You can pass the -moptlinux flag if you don't want
windows code lurking in your binary. See tool/cosmocc/README.md for more
details on how these flags may be used and their important implications.
2024-07-26 05:10:25 -07:00
Justine Tunney
59692b0882
Make spinlocks faster (take two)
This change is green on x86 and arm test fleet.
2024-07-26 00:45:24 -07:00
Justine Tunney
02e1cbcd00
Revert "Make spin locks go faster"
This reverts commit c8e25d811c.
2024-07-25 22:24:32 -07:00
Justine Tunney
0679cfeb41
Fix build 2024-07-25 22:12:08 -07:00
Justine Tunney
edd5e2c8e3
Expose gethostbyname() 2024-07-25 19:21:35 -07:00
Justine Tunney
c8e25d811c
Make spin locks go faster 2024-07-25 17:37:11 -07:00
Justine Tunney
a31d5ea399
Remove cosmoc++ compiler warning 2024-07-25 15:20:41 -07:00
Justine Tunney
7d88343973
Release Cosmopolitan v3.6.1 2024-07-25 13:34:02 -07:00
Justine Tunney
3915ca0f71
Have cosmocc define -Wno-implicit-int
GCC 13+ changed its policies to be very aggressive about breaking builds
that have anything resembling K&R C. I very strongly disagree with these
decisions. Users who think their compiler should should also be a linter
are perfectly welcome to opt-in to -Wimplicit-int.
2024-07-25 05:51:57 -07:00
Justine Tunney
2c4b88753b
Add special errno handling to libcxx 2024-07-25 01:23:02 -07:00
Justine Tunney
0f486a13c8
Fix fdlibm license 2024-07-24 20:42:08 -07:00
Justine Tunney
1020dd41cc
bzero() should be defined without special defines 2024-07-24 16:15:30 -07:00
Justine Tunney
d3a13e8d70
Improve lock hierarchy
- NetBSD no longer needs a spin lock to create semaphores
- Windows fork() now locks process manager in correct order
2024-07-24 16:05:48 -07:00
Justine Tunney
7ba9a73840
Remove more _Atomic keywords from public headers
It's been thirteen years and C++ still hasn't implemented this wonderful
simple builtin keyword. In C++23 a solution was provided for making this
work in C++ which is libcxx's stdatomic.h. Including that header schleps
in literally 253 unique header files!! Many of the header files it needs
are libc header files like pthread.h where we need to have the _Atomic()
keyword, but since <atomic> depends on pthreads we can't have it include
the <stdatomic.h> header that defines _Atomic for C++ users, and instead
we simply make the type non-atomic, hoping and praying only C code shall
use those internal data structures. This just shows how STL clowns can't
be trusted to define the innermost primitives of a language. They should
instead be focusing on being the best at algorithms and data structures.
2024-07-24 13:56:03 -07:00
Justine Tunney
4a1ae86124
Make some code build faster 2024-07-24 12:26:52 -07:00
Justine Tunney
5dd7ddb9ea
Remove bad defines from early days of project
These definitions were causing issues with building LLVM. It is possible
they also caused crashes we've seen with our MacOS ARM64 OpenMP support.
2024-07-24 12:11:21 -07:00
Justine Tunney
f25fbbaaeb
Use libcxx abi v1 2024-07-24 09:49:48 -07:00
Justine Tunney
fbc4b03d4c
Restore support for AMD K8 2024-07-24 08:59:29 -07:00
Justine Tunney
e398f3887c
Make more improvements to threads and mappings
- NetBSD should now have faster synchronization
- POSIX barriers may now be shared across processes
- An edge case with memory map tracking has been fixed
- Grand Central Dispatch is no longer used on MacOS ARM64
- POSIX mutexes in normal mode now use futexes across processes
2024-07-24 01:19:54 -07:00
Justine Tunney
2187d6d2dd
Fix MODE=optlinux for GitHub Actions 2024-07-23 04:17:22 -07:00
Justine Tunney
0602ff6bab
Fix MODE=optlinux and MODE=tiny builds 2024-07-23 04:04:19 -07:00
3173 changed files with 348935 additions and 36296 deletions

View file

@ -27,3 +27,5 @@ da8baf2aa5ce93b958aca90a0ae69f537806324b
369f9740de4534c28d0e81ab2afc99decbb9a3e6
# Get rid of .internal.h convention in LIBC_INTRIN
86d884cce24d773e298a2714c1e3d91ecab9be45
# Remove .internal from more header filenames
31194165d2afca36c2315a6e7ca2f0797dde09e3

8
.gitattributes vendored
View file

@ -1,4 +1,10 @@
# -*- conf -*-
*.gz binary
/build/bootstrap/*.com binary
*.so binary
*.dll binary
*.dylib binary
/build/bootstrap/* binary
/usr/share/terminfo/* binary
/usr/share/terminfo/*/* binary
/usr/share/zoneinfo/* binary
/usr/share/zoneinfo/*/* binary

View file

@ -1,5 +1,8 @@
name: build
env:
COSMOCC_VERSION: 3.9.2
on:
push:
branches:
@ -19,13 +22,48 @@ jobs:
matrix:
mode: ["", tiny, rel, tinylinux, optlinux]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
# Full checkout needed for git-restore-mtime-bare.
fetch-depth: 0
# TODO(jart): fork this action.
- uses: chetan/git-restore-mtime-action@v2
- uses: actions/cache/restore@v4
id: cache
with:
path: |
.cosmocc
o
key: ${{ env.COSMOCC_VERSION }}-${{ matrix.mode }}-${{ github.sha }}
restore-keys: |
${{ env.COSMOCC_VERSION }}-${{ matrix.mode }}-
${{ env.COSMOCC_VERSION }}-
- name: Restore mtimes
if: steps.cache.outputs.cache-hit == 'true'
run: |
while read mtime file; do
[ -f "$file" ] && touch -d "@$mtime" "$file"
done < o/.mtimes
- name: support ape bins 1
run: sudo cp build/bootstrap/ape.elf /usr/bin/ape
run: sudo cp -a build/bootstrap/ape.elf /usr/bin/ape
- name: support ape bins 2
run: sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
- name: make matrix
run: V=0 make -j2 MODE=${{ matrix.mode }}
- name: Save mtimes
run: |
find o -type f -exec stat -c "%Y %n" {} \; > o/.mtimes
- uses: actions/cache/save@v4
with:
path: |
.cosmocc
o
key: ${{ env.COSMOCC_VERSION }}-${{ matrix.mode }}-${{ github.sha }}

24
.github/workflows/nightly-cosmocc.yml vendored Normal file
View file

@ -0,0 +1,24 @@
name: Nightly cosmocc
on:
schedule:
# https://crontab.guru/#37_4_*_*_*
- cron: "37 4 * * *"
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true
jobs:
build-cosmocc:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: |
sudo cp build/bootstrap/ape.elf /usr/bin/ape
sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
- run: tool/cosmocc/package.sh
# https://github.com/actions/upload-artifact/issues/590
- uses: actions/upload-artifact@v4.3.5
with:
name: cosmocc
path: cosmocc
compression-level: 9

1
.gitignore vendored
View file

@ -15,3 +15,4 @@ __pycache__
/tool/emacs/*.elc
/perf.data
/perf.data.old
/qemu*core

36
.vscode/settings.json vendored Normal file
View file

@ -0,0 +1,36 @@
{
"C_Cpp.default.compilerPath": ".cosmocc/3.9.2/bin/aarch64-linux-cosmo-c++",
"C_Cpp.default.compilerArgs": [
"-nostdinc",
"-nostdlib",
"-iquote.",
"-isystemlibc/isystem",
"-isystemthird_party/libcxx",
"-includelibc/integral/normalize.inc",
"-D_COSMO_SOURCE",
"-D__aarch64__"
],
"[c]": {
"editor.tabSize": 2,
"editor.insertSpaces": true
},
"[cpp]": {
"editor.tabSize": 2,
"editor.insertSpaces": true
},
"[makefile]": {
"editor.tabSize": 8,
"editor.insertSpaces": false
},
"[make]": {
"editor.tabSize": 8,
"editor.insertSpaces": false
},
"[assembly]": {
"editor.tabSize": 8,
"editor.insertSpaces": true
},
"files.associations": {
"log.h": "c"
}
}

141
Makefile
View file

@ -77,7 +77,8 @@ COMMA := ,
PWD := $(shell pwd)
# detect wsl2 running cosmopolitan binaries on the host by checking whether:
# - user ran build/bootstrap/make, in which case make's working directory is in wsl
# - user ran .cosmocc/current/bin/make, in which case make's working directory
# is in wsl
# - user ran make, in which case cocmd's working directory is in wsl
ifneq ($(findstring //wsl.localhost/,$(CURDIR) $(PWD)),)
$(warning wsl2 interop is enabled)
@ -89,7 +90,7 @@ UNAME_S := $(shell uname -s)
# apple still distributes a 17 year old version of gnu make
ifeq ($(MAKE_VERSION), 3.81)
$(error please use build/bootstrap/make)
$(error please use https://cosmo.zip/pub/cosmos/bin/make)
endif
LC_ALL = C
@ -115,10 +116,8 @@ ZIPCOPY = $(BOOTSTRAP)/zipcopy
PECHECK = $(BOOTSTRAP)/pecheck
FIXUPOBJ = $(BOOTSTRAP)/fixupobj
OBJBINCOPY = $(BOOTSTRAP)/objbincopy
MKDIR = build/bootstrap/mkdir -p
COMPILE = build/bootstrap/compile -V9 -M2048m -P8192 $(QUOTA)
IGNORE := $(shell $(MKDIR) $(TMPDIR))
MKDIR = $(BOOTSTRAP)/mkdir.ape -p
COMPILE = $(BOOTSTRAP)/compile.ape -V9 -M2048m -P8192 $(QUOTA)
# the default build modes is empty string
# on x86_64 hosts, MODE= is the same as MODE=x86_64
@ -134,14 +133,13 @@ endif
ifneq ($(findstring aarch64,$(MODE)),)
ARCH = aarch64
HOSTS ?= pi studio freebsdarm
HOSTS ?= pi pi5 studio freebsdarm
else
ARCH = x86_64
HOSTS ?= freebsd rhel7 xnu openbsd netbsd win10
HOSTS ?= freebsd rhel7 xnu openbsd netbsd win10 luna
endif
ZIPOBJ_FLAGS += -a$(ARCH)
IGNORE := $(shell $(MKDIR) $(TMPDIR))
export ADDR2LINE
export LC_ALL
@ -150,10 +148,12 @@ export MODE
export SOURCE_DATE_EPOCH
export TMPDIR
COSMOCC = .cosmocc/3.6.0
COSMOCC = .cosmocc/3.9.2
BOOTSTRAP = $(COSMOCC)/bin
TOOLCHAIN = $(COSMOCC)/bin/$(ARCH)-linux-cosmo-
DOWNLOAD := $(shell build/download-cosmocc.sh $(COSMOCC) 3.6.0 4918c45ac3e0972ff260e2a249e25716881e39fb679d5e714ae216a2ef6c3f7e)
DOWNLOAD := $(shell build/download-cosmocc.sh $(COSMOCC) 3.9.2 f4ff13af65fcd309f3f1cfd04275996fb7f72a4897726628a8c9cf732e850193)
IGNORE := $(shell $(MKDIR) $(TMPDIR))
AS = $(TOOLCHAIN)as
CC = $(TOOLCHAIN)gcc
@ -275,10 +275,16 @@ include libc/BUILD.mk #─┘
include libc/sock/BUILD.mk #─┐
include net/http/BUILD.mk # ├──ONLINE RUNTIME
include third_party/musl/BUILD.mk # │ You can communicate with the network
include third_party/regex/BUILD.mk # │
include third_party/tr/BUILD.mk # │
include third_party/sed/BUILD.mk # │
include libc/system/BUILD.mk # │
include libc/x/BUILD.mk # │
include dsp/scale/BUILD.mk # │
include dsp/mpeg/BUILD.mk # │
include dsp/tty/BUILD.mk # │
include dsp/audio/BUILD.mk # │
include dsp/prog/BUILD.mk # │
include dsp/BUILD.mk # │
include third_party/stb/BUILD.mk # │
include third_party/mbedtls/BUILD.mk # │
@ -292,8 +298,7 @@ include third_party/libcxx/BUILD.mk # │
include third_party/openmp/BUILD.mk # │
include third_party/pcre/BUILD.mk # │
include third_party/less/BUILD.mk # │
include net/https/BUILD.mk # │
include third_party/regex/BUILD.mk #─┘
include net/https/BUILD.mk #─┘
include third_party/tidy/BUILD.mk
include third_party/BUILD.mk
include third_party/nsync/testing/BUILD.mk
@ -312,8 +317,6 @@ include third_party/double-conversion/test/BUILD.mk
include third_party/lua/BUILD.mk
include third_party/tree/BUILD.mk
include third_party/zstd/BUILD.mk
include third_party/tr/BUILD.mk
include third_party/sed/BUILD.mk
include third_party/awk/BUILD.mk
include third_party/hiredis/BUILD.mk
include third_party/make/BUILD.mk
@ -366,6 +369,7 @@ include test/libc/fmt/BUILD.mk
include test/libc/time/BUILD.mk
include test/libc/proc/BUILD.mk
include test/libc/stdio/BUILD.mk
include test/libc/system/BUILD.mk
include test/libc/BUILD.mk
include test/net/http/BUILD.mk
include test/net/https/BUILD.mk
@ -429,68 +433,71 @@ HTAGS: o/$(MODE)/hdrs-old.txt $(filter-out third_party/libcxx/%,$(HDRS)) #o/$(MO
loc: private .UNSANDBOXED = 1
loc: o/$(MODE)/tool/build/summy
find -name \*.h -or -name \*.c -or -name \*.S | \
find -name \*.h -or -name \*.hpp -or -name \*.c -or -name \*.cc -or -name \*.cpp -or -name \*.S -or -name \*.mk | \
$(XARGS) wc -l | grep total | awk '{print $$1}' | $<
# PLEASE: MAINTAIN TOPOLOGICAL ORDER
# FROM HIGHEST LEVEL TO LOWEST LEVEL
COSMOPOLITAN_OBJECTS = \
COSMOPOLITAN = \
CTL \
THIRD_PARTY_DOUBLECONVERSION \
THIRD_PARTY_OPENMP \
TOOL_ARGS \
NET_HTTP \
LIBC_SOCK \
LIBC_NT_WS2_32 \
LIBC_NT_IPHLPAPI \
LIBC_X \
THIRD_PARTY_GETOPT \
DSP_AUDIO \
LIBC_CALLS \
LIBC_DLOPEN \
LIBC_ELF \
LIBC_FMT \
LIBC_INTRIN \
LIBC_IRQ \
LIBC_LOG \
THIRD_PARTY_TZ \
THIRD_PARTY_MUSL \
THIRD_PARTY_ZLIB_GZ \
LIBC_MEM \
LIBC_NEXGEN32E \
LIBC_NT_ADVAPI32 \
LIBC_NT_BCRYPTPRIMITIVES \
LIBC_NT_COMDLG32 \
LIBC_NT_GDI32 \
LIBC_NT_IPHLPAPI \
LIBC_NT_KERNEL32 \
LIBC_NT_NTDLL \
LIBC_NT_PDH \
LIBC_NT_POWRPROF \
LIBC_NT_PSAPI \
LIBC_NT_REALTIME \
LIBC_NT_SHELL32 \
LIBC_NT_SYNCHRONIZATION \
LIBC_NT_USER32 \
LIBC_NT_WS2_32 \
LIBC_PROC \
LIBC_RUNTIME \
LIBC_SOCK \
LIBC_STDIO \
LIBC_STR \
LIBC_SYSTEM \
LIBC_SYSV \
LIBC_SYSV_CALLS \
LIBC_THREAD \
LIBC_TINYMATH \
LIBC_VGA \
LIBC_X \
NET_HTTP \
THIRD_PARTY_COMPILER_RT \
THIRD_PARTY_DLMALLOC \
THIRD_PARTY_DOUBLECONVERSION \
THIRD_PARTY_GDTOA \
THIRD_PARTY_GETOPT \
THIRD_PARTY_LIBCXXABI \
THIRD_PARTY_LIBUNWIND \
LIBC_STDIO \
THIRD_PARTY_GDTOA \
THIRD_PARTY_REGEX \
LIBC_THREAD \
LIBC_PROC \
THIRD_PARTY_NSYNC_MEM \
LIBC_MEM \
THIRD_PARTY_DLMALLOC \
LIBC_DLOPEN \
LIBC_RUNTIME \
THIRD_PARTY_MUSL \
THIRD_PARTY_NSYNC \
LIBC_ELF \
LIBC_IRQ \
LIBC_CALLS \
LIBC_SYSV_CALLS \
LIBC_VGA \
LIBC_NT_PSAPI \
LIBC_NT_POWRPROF \
LIBC_NT_PDH \
LIBC_NT_GDI32 \
LIBC_NT_COMDLG32 \
LIBC_NT_USER32 \
LIBC_NT_NTDLL \
LIBC_NT_ADVAPI32 \
LIBC_NT_SYNCHRONIZATION \
LIBC_FMT \
THIRD_PARTY_ZLIB \
THIRD_PARTY_NSYNC_MEM \
THIRD_PARTY_OPENMP \
THIRD_PARTY_PUFF \
THIRD_PARTY_COMPILER_RT \
LIBC_TINYMATH \
THIRD_PARTY_REGEX \
THIRD_PARTY_TZ \
THIRD_PARTY_XED \
LIBC_STR \
LIBC_SYSV \
LIBC_INTRIN \
LIBC_NT_BCRYPTPRIMITIVES \
LIBC_NT_KERNEL32 \
LIBC_NEXGEN32E
THIRD_PARTY_ZLIB \
THIRD_PARTY_ZLIB_GZ \
TOOL_ARGS \
COSMOPOLITAN_H_PKGS = \
APE \
DSP_AUDIO \
LIBC \
LIBC_CALLS \
LIBC_ELF \
@ -534,14 +541,14 @@ COSMOCC_PKGS = \
THIRD_PARTY_INTEL
o/$(MODE)/cosmopolitan.a: \
$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_A_OBJS))
$(call reverse,$(call uniq,$(foreach x,$(COSMOPOLITAN),$($(x)))))
COSMOCC_HDRS = \
$(wildcard libc/integral/*) \
$(foreach x,$(COSMOCC_PKGS),$($(x)_HDRS)) \
$(foreach x,$(COSMOCC_PKGS),$($(x)_INCS))
o/cosmocc.h.txt: Makefile
o/cosmocc.h.txt: Makefile libc $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS) $(INCS),$(dir $(x)))) $(HDRS) $(INCS)
$(file >$@, $(call uniq,$(COSMOCC_HDRS)))
COSMOPOLITAN_H_ROOT_HDRS = \

View file

@ -3,12 +3,12 @@
[![build](https://github.com/jart/cosmopolitan/actions/workflows/build.yml/badge.svg)](https://github.com/jart/cosmopolitan/actions/workflows/build.yml)
# Cosmopolitan
[Cosmopolitan Libc](https://justine.lol/cosmopolitan/index.html) makes C
[Cosmopolitan Libc](https://justine.lol/cosmopolitan/index.html) makes C/C++
a build-once run-anywhere language, like Java, except it doesn't need an
interpreter or virtual machine. Instead, it reconfigures stock GCC and
Clang to output a POSIX-approved polyglot format that runs natively on
Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS with the best
possible performance and the tiniest footprint imaginable.
Linux + Mac + Windows + FreeBSD + OpenBSD 7.3 + NetBSD + BIOS with the
best possible performance and the tiniest footprint imaginable.
## Background
@ -87,15 +87,22 @@ ape/apeinstall.sh
```
You can now build the mono repo with any modern version of GNU Make. To
make life easier, we've included one in the cosmocc toolchain, which is
guaranteed to be compatible and furthermore includes our extensions for
doing build system sandboxing.
bootstrap your build, you can install Cosmopolitan Make from this site:
https://cosmo.zip/pub/cosmos/bin/make
E.g.:
```sh
build/bootstrap/make -j8
curl -LO https://cosmo.zip/pub/cosmos/bin/make
./make -j8
o//examples/hello
```
After you've built the repo once, you can also use the make from your
cosmocc at `.cosmocc/current/bin/make`. You might even prefer to alias
make to `$COSMO/.cosmocc/current/bin/make`.
Since the Cosmopolitan repository is very large, you might only want to
build one particular thing. Here's an example of a target that can be
compiled relatively quickly, which is a simple POSIX test that only
@ -103,7 +110,7 @@ depends on core LIBC packages.
```sh
rm -rf o//libc o//test
build/bootstrap/make o//test/posix/signal_test
.cosmocc/current/bin/make o//test/posix/signal_test
o//test/posix/signal_test
```
@ -112,21 +119,21 @@ list out each individual one. For example if you wanted to build and run
all the unit tests in the `TEST_POSIX` package, you could say:
```sh
build/bootstrap/make o//test/posix
.cosmocc/current/bin/make o//test/posix
```
Cosmopolitan provides a variety of build modes. For example, if you want
really tiny binaries (as small as 12kb in size) then you'd say:
```sh
build/bootstrap/make m=tiny
.cosmocc/current/bin/make m=tiny
```
You can furthermore cut out the bloat of other operating systems, and
have Cosmopolitan become much more similar to Musl Libc.
```sh
build/bootstrap/make m=tinylinux
.cosmocc/current/bin/make m=tinylinux
```
For further details, see [//build/config.mk](build/config.mk).
@ -242,16 +249,16 @@ server. You're welcome to join us! <https://discord.gg/FwAVVu7eJ4>
## Support Vector
| Platform | Min Version | Circa |
| :--- | ---: | ---: |
| AMD | K8 Venus | 2005 |
| Intel | Core | 2006 |
| Linux | 2.6.18 | 2007 |
| Windows | 8 [1] | 2012 |
| Darwin (macOS) | 23.1.0+ | 2023 |
| OpenBSD | 7 | 2021 |
| FreeBSD | 13 | 2020 |
| NetBSD | 9.2 | 2021 |
| Platform | Min Version | Circa |
| :--- | ---: | ---: |
| AMD | K8 | 2003 |
| Intel | Core | 2006 |
| Linux | 2.6.18 | 2007 |
| Windows | 8 [1] | 2012 |
| Darwin (macOS) | 23.1.0+ | 2023 |
| OpenBSD | 7.3 or earlier | 2023 |
| FreeBSD | 13 | 2020 |
| NetBSD | 9.2 | 2021 |
[1] See our [vista branch](https://github.com/jart/cosmopolitan/tree/vista)
for a community supported version of Cosmopolitan that works on Windows

View file

@ -45,10 +45,10 @@ o/$(MODE)/ape: $(APE)
o/$(MODE)/ape/aarch64.lds: \
ape/aarch64.lds \
libc/zip.internal.h \
libc/zip.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/macros.internal.h \
libc/macros.h \
libc/str/str.h
APE_LOADER_LDFLAGS = \
@ -162,8 +162,8 @@ o/$(MODE)/ape/ape-no-modify-self.o: \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/macho.h \
libc/macros.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nt/pedef.internal.h \
@ -188,8 +188,8 @@ o/$(MODE)/ape/ape-copy-self.o: \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/macho.h \
libc/macros.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nt/pedef.internal.h \
@ -246,8 +246,6 @@ o/$(MODE)/ape: $(APE_CHECKS) \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.macho \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o
endif
@ -261,8 +259,8 @@ o/$(MODE)/ape/ape.o: \
libc/thread/tls.h \
ape/ape.internal.h \
ape/macros.internal.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/macho.h \
libc/macros.h \
libc/sysv/consts/prot.h \
libc/nt/pedef.internal.h \
libc/runtime/pc.internal.h \
@ -283,7 +281,7 @@ o/$(MODE)/ape/ape.lds: \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/macros.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.internal.h
libc/zip.h

View file

@ -103,10 +103,8 @@ SECTIONS {
*(.eh_frame_entry .eh_frame_entry.*)
}
.eh_frame : ONLY_IF_RO {
KEEP(*(.eh_frame))
*(.eh_frame.*)
}
__eh_frame_hdr_start = SIZEOF(.eh_frame_hdr) > 0 ? ADDR(.eh_frame_hdr) : 0;
__eh_frame_hdr_end = SIZEOF(.eh_frame_hdr) > 0 ? . : 0;
.gcc_except_table : ONLY_IF_RO {
*(.gcc_except_table .gcc_except_table.*)
@ -127,9 +125,11 @@ SECTIONS {
. += CONSTANT(MAXPAGESIZE);
. = DATA_SEGMENT_ALIGN(CONSTANT(MAXPAGESIZE), CONSTANT(COMMONPAGESIZE));
.eh_frame : ONLY_IF_RW {
.eh_frame : {
__eh_frame_start = .;
KEEP(*(.eh_frame))
*(.eh_frame.*)
__eh_frame_end = .;
}
.gnu_extab : ONLY_IF_RW {
@ -259,6 +259,9 @@ SECTIONS {
.debug_ranges 0 : { *(.debug_ranges) }
.debug_macro 0 : { *(.debug_macro) }
.debug_addr 0 : { *(.debug_addr) }
.debug_names 0 : { *(.debug_names) }
.debug_loclists 0 : { *(.debug_loclists) }
.debug_str_offsets 0 : { *(.debug_str_offsets) }
.ARM.attributes 0 : { KEEP(*(.ARM.attributes)) KEEP(*(.gnu.attributes)) }
.note.gnu.arm.ident 0 : { KEEP(*(.note.gnu.arm.ident)) }

View file

@ -16,6 +16,12 @@
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __APPLE__
#error "ape/ape-m1.c is for apple silicon. chances you want ape/loader.c"
#endif
#ifndef __aarch64__
#error "ape/ape-m1.c is for apple silicon; you want: make o//ape/ape.macho"
#endif
#include <assert.h>
#include <dispatch/dispatch.h>
#include <dlfcn.h>

View file

@ -37,7 +37,7 @@
#include "libc/calls/metalfile.internal.h"
#include "libc/dce.h"
#include "libc/elf/def.h"
#include "libc/macho.internal.h"
#include "libc/macho.h"
#include "libc/nexgen32e/uart.internal.h"
#include "libc/nt/pedef.internal.h"
#include "libc/runtime/pc.internal.h"

View file

@ -310,7 +310,7 @@ SECTIONS {
. = ALIGN(__privileged_end > __privileged_start ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: morphable code */
__privileged_start = .;
*(.privileged)
*(.privileged .privileged.*)
__privileged_end = .;
KEEP(*(.ape.pad.text))
@ -329,6 +329,10 @@ SECTIONS {
*(.ubsan.types)
*(.ubsan.data)
__eh_frame_hdr_start_actual = .;
*(.eh_frame_hdr)
__eh_frame_hdr_end_actual = .;
/* Legal Notices */
__notices = .;
KEEP(*(.notice))
@ -382,6 +386,13 @@ SECTIONS {
_tbss_end = .;
} :Tls
.eh_frame : {
__eh_frame_start = .;
KEEP(*(.eh_frame))
*(.eh_frame.*)
__eh_frame_end = .;
} :Ram
.data . : {
/*BEGIN: Read/Write Data */
#if SupportsWindows()
@ -430,7 +441,6 @@ SECTIONS {
KEEP(*(.piro.pad.data))
*(.igot.plt)
KEEP(*(.dataepilogue))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: NT FORK COPYING */
_edata = .;
@ -510,6 +520,9 @@ SECTIONS {
.debug_rnglists 0 : { *(.debug_rnglists) }
.debug_macro 0 : { *(.debug_macro) }
.debug_addr 0 : { *(.debug_addr) }
.debug_names 0 : { *(.debug_names) }
.debug_loclists 0 : { *(.debug_loclists) }
.debug_str_offsets 0 : { *(.debug_str_offsets) }
.gnu.attributes 0 : { KEEP(*(.gnu.attributes)) }
.GCC.command.line 0 : { *(.GCC.command.line) }
@ -573,11 +586,11 @@ ape_rom_memsz = ape_rom_filesz;
ape_rom_align = CONSTANT(COMMONPAGESIZE);
ape_rom_rva = RVA(ape_rom_vaddr);
ape_ram_vaddr = ADDR(.data);
ape_ram_vaddr = ADDR(.eh_frame);
ape_ram_offset = ape_ram_vaddr - __executable_start;
ape_ram_paddr = LOADADDR(.data);
ape_ram_filesz = ADDR(.bss) - ADDR(.data);
ape_ram_memsz = _end - ADDR(.data);
ape_ram_paddr = LOADADDR(.eh_frame);
ape_ram_filesz = ADDR(.bss) - ADDR(.eh_frame);
ape_ram_memsz = _end - ADDR(.eh_frame);
ape_ram_align = CONSTANT(COMMONPAGESIZE);
ape_ram_rva = RVA(ape_ram_vaddr);
@ -587,7 +600,7 @@ ape_stack_offset = 0;
ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000;
ape_stack_paddr = ape_ram_paddr + ape_ram_filesz;
ape_stack_filesz = 0;
ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : 8 * 1024 * 1024;
ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : 4 * 1024 * 1024;
ape_note_offset = ape_cod_offset + (ape_note - ape_cod_vaddr);
ape_note_filesz = ape_note_end - ape_note;
@ -601,6 +614,9 @@ ape_text_memsz = ape_text_filesz;
ape_text_align = CONSTANT(COMMONPAGESIZE);
ape_text_rva = RVA(ape_text_vaddr);
__eh_frame_hdr_start = __eh_frame_hdr_end_actual > __eh_frame_hdr_start_actual ? __eh_frame_hdr_start_actual : 0;
__eh_frame_hdr_end = __eh_frame_hdr_end_actual > __eh_frame_hdr_start_actual ? __eh_frame_hdr_end_actual : 0;
/* we roundup here because xnu wants the file load segments page-aligned */
/* but we don't want to add the nop padding to the ape program, so we'll */
/* let ape.S dd read past the end of the file into the wrapping binaries */

View file

@ -10,8 +10,8 @@ if [ ! -f ape/loader.c ]; then
cd "$COSMO" || exit
fi
if [ -x build/bootstrap/make ]; then
MAKE=build/bootstrap/make
if [ -x .cosmocc/current/bin/make ]; then
MAKE=.cosmocc/current/bin/make
else
MAKE=make
fi

View file

@ -16,7 +16,7 @@
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE.
*/
#include "libc/macros.internal.h"
#include "libc/macros.h"
// Calls _start() function of loaded program.
//

View file

@ -16,10 +16,10 @@
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE.
*/
#include "libc/macho.internal.h"
#include "libc/macho.h"
#include "libc/sysv/consts/prot.h"
#include "libc/dce.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
// Apple Mach-O Executable Headers
// Fixups are applied by objbincopy

View file

@ -18,7 +18,7 @@
*/
#ifndef APE_MACROS_H_
#define APE_MACROS_H_
#include "libc/macros.internal.h"
#include "libc/macros.h"
#ifdef __ASSEMBLER__
/* clang-format off */

View file

@ -18,7 +18,7 @@
*/
#include "libc/dce.h"
#include "ape/ape.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
#ifdef __aarch64__

View file

@ -16,7 +16,7 @@
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE.
*/
#include "libc/macros.internal.h"
#include "libc/macros.h"
// Invokes system call.
//

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View file

@ -14,7 +14,6 @@ ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CCFLAGS += -O2 $(BACKTRACES)
CONFIG_CPPFLAGS += -DSYSDEBUG
TARGET_ARCH ?= -msse3
endif
ifeq ($(MODE), x86_64)
ENABLE_FTRACE = 1
@ -64,7 +63,6 @@ ENABLE_FTRACE = 1
CONFIG_CCFLAGS += $(BACKTRACES) -O
CONFIG_CPPFLAGS += -DSYSDEBUG -DDWARFLESS
CONFIG_LDFLAGS += -S
TARGET_ARCH ?= -msse3
endif
# Optimized Mode
@ -84,7 +82,7 @@ ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG
CONFIG_CCFLAGS += $(BACKTRACES) -O3 -fmerge-all-constants
TARGET_ARCH ?= -march=native
CONFIG_TARGET_ARCH ?= -march=native
endif
# Optimized Linux Mode
@ -102,7 +100,20 @@ CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG -DSUPPORT_VECTOR=1
CONFIG_CCFLAGS += -O3 -fmerge-all-constants
CONFIG_COPTS += -mred-zone
TARGET_ARCH ?= -march=native
CONFIG_TARGET_ARCH ?= -march=native
endif
ifeq ($(MODE), x86_64-optlinux)
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG -DSUPPORT_VECTOR=1
CONFIG_CCFLAGS += -O3 -fmerge-all-constants
CONFIG_COPTS += -mred-zone
CONFIG_TARGET_ARCH ?= -march=native
endif
ifeq ($(MODE), aarch64-optlinux)
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG -DSUPPORT_VECTOR=1
CONFIG_CCFLAGS += -O3 -fmerge-all-constants
CONFIG_COPTS += -mred-zone
endif
# Release Mode
@ -123,7 +134,6 @@ endif
ifeq ($(MODE), rel)
CONFIG_CPPFLAGS += -DNDEBUG -DDWARFLESS
CONFIG_CCFLAGS += $(BACKTRACES) -O2
TARGET_ARCH ?= -msse3
PYFLAGS += -O1
endif
@ -139,18 +149,32 @@ endif
ifeq ($(MODE), dbg)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DMODE_DBG -D__SANITIZE_UNDEFINED__
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG -O0 -fno-inline
OVERRIDE_CFLAGS += -O0
OVERRIDE_CXXFLAGS += -O0
CONFIG_CPPFLAGS += -DMODE_DBG -D__SANITIZE_UNDEFINED__ -Wno-unused-variable -Wno-unused-but-set-variable
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG
CONFIG_COPTS += -fsanitize=undefined
OVERRIDE_CCFLAGS += -fno-pie
QUOTA ?= -C64 -L300
endif
ifeq ($(MODE), x86_64-dbg)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
OVERRIDE_CFLAGS += -O0
OVERRIDE_CXXFLAGS += -O0
CONFIG_CPPFLAGS += -DMODE_DBG -D__SANITIZE_UNDEFINED__ -Wno-unused-variable -Wno-unused-but-set-variable
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG
CONFIG_COPTS += -fsanitize=undefined
TARGET_ARCH ?= -msse3
OVERRIDE_CCFLAGS += -fno-pie
QUOTA ?= -C64 -L300
endif
ifeq ($(MODE), aarch64-dbg)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DMODE_DBG -D__SANITIZE_UNDEFINED__
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG -O0 -fno-inline -fdce
OVERRIDE_CFLAGS += -O0 -fdce
OVERRIDE_CXXFLAGS += -O0 -fdce
CONFIG_CPPFLAGS += -DMODE_DBG -D__SANITIZE_UNDEFINED__ -Wno-unused-variable -Wno-unused-but-set-variable
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG
CONFIG_COPTS += -fsanitize=undefined
QUOTA ?= -C64 -L300
endif
@ -170,7 +194,6 @@ ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CCFLAGS += $(BACKTRACES) -O2
CONFIG_CPPFLAGS += -DSYSDEBUG -DSUPPORT_VECTOR=121
TARGET_ARCH ?= -msse3
endif
# Tiny Mode
@ -200,8 +223,6 @@ CONFIG_CCFLAGS += \
-momit-leaf-frame-pointer \
-foptimize-sibling-calls \
-DDWARFLESS
TARGET_ARCH ?= \
-msse3
PYFLAGS += \
-O2 \
-B
@ -221,8 +242,6 @@ CONFIG_CCFLAGS += \
-momit-leaf-frame-pointer \
-foptimize-sibling-calls \
-DDWARFLESS
TARGET_ARCH ?= \
-msse3
PYFLAGS += \
-O2 \
-B
@ -274,8 +293,6 @@ CONFIG_CCFLAGS += \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops
TARGET_ARCH ?= \
-msse3
endif
# Linux+BSD Tiny Mode
@ -305,8 +322,6 @@ CONFIG_CCFLAGS += \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops
TARGET_ARCH ?= \
-msse3
endif
# Unix Tiny Mode
@ -335,8 +350,6 @@ CONFIG_CCFLAGS += \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops
TARGET_ARCH ?= \
-msse3
endif
# Tiny Metallic Unix Mode
@ -365,8 +378,6 @@ CONFIG_CCFLAGS += \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops
TARGET_ARCH ?= \
-msse3
endif
# no x87 instructions mode
@ -388,7 +399,6 @@ ENABLE_FTRACE = 1
CONFIG_COPTS += -mlong-double-64
CONFIG_CCFLAGS += $(BACKTRACES) -O2
CONFIG_CPPFLAGS += -DSYSDEBUG -DNOX87
TARGET_ARCH ?= -msse3
endif
# LLVM Mode
@ -401,7 +411,6 @@ endif
#
ifeq ($(MODE), llvm)
.STRICT = 0
TARGET_ARCH ?= -msse3
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG -O2
AS = clang
CC = clang
@ -444,7 +453,6 @@ ifeq ($(MODE), ansi)
CONFIG_CFLAGS += -std=c11
#CONFIG_CPPFLAGS += -ansi
CONFIG_CXXFLAGS += -std=c++11
TARGET_ARCH ?= -msse3
endif
ifneq ($(ENABLE_FTRACE),)
@ -502,3 +510,5 @@ ifeq ($(ARCH), aarch64)
CONFIG_CCFLAGS += -fpatchable-function-entry=7,6
endif
endif
TARGET_ARCH ?= $(CONFIG_TARGET_ARCH)

View file

@ -92,10 +92,7 @@ DEFAULT_COPTS ?= \
-fno-gnu-unique \
-fstrict-aliasing \
-fstrict-overflow \
-fno-semantic-interposition \
-fno-dwarf2-cfi-asm \
-fno-unwind-tables \
-fno-asynchronous-unwind-tables
-fno-semantic-interposition
ifeq ($(ARCH), x86_64)
# Microsoft says "[a]ny memory below the stack beyond the red zone
@ -115,14 +112,10 @@ ifeq ($(ARCH), aarch64)
# - Cosmopolitan Libc uses x28 for thread-local storage because Apple
# forbids us from using tpidr_el0 too.
#
# - Cosmopolitan currently lacks an implementation of the runtime
# libraries needed by the -moutline-atomics flag
#
DEFAULT_COPTS += \
-ffixed-x18 \
-ffixed-x28 \
-fsigned-char \
-mno-outline-atomics
-fsigned-char
endif
MATHEMATICAL = \
@ -143,8 +136,6 @@ DEFAULT_CFLAGS = \
DEFAULT_CXXFLAGS = \
-std=gnu++23 \
-fno-rtti \
-fno-exceptions \
-fuse-cxa-atexit \
-Wno-int-in-bool-context \
-Wno-narrowing \

View file

@ -99,3 +99,8 @@ rm -f cosmocc.zip cosmocc.zip.sha256sum
# commit output directory
cd "${OLDPWD}" || die
mv "${OUTPUT_TMP}" "${OUTPUT_DIR}" || die
# update current symlink
BASE=$(basename "${OUTPUT_DIR}")
DIR=$(dirname "${OUTPUT_DIR}")
ln -sfn "$BASE" "$DIR/current"

View file

@ -6,14 +6,14 @@ if [ -n "$OBJDUMP" ]; then
fi
find_objdump() {
if [ -x .cosmocc/3.3.5/bin/$1-linux-cosmo-objdump ]; then
OBJDUMP=.cosmocc/3.3.5/bin/$1-linux-cosmo-objdump
elif [ -x .cosmocc/3.3.5/bin/$1-linux-musl-objdump ]; then
OBJDUMP=.cosmocc/3.3.5/bin/$1-linux-musl-objdump
elif [ -x "$COSMO/.cosmocc/3.3.5/bin/$1-linux-cosmo-objdump" ]; then
OBJDUMP="$COSMO/.cosmocc/3.3.5/bin/$1-linux-cosmo-objdump"
elif [ -x "$COSMO/.cosmocc/3.3.5/bin/$1-linux-musl-objdump" ]; then
OBJDUMP="$COSMO/.cosmocc/3.3.5/bin/$1-linux-musl-objdump"
if [ -x .cosmocc/3.9.2/bin/$1-linux-cosmo-objdump ]; then
OBJDUMP=.cosmocc/3.9.2/bin/$1-linux-cosmo-objdump
elif [ -x .cosmocc/3.9.2/bin/$1-linux-musl-objdump ]; then
OBJDUMP=.cosmocc/3.9.2/bin/$1-linux-musl-objdump
elif [ -x "$COSMO/.cosmocc/3.9.2/bin/$1-linux-cosmo-objdump" ]; then
OBJDUMP="$COSMO/.cosmocc/3.9.2/bin/$1-linux-cosmo-objdump"
elif [ -x "$COSMO/.cosmocc/3.9.2/bin/$1-linux-musl-objdump" ]; then
OBJDUMP="$COSMO/.cosmocc/3.9.2/bin/$1-linux-musl-objdump"
else
echo "error: toolchain not found (try running 'cosmocc --update' or 'make' in the cosmo monorepo)" >&2
exit 1

View file

@ -4,5 +4,5 @@ UNAMES=$(uname -s)
if [ x"$UNAMES" = x"Darwin" ] && [ x"$UNAMEM" = x"arm64" ]; then
exec ape "$@"
else
exec "$@"
exec rusage "$@"
fi

View file

@ -17,6 +17,9 @@ struct conditional<false, T, F>
typedef F type;
};
template<bool B, typename T, typename F>
using conditional_t = typename conditional<B, T, F>::type;
} // namespace ctl
#endif // CTL_CONDITIONAL_H_

View file

@ -19,6 +19,9 @@ template<typename _Tp>
struct is_void : public is_void_<typename ctl::remove_cv<_Tp>::type>::type
{};
template<typename T>
inline constexpr bool is_void_v = is_void<T>::value;
} // namespace ctl
#endif // CTL_IS_VOID_H_

View file

@ -241,8 +241,9 @@ class set
private:
friend class set;
node_type* node_;
node_type* root_;
explicit reverse_iterator(node_type* node) : node_(node)
explicit reverse_iterator(node_type* node, node_type* root) : node_(node), root_(root)
{
}
};
@ -347,17 +348,17 @@ class set
reverse_iterator rbegin()
{
return reverse_iterator(rightmost(root_));
return reverse_iterator(rightmost(root_), root_);
}
const_reverse_iterator rbegin() const
{
return const_reverse_iterator(rightmost(root_));
return const_reverse_iterator(rightmost(root_), root_);
}
const_reverse_iterator crbegin() const
{
return const_reverse_iterator(rightmost(root_));
return const_reverse_iterator(rightmost(root_), root_);
}
iterator end() noexcept
@ -377,17 +378,17 @@ class set
reverse_iterator rend()
{
return reverse_iterator(nullptr);
return reverse_iterator(nullptr, root_);
}
const_reverse_iterator rend() const
{
return const_reverse_iterator(nullptr);
return const_reverse_iterator(nullptr, root_);
}
const_reverse_iterator crend() const
{
return const_reverse_iterator(nullptr);
return const_reverse_iterator(nullptr, root_);
}
void clear() noexcept

618
ctl/shared_ptr.h Normal file
View file

@ -0,0 +1,618 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_SHARED_PTR_H_
#define CTL_SHARED_PTR_H_
#include "exception.h"
#include "is_base_of.h"
#include "is_constructible.h"
#include "is_convertible.h"
#include "remove_extent.h"
#include "unique_ptr.h"
// XXX currently needed to use placement-new syntax (move to cxx.inc?)
void*
operator new(size_t, void*) noexcept;
namespace ctl {
class bad_weak_ptr : public exception
{
public:
const char* what() const noexcept override
{
return "ctl::bad_weak_ptr";
}
};
namespace __ {
template<typename T>
struct ptr_ref
{
using type = T&;
};
template<>
struct ptr_ref<void>
{
using type = void;
};
static inline __attribute__((always_inline)) void
incref(size_t* r) noexcept
{
#ifdef NDEBUG
__atomic_fetch_add(r, 1, __ATOMIC_RELAXED);
#else
ssize_t refs = __atomic_fetch_add(r, 1, __ATOMIC_RELAXED);
if (refs < 0)
__builtin_trap();
#endif
}
static inline __attribute__((always_inline)) bool
decref(size_t* r) noexcept
{
if (!__atomic_fetch_sub(r, 1, __ATOMIC_RELEASE)) {
__atomic_thread_fence(__ATOMIC_ACQUIRE);
return true;
}
return false;
}
class shared_ref
{
public:
constexpr shared_ref() noexcept = default;
shared_ref(const shared_ref&) = delete;
shared_ref& operator=(const shared_ref&) = delete;
virtual ~shared_ref() = default;
void keep_shared() noexcept
{
incref(&shared);
}
void drop_shared() noexcept
{
if (decref(&shared)) {
dispose();
drop_weak();
}
}
void keep_weak() noexcept
{
incref(&weak);
}
void drop_weak() noexcept
{
if (decref(&weak)) {
delete this;
}
}
size_t use_count() const noexcept
{
return __atomic_load_n(&shared, __ATOMIC_RELAXED) + 1;
}
size_t weak_count() const noexcept
{
return __atomic_load_n(&weak, __ATOMIC_RELAXED);
}
private:
virtual void dispose() noexcept = 0;
size_t shared = 0;
size_t weak = 0;
};
template<typename T, typename D>
class shared_pointer : public shared_ref
{
public:
static shared_pointer* make(T* const p, D d)
{
return make(unique_ptr<T, D>(p, move(d)));
}
static shared_pointer* make(unique_ptr<T, D> p)
{
return new shared_pointer(p.release(), move(p.get_deleter()));
}
private:
shared_pointer(T* const p, D d) noexcept : p(p), d(move(d))
{
}
void dispose() noexcept override
{
move(d)(p);
}
T* const p;
[[no_unique_address]] D d;
};
template<typename T>
class shared_emplace : public shared_ref
{
public:
union
{
T t;
};
~shared_emplace() override
{
}
template<typename... Args>
void construct(Args&&... args)
{
::new (&t) T(forward<Args>(args)...);
}
static unique_ptr<shared_emplace> make()
{
return unique_ptr(new shared_emplace());
}
private:
explicit constexpr shared_emplace() noexcept
{
}
void dispose() noexcept override
{
t.~T();
}
};
template<typename T, typename U>
concept shared_ptr_compatible = is_convertible_v<U*, T*>;
} // namespace __
template<typename T>
class weak_ptr;
template<typename T>
class shared_ptr
{
public:
using element_type = remove_extent_t<T>;
using weak_type = weak_ptr<T>;
constexpr shared_ptr() noexcept = default;
constexpr shared_ptr(nullptr_t) noexcept
{
}
template<typename U>
requires __::shared_ptr_compatible<T, U>
explicit shared_ptr(U* const p) : shared_ptr(p, default_delete<U>())
{
}
template<typename U, typename D>
requires __::shared_ptr_compatible<T, U>
shared_ptr(U*, D);
template<typename U>
shared_ptr(const shared_ptr<U>& r, element_type* p) noexcept
: p(p), rc(r.rc)
{
if (rc)
rc->keep_shared();
}
template<typename U>
shared_ptr(shared_ptr<U>&& r, element_type* p) noexcept : p(p), rc(r.rc)
{
r.p = nullptr;
r.rc = nullptr;
}
template<typename U>
requires __::shared_ptr_compatible<T, U>
shared_ptr(const shared_ptr<U>& r) noexcept : p(r.p), rc(r.rc)
{
if (rc)
rc->keep_shared();
}
template<typename U>
requires __::shared_ptr_compatible<T, U>
shared_ptr(shared_ptr<U>&& r) noexcept : p(r.p), rc(r.rc)
{
r.p = nullptr;
r.rc = nullptr;
}
shared_ptr(const shared_ptr& r) noexcept : p(r.p), rc(r.rc)
{
if (rc)
rc->keep_shared();
}
shared_ptr(shared_ptr&& r) noexcept : p(r.p), rc(r.rc)
{
r.p = nullptr;
r.rc = nullptr;
}
template<typename U>
requires __::shared_ptr_compatible<T, U>
explicit shared_ptr(const weak_ptr<U>& r) : p(r.p), rc(r.rc)
{
if (r.expired()) {
throw bad_weak_ptr();
}
rc->keep_shared();
}
template<typename U, typename D>
requires __::shared_ptr_compatible<T, U>
shared_ptr(unique_ptr<U, D>&& r)
: p(r.p), rc(__::shared_pointer<U, D>::make(move(r)))
{
}
~shared_ptr()
{
if (rc)
rc->drop_shared();
}
shared_ptr& operator=(shared_ptr r) noexcept
{
swap(r);
return *this;
}
template<typename U>
requires __::shared_ptr_compatible<T, U>
shared_ptr& operator=(shared_ptr<U> r) noexcept
{
shared_ptr<T>(move(r)).swap(*this);
return *this;
}
void reset() noexcept
{
shared_ptr().swap(*this);
}
template<typename U>
requires __::shared_ptr_compatible<T, U>
void reset(U* const p2)
{
shared_ptr<T>(p2).swap(*this);
}
template<typename U, typename D>
requires __::shared_ptr_compatible<T, U>
void reset(U* const p2, D d)
{
shared_ptr<T>(p2, d).swap(*this);
}
void swap(shared_ptr& r) noexcept
{
using ctl::swap;
swap(p, r.p);
swap(rc, r.rc);
}
element_type* get() const noexcept
{
return p;
}
typename __::ptr_ref<T>::type operator*() const noexcept
{
if (!p)
__builtin_trap();
return *p;
}
T* operator->() const noexcept
{
if (!p)
__builtin_trap();
return p;
}
long use_count() const noexcept
{
return rc ? rc->use_count() : 0;
}
explicit operator bool() const noexcept
{
return p;
}
template<typename U>
bool owner_before(const shared_ptr<U>& r) const noexcept
{
return rc < r.rc;
}
template<typename U>
bool owner_before(const weak_ptr<U>& r) const noexcept
{
return rc < r.rc;
}
private:
template<typename U>
friend class weak_ptr;
template<typename U>
friend class shared_ptr;
template<typename U, typename... Args>
friend shared_ptr<U> make_shared(Args&&... args);
element_type* p = nullptr;
__::shared_ref* rc = nullptr;
};
template<typename T>
class weak_ptr
{
public:
using element_type = remove_extent_t<T>;
constexpr weak_ptr() noexcept = default;
template<typename U>
requires __::shared_ptr_compatible<T, U>
weak_ptr(const shared_ptr<U>& r) noexcept : p(r.p), rc(r.rc)
{
if (rc)
rc->keep_weak();
}
weak_ptr(const weak_ptr& r) noexcept : p(r.p), rc(r.rc)
{
if (rc)
rc->keep_weak();
}
template<typename U>
requires __::shared_ptr_compatible<T, U>
weak_ptr(const weak_ptr<U>& r) noexcept : p(r.p), rc(r.rc)
{
if (rc)
rc->keep_weak();
}
weak_ptr(weak_ptr&& r) noexcept : p(r.p), rc(r.rc)
{
r.p = nullptr;
r.rc = nullptr;
}
template<typename U>
requires __::shared_ptr_compatible<T, U>
weak_ptr(weak_ptr<U>&& r) noexcept : p(r.p), rc(r.rc)
{
r.p = nullptr;
r.rc = nullptr;
}
~weak_ptr()
{
if (rc)
rc->drop_weak();
}
long use_count() const noexcept
{
return rc ? rc->use_count() : 0;
}
bool expired() const noexcept
{
return !use_count();
}
void reset() noexcept
{
weak_ptr().swap(*this);
}
void swap(weak_ptr& r) noexcept
{
using ctl::swap;
swap(p, r.p);
swap(rc, r.rc);
}
weak_ptr& operator=(weak_ptr r) noexcept
{
swap(r);
return *this;
}
template<typename U>
requires __::shared_ptr_compatible<T, U>
weak_ptr& operator=(weak_ptr<U> r) noexcept
{
weak_ptr<T>(move(r)).swap(*this);
}
shared_ptr<T> lock() const noexcept
{
if (expired())
return nullptr;
shared_ptr<T> r;
r.p = p;
r.rc = rc;
if (rc)
rc->keep_shared();
return r;
}
template<typename U>
bool owner_before(const weak_ptr<U>& r) const noexcept
{
return rc < r.rc;
}
template<typename U>
bool owner_before(const shared_ptr<U>& r) const noexcept
{
return rc < r.rc;
}
private:
template<typename U>
friend class shared_ptr;
template<typename U, typename... Args>
friend shared_ptr<U> make_shared(Args&&...);
element_type* p = nullptr;
__::shared_ref* rc = nullptr;
};
template<typename T>
class enable_shared_from_this
{
public:
shared_ptr<T> shared_from_this()
{
return shared_ptr<T>(weak_this);
}
shared_ptr<T const> shared_from_this() const
{
return shared_ptr<T>(weak_this);
}
weak_ptr<T> weak_from_this()
{
return weak_this;
}
weak_ptr<T const> weak_from_this() const
{
return weak_this;
}
protected:
constexpr enable_shared_from_this() noexcept = default;
enable_shared_from_this(const enable_shared_from_this& r) noexcept
{
}
~enable_shared_from_this() = default;
enable_shared_from_this& operator=(
const enable_shared_from_this& r) noexcept
{
return *this;
}
private:
template<typename U, typename... Args>
friend shared_ptr<U> make_shared(Args&&...);
template<typename U>
friend class shared_ptr;
weak_ptr<T> weak_this;
};
template<typename T>
template<typename U, typename D>
requires __::shared_ptr_compatible<T, U>
shared_ptr<T>::shared_ptr(U* const p, D d)
: p(p), rc(__::shared_pointer<U, D>::make(p, move(d)))
{
if constexpr (is_base_of_v<enable_shared_from_this<U>, U>) {
p->weak_this = *this;
}
}
// Our make_shared supports passing a weak self reference as the first parameter
// to your constructor, e.g.:
//
// struct Tree : ctl::weak_self_base
// {
// ctl::shared_ptr<Tree> l, r;
// ctl::weak_ptr<Tree> parent;
// Tree(weak_ptr<Tree> const& self, auto&& l2, auto&& r2)
// : l(ctl::forward<decltype(l2)>(l2)),
// r(ctl::forward<decltype(r2)>(r2))
// {
// if (l) l->parent = self;
// if (r) r->parent = self;
// }
// };
//
// int main() {
// auto t = ctl::make_shared<Tree>(
// ctl::make_shared<Tree>(nullptr, nullptr), nullptr);
// return t->l->parent.lock().get() == t.get() ? 0 : 1;
// }
//
// As shown, passing the parameter at object construction time lets you complete
// object construction without needing a separate Init method. But because we go
// off spec as far as the STL is concerned, there is a potential ambiguity where
// you might have a constructor with a weak_ptr first parameter that is intended
// to be something other than a self-reference. So this feature is opt-in by way
// of inheriting from the following struct.
struct weak_self_base
{};
template<typename T, typename... Args>
shared_ptr<T>
make_shared(Args&&... args)
{
unique_ptr rc = __::shared_emplace<T>::make();
if constexpr (is_base_of_v<weak_self_base, T> &&
is_constructible_v<T, const weak_ptr<T>&, Args...>) {
// A __::shared_ref has a virtual weak reference that is owned by all of
// the shared references. We can avoid some unnecessary refcount changes
// by "borrowing" that reference and passing it to the constructor, then
// promoting it to a shared reference by swapping it with the shared_ptr
// that we return.
weak_ptr<T> w;
w.p = &rc->t;
w.rc = rc.get();
try {
rc->construct(const_cast<const weak_ptr<T>&>(w),
forward<Args>(args)...);
} catch (...) {
w.p = nullptr;
w.rc = nullptr;
throw;
}
rc.release();
shared_ptr<T> r;
swap(r.p, w.p);
swap(r.rc, w.rc);
return r;
} else {
rc->construct(forward<Args>(args)...);
shared_ptr<T> r;
r.p = &rc->t;
r.rc = rc.release();
if constexpr (is_base_of_v<enable_shared_from_this<T>, T>) {
r->weak_this = r;
}
return r;
}
}
} // namespace ctl
#endif // CTL_SHARED_PTR_H_

View file

@ -383,4 +383,72 @@ string::erase(const size_t pos, size_t count) noexcept
return *this;
}
void
string::append(const ctl::string_view& s, size_t pos, size_t count) noexcept
{
append(s.substr(pos, count));
}
size_t
string::find_last_of(char c, size_t pos) const noexcept
{
const char* b = data();
size_t n = size();
if (pos > n)
pos = n;
const char* p = (const char*)memrchr(b, c, pos);
return p ? p - b : npos;
}
size_t
string::find_last_of(ctl::string_view set, size_t pos) const noexcept
{
if (empty() || set.empty())
return npos;
bool lut[256] = {};
for (char c : set)
lut[c & 255] = true;
const char* b = data();
size_t last = size() - 1;
if (pos > last)
pos = last;
for (;;) {
if (lut[b[pos] & 255])
return pos;
if (!pos)
return npos;
--pos;
}
}
size_t
string::find_first_of(char c, size_t pos) const noexcept
{
size_t n = size();
if (pos >= n)
return npos;
const char* b = data();
const char* p = (const char*)memchr(b + pos, c, n - pos);
return p ? p - b : npos;
}
size_t
string::find_first_of(ctl::string_view set, size_t pos) const noexcept
{
if (set.empty())
return npos;
bool lut[256] = {};
for (char c : set)
lut[c & 255] = true;
const char* b = data();
size_t n = size();
for (;;) {
if (pos >= n)
return npos;
if (lut[b[pos] & 255])
return pos;
++pos;
}
}
} // namespace ctl

View file

@ -125,6 +125,7 @@ class string
void append(char, size_t) noexcept;
void append(unsigned long) noexcept;
void append(const void*, size_t) noexcept;
void append(const ctl::string_view&, size_t, size_t = npos) noexcept;
string& insert(size_t, ctl::string_view) noexcept;
string& erase(size_t = 0, size_t = npos) noexcept;
string substr(size_t = 0, size_t = npos) const noexcept;
@ -136,6 +137,10 @@ class string
bool starts_with(ctl::string_view) const noexcept;
size_t find(char, size_t = 0) const noexcept;
size_t find(ctl::string_view, size_t = 0) const noexcept;
size_t find_first_of(char, size_t = 0) const noexcept;
size_t find_first_of(ctl::string_view, size_t = 0) const noexcept;
size_t find_last_of(char, size_t = npos) const noexcept;
size_t find_last_of(ctl::string_view, size_t = npos) const noexcept;
void swap(string& s) noexcept
{
@ -302,7 +307,7 @@ class string
append(ch);
}
void append(const ctl::string_view s) noexcept
void append(const ctl::string_view& s) noexcept
{
append(s.p, s.n);
}

View file

@ -108,4 +108,66 @@ string_view::starts_with(const string_view s) const noexcept
return !memcmp(p, s.p, s.n);
}
size_t
string_view::find_last_of(char c, size_t pos) const noexcept
{
const char* b = data();
size_t n = size();
if (pos > n)
pos = n;
const char* p = (const char*)memrchr(b, c, pos);
return p ? p - b : npos;
}
size_t
string_view::find_last_of(ctl::string_view set, size_t pos) const noexcept
{
if (empty() || set.empty())
return npos;
bool lut[256] = {};
for (char c : set)
lut[c & 255] = true;
const char* b = data();
size_t last = size() - 1;
if (pos > last)
pos = last;
for (;;) {
if (lut[b[pos] & 255])
return pos;
if (!pos)
return npos;
--pos;
}
}
size_t
string_view::find_first_of(char c, size_t pos) const noexcept
{
size_t n = size();
if (pos >= n)
return npos;
const char* b = data();
const char* p = (const char*)memchr(b + pos, c, n - pos);
return p ? p - b : npos;
}
size_t
string_view::find_first_of(ctl::string_view set, size_t pos) const noexcept
{
if (set.empty())
return npos;
bool lut[256] = {};
for (char c : set)
lut[c & 255] = true;
const char* b = data();
size_t n = size();
for (;;) {
if (pos >= n)
return npos;
if (lut[b[pos] & 255])
return pos;
++pos;
}
}
} // namespace ctl

View file

@ -45,6 +45,10 @@ struct string_view
string_view substr(size_t = 0, size_t = npos) const noexcept;
size_t find(char, size_t = 0) const noexcept;
size_t find(string_view, size_t = 0) const noexcept;
size_t find_first_of(char, size_t = 0) const noexcept;
size_t find_first_of(ctl::string_view, size_t = 0) const noexcept;
size_t find_last_of(char, size_t = npos) const noexcept;
size_t find_last_of(ctl::string_view, size_t = npos) const noexcept;
constexpr string_view& operator=(const string_view s) noexcept
{
@ -109,12 +113,12 @@ struct string_view
return p[n - 1];
}
constexpr const_iterator begin() noexcept
constexpr const_iterator begin() const noexcept
{
return p;
}
constexpr const_iterator end() noexcept
constexpr const_iterator end() const noexcept
{
return p + n;
}

View file

@ -2,7 +2,9 @@
#── vi: set noet ft=make ts=8 sw=8 fenc=utf-8 :vi ────────────────────┘
.PHONY: o/$(MODE)/dsp
o/$(MODE)/dsp: o/$(MODE)/dsp/core \
o/$(MODE)/dsp: o/$(MODE)/dsp/audio \
o/$(MODE)/dsp/core \
o/$(MODE)/dsp/mpeg \
o/$(MODE)/dsp/scale \
o/$(MODE)/dsp/prog \
o/$(MODE)/dsp/tty

56
dsp/audio/BUILD.mk Normal file
View file

@ -0,0 +1,56 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#── vi: set noet ft=make ts=8 sw=8 fenc=utf-8 :vi ────────────────────┘
PKGS += DSP_AUDIO
DSP_AUDIO_ARTIFACTS += DSP_AUDIO_A
DSP_AUDIO = $(DSP_AUDIO_A_DEPS) $(DSP_AUDIO_A)
DSP_AUDIO_A = o/$(MODE)/dsp/audio/audio.a
DSP_AUDIO_A_FILES := $(wildcard dsp/audio/*)
DSP_AUDIO_A_HDRS = $(filter %.h,$(DSP_AUDIO_A_FILES)) dsp/audio/cosmoaudio/cosmoaudio.h
DSP_AUDIO_A_SRCS = $(filter %.c,$(DSP_AUDIO_A_FILES))
DSP_AUDIO_A_DATA = \
dsp/audio/cosmoaudio/miniaudio.h \
dsp/audio/cosmoaudio/cosmoaudio.c \
dsp/audio/cosmoaudio/cosmoaudio.h \
dsp/audio/cosmoaudio/cosmoaudio.dll \
DSP_AUDIO_A_OBJS = \
$(DSP_AUDIO_A_SRCS:%.c=o/$(MODE)/%.o) \
$(DSP_AUDIO_A_DATA:%=o/$(MODE)/%.zip.o) \
DSP_AUDIO_A_CHECKS = \
$(DSP_AUDIO_A).pkg \
$(DSP_AUDIO_A_HDRS:%=o/$(MODE)/%.ok)
DSP_AUDIO_A_DIRECTDEPS = \
LIBC_CALLS \
LIBC_DLOPEN \
LIBC_INTRIN \
LIBC_NEXGEN32E \
LIBC_STR \
LIBC_SYSV \
LIBC_PROC \
LIBC_THREAD \
DSP_AUDIO_A_DEPS := \
$(call uniq,$(foreach x,$(DSP_AUDIO_A_DIRECTDEPS),$($(x))))
$(DSP_AUDIO_A): dsp/audio/ \
$(DSP_AUDIO_A).pkg \
$(DSP_AUDIO_A_OBJS)
$(DSP_AUDIO_A).pkg: \
$(DSP_AUDIO_A_OBJS) \
$(foreach x,$(DSP_AUDIO_A_DIRECTDEPS),$($(x)_A).pkg)
DSP_AUDIO_LIBS = $(foreach x,$(DSP_AUDIO_ARTIFACTS),$($(x)))
DSP_AUDIO_SRCS = $(foreach x,$(DSP_AUDIO_ARTIFACTS),$($(x)_SRCS))
DSP_AUDIO_HDRS = $(foreach x,$(DSP_AUDIO_ARTIFACTS),$($(x)_HDRS))
DSP_AUDIO_CHECKS = $(foreach x,$(DSP_AUDIO_ARTIFACTS),$($(x)_CHECKS))
DSP_AUDIO_OBJS = $(foreach x,$(DSP_AUDIO_ARTIFACTS),$($(x)_OBJS))
$(DSP_AUDIO_OBJS): $(BUILD_FILES) dsp/audio/BUILD.mk
.PHONY: o/$(MODE)/dsp/audio
o/$(MODE)/dsp/audio: $(DSP_AUDIO_CHECKS)

358
dsp/audio/audio.c Normal file
View file

@ -0,0 +1,358 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
vi: set et ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi
Copyright 2024 Justine Alexandra Roberts Tunney
Permission to use, copy, modify, and/or distribute this software for
any purpose with or without fee is hereby granted, provided that the
above copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE
AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL
DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR
PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE.
*/
#include "dsp/audio/cosmoaudio/cosmoaudio.h"
#include "dsp/audio/describe.h"
#include "libc/calls/blockcancel.internal.h"
#include "libc/calls/calls.h"
#include "libc/calls/struct/sigset.internal.h"
#include "libc/calls/struct/stat.h"
#include "libc/calls/struct/timespec.h"
#include "libc/dce.h"
#include "libc/dlopen/dlfcn.h"
#include "libc/errno.h"
#include "libc/intrin/describeflags.h"
#include "libc/intrin/strace.h"
#include "libc/limits.h"
#include "libc/macros.h"
#include "libc/proc/posix_spawn.h"
#include "libc/runtime/runtime.h"
#include "libc/str/str.h"
#include "libc/sysv/consts/o.h"
#include "libc/temp.h"
#include "libc/thread/thread.h"
#define COSMOAUDIO_MINIMUM_VERISON 1
#define COSMOAUDIO_DSO_NAME "cosmoaudio." STRINGIFY(COSMOAUDIO_MINIMUM_VERISON)
__static_yoink("dsp/audio/cosmoaudio/miniaudio.h");
__static_yoink("dsp/audio/cosmoaudio/cosmoaudio.h");
__static_yoink("dsp/audio/cosmoaudio/cosmoaudio.c");
__static_yoink("dsp/audio/cosmoaudio/cosmoaudio.dll");
static const struct Source {
const char *zip;
const char *name;
} srcs[] = {
{"/zip/dsp/audio/cosmoaudio/miniaudio.h", "miniaudio.h"},
{"/zip/dsp/audio/cosmoaudio/cosmoaudio.h", "cosmoaudio.h"},
{"/zip/dsp/audio/cosmoaudio/cosmoaudio.c", "cosmoaudio.c"}, // must last
};
static struct {
pthread_once_t once;
typeof(cosmoaudio_open) *open;
typeof(cosmoaudio_close) *close;
typeof(cosmoaudio_write) *write;
typeof(cosmoaudio_flush) *flush;
typeof(cosmoaudio_read) *read;
typeof(cosmoaudio_poll) *poll;
} g_audio;
static const char *cosmoaudio_tmp_dir(void) {
const char *tmpdir;
if (!(tmpdir = getenv("TMPDIR")) || !*tmpdir)
if (!(tmpdir = getenv("HOME")) || !*tmpdir)
tmpdir = ".";
return tmpdir;
}
static bool cosmoaudio_app_dir(char *path, size_t size) {
strlcpy(path, cosmoaudio_tmp_dir(), size);
strlcat(path, "/.cosmo/", size);
if (makedirs(path, 0755))
return false;
return true;
}
static bool cosmoaudio_dso_path(char *path, size_t size) {
if (!cosmoaudio_app_dir(path, size))
return false;
strlcat(path, COSMOAUDIO_DSO_NAME, size);
if (IsWindows()) {
strlcat(path, ".dll", size);
} else if (IsXnu()) {
strlcat(path, ".dylib", size);
} else {
strlcat(path, ".so", size);
}
return true;
}
static bool cosmoaudio_extract(const char *zip, const char *to) {
int fdin, fdout;
char stage[PATH_MAX];
strlcpy(stage, to, sizeof(stage));
if (strlcat(stage, ".XXXXXX", sizeof(stage)) >= sizeof(stage)) {
errno = ENAMETOOLONG;
return false;
}
if ((fdout = mkostemp(stage, O_CLOEXEC)) == -1)
return false;
if ((fdin = open(zip, O_RDONLY | O_CLOEXEC)) == -1) {
close(fdout);
unlink(stage);
return false;
}
if (copyfd(fdin, fdout, -1) == -1) {
close(fdin);
close(fdout);
unlink(stage);
return false;
}
if (close(fdout)) {
close(fdin);
unlink(stage);
return false;
}
if (close(fdin)) {
unlink(stage);
return false;
}
if (rename(stage, to)) {
unlink(stage);
return false;
}
return true;
}
static bool cosmoaudio_build(const char *dso) {
// extract sauce
char src[PATH_MAX];
for (int i = 0; i < sizeof(srcs) / sizeof(*srcs); ++i) {
if (!cosmoaudio_app_dir(src, PATH_MAX))
return false;
strlcat(src, srcs[i].name, sizeof(src));
if (!cosmoaudio_extract(srcs[i].zip, src))
return false;
}
// create temporary name for compiled dso
// it'll ensure build operation is atomic
int fd;
char tmpdso[PATH_MAX];
strlcpy(tmpdso, dso, sizeof(tmpdso));
strlcat(tmpdso, ".XXXXXX", sizeof(tmpdso));
if ((fd = mkostemp(tmpdso, O_CLOEXEC)) != -1) {
close(fd);
} else {
return false;
}
// build cosmoaudio with host c compiler
char *args[] = {
"cc", //
"-w", //
"-I.", //
"-O2", //
"-fPIC", //
"-shared", //
"-pthread", //
"-DNDEBUG", //
IsAarch64() ? "-ffixed-x28" : "-DIGNORE1", //
src, //
"-o", //
tmpdso, //
"-lm", //
IsNetbsd() ? 0 : "-ldl", //
NULL,
};
int pid, ws;
errno_t err = posix_spawnp(&pid, args[0], NULL, NULL, args, environ);
if (err)
return false;
while (waitpid(pid, &ws, 0) == -1)
if (errno != EINTR)
return false;
if (ws)
return false;
// move dso to its final destination
if (rename(tmpdso, dso))
return false;
return true;
}
static void *cosmoaudio_dlopen(const char *name) {
void *handle;
if ((handle = cosmo_dlopen(name, RTLD_NOW))) {
typeof(cosmoaudio_version) *version;
if ((version = cosmo_dlsym(handle, "cosmoaudio_version")))
if (version() >= COSMOAUDIO_MINIMUM_VERISON)
return handle;
cosmo_dlclose(handle);
}
return 0;
}
static void cosmoaudio_setup_impl(void) {
void *handle;
if (IsOpenbsd())
return; // no dlopen support yet
if (IsXnu() && !IsXnuSilicon())
return; // no dlopen support yet
if (!(handle = cosmoaudio_dlopen(COSMOAUDIO_DSO_NAME ".so")) &&
!(handle = cosmoaudio_dlopen("lib" COSMOAUDIO_DSO_NAME ".so")) &&
!(handle = cosmoaudio_dlopen("cosmoaudio.so")) &&
!(handle = cosmoaudio_dlopen("libcosmoaudio.so"))) {
char dso[PATH_MAX];
if (!cosmoaudio_dso_path(dso, sizeof(dso)))
return;
if ((handle = cosmoaudio_dlopen(dso)))
goto WeAreGood;
if (IsWindows()) {
if (cosmoaudio_extract("/zip/dsp/audio/cosmoaudio/cosmoaudio.dll", dso)) {
if ((handle = cosmoaudio_dlopen(dso))) {
goto WeAreGood;
} else {
return;
}
}
}
if (!cosmoaudio_build(dso))
return;
if (!(handle = cosmoaudio_dlopen(dso)))
return;
}
WeAreGood:
g_audio.open = cosmo_dlsym(handle, "cosmoaudio_open");
g_audio.close = cosmo_dlsym(handle, "cosmoaudio_close");
g_audio.write = cosmo_dlsym(handle, "cosmoaudio_write");
g_audio.flush = cosmo_dlsym(handle, "cosmoaudio_flush");
g_audio.read = cosmo_dlsym(handle, "cosmoaudio_read");
g_audio.poll = cosmo_dlsym(handle, "cosmoaudio_poll");
}
static void cosmoaudio_setup(void) {
BLOCK_CANCELATION;
cosmoaudio_setup_impl();
ALLOW_CANCELATION;
}
static void cosmoaudio_init(void) {
pthread_once(&g_audio.once, cosmoaudio_setup);
}
COSMOAUDIO_ABI int cosmoaudio_open(
struct CosmoAudio **out_ca, const struct CosmoAudioOpenOptions *options) {
int status;
char sbuf[32];
char dbuf[256];
cosmoaudio_init();
if (g_audio.open) {
BLOCK_SIGNALS;
status = g_audio.open(out_ca, options);
ALLOW_SIGNALS;
} else {
status = COSMOAUDIO_ELINK;
}
STRACE("cosmoaudio_open([%p], %s) → %s",
out_ca ? *out_ca : (struct CosmoAudio *)-1,
cosmoaudio_describe_open_options(dbuf, sizeof(dbuf), options),
cosmoaudio_describe_status(sbuf, sizeof(sbuf), status));
return status;
}
COSMOAUDIO_ABI int cosmoaudio_close(struct CosmoAudio *ca) {
int status;
char sbuf[32];
if (g_audio.close) {
BLOCK_SIGNALS;
status = g_audio.close(ca);
ALLOW_SIGNALS;
} else {
status = COSMOAUDIO_ELINK;
}
STRACE("cosmoaudio_close(%p) → %s", ca,
cosmoaudio_describe_status(sbuf, sizeof(sbuf), status));
return status;
}
COSMOAUDIO_ABI int cosmoaudio_write(struct CosmoAudio *ca, const float *data,
int frames) {
int status;
char sbuf[32];
if (g_audio.write) {
BLOCK_SIGNALS;
status = g_audio.write(ca, data, frames);
ALLOW_SIGNALS;
} else {
status = COSMOAUDIO_ELINK;
}
if (frames <= 0 || frames >= 160)
DATATRACE("cosmoaudio_write(%p, %p, %d) → %s", ca, data, frames,
cosmoaudio_describe_status(sbuf, sizeof(sbuf), status));
return status;
}
COSMOAUDIO_ABI int cosmoaudio_read(struct CosmoAudio *ca, float *data,
int frames) {
int status;
char sbuf[32];
if (g_audio.read) {
BLOCK_SIGNALS;
status = g_audio.read(ca, data, frames);
ALLOW_SIGNALS;
} else {
status = COSMOAUDIO_ELINK;
}
if (frames <= 0 || frames >= 160)
DATATRACE("cosmoaudio_read(%p, %p, %d) → %s", ca, data, frames,
cosmoaudio_describe_status(sbuf, sizeof(sbuf), status));
return status;
}
COSMOAUDIO_ABI int cosmoaudio_flush(struct CosmoAudio *ca) {
int status;
char sbuf[32];
if (g_audio.flush) {
BLOCK_SIGNALS;
status = g_audio.flush(ca);
ALLOW_SIGNALS;
} else {
status = COSMOAUDIO_ELINK;
}
DATATRACE("cosmoaudio_flush(%p) → %s", ca,
cosmoaudio_describe_status(sbuf, sizeof(sbuf), status));
return status;
}
COSMOAUDIO_ABI int cosmoaudio_poll(struct CosmoAudio *ca,
int *in_out_readFrames,
int *in_out_writeFrames) {
int status;
char sbuf[32];
char fbuf[2][20];
if (g_audio.poll) {
BLOCK_SIGNALS;
status = g_audio.poll(ca, in_out_readFrames, in_out_writeFrames);
ALLOW_SIGNALS;
} else {
status = COSMOAUDIO_ELINK;
}
DATATRACE("cosmoaudio_poll(%p, %s, %s) → %s", ca,
cosmoaudio_describe_poll_frames(fbuf[0], sizeof(fbuf[0]),
in_out_readFrames),
cosmoaudio_describe_poll_frames(fbuf[1], sizeof(fbuf[1]),
in_out_writeFrames),
cosmoaudio_describe_status(sbuf, sizeof(sbuf), status));
return status;
}

3
dsp/audio/cosmoaudio/.gitignore vendored Normal file
View file

@ -0,0 +1,3 @@
*.o
/Debug
/Release

View file

@ -0,0 +1,87 @@
# Makefile for MSVC x64 Command Line Developer Tools
#
# nmake /f Makefile.msvc check
# nmake /f Makefile.msvc MODE=debug check
#
# Note: MSVC 2019 makes the DLL 64kb smaller than MSVC 2022.
# Compiler and linker
CC=cl
LINK=link
# Build mode (can be overridden from command line)
!IFNDEF MODE
MODE=release
!ENDIF
# Library dependencies.
TEST_LIBS=OneCore.lib
# Compiler flags
CFLAGS_COMMON=/nologo /W4 /Gy /EHsc
CFLAGS_DEBUG=/Od /Zi /MDd /D_DEBUG
CFLAGS_RELEASE=/O2 /MT /DNDEBUG
!IF "$(MODE)"=="debug"
CFLAGS=$(CFLAGS_COMMON) $(CFLAGS_DEBUG)
LDFLAGS=/DEBUG
OUT_DIR=Debug
!ELSE
CFLAGS=$(CFLAGS_COMMON) $(CFLAGS_RELEASE) /GL
LDFLAGS=/RELEASE /OPT:REF /OPT:ICF /LTCG /INCREMENTAL:NO
OUT_DIR=Release
!ENDIF
# Additional flags for DLL
DLL_CFLAGS=$(CFLAGS) /D_USRDLL /D_WINDLL
# Linker flags
LDFLAGS=/NOLOGO /SUBSYSTEM:CONSOLE $(LDFLAGS)
# Output file names
DLL_TARGET=$(OUT_DIR)\cosmoaudio.dll
TEST_TARGET=$(OUT_DIR)\test.exe
# Source files
DLL_SOURCES=cosmoaudio.c
TEST_SOURCES=test.c
# Object files
DLL_OBJECTS=$(OUT_DIR)\cosmoaudio.obj
TEST_OBJECTS=$(OUT_DIR)\test.obj
# Default target
all: $(OUT_DIR) $(DLL_TARGET) $(TEST_TARGET)
# Create output directory
$(OUT_DIR):
if not exist $(OUT_DIR) mkdir $(OUT_DIR)
# Rule to build the DLL
$(DLL_TARGET): $(OUT_DIR) $(DLL_OBJECTS)
$(LINK) /DLL $(LDFLAGS) /OUT:$(DLL_TARGET) $(DLL_OBJECTS)
# Rule to build the test program
$(TEST_TARGET): $(OUT_DIR) $(TEST_OBJECTS) $(DLL_TARGET)
$(LINK) $(LDFLAGS) /OUT:$(TEST_TARGET) $(TEST_OBJECTS) $(DLL_TARGET:.dll=.lib) $(TEST_LIBS)
# Rules to compile .c files to .obj files with header dependencies
{.}.c{$(OUT_DIR)}.obj:
$(CC) $(DLL_CFLAGS) /c /Fo$(OUT_DIR)\ $<
$(OUT_DIR)\test.obj: $(OUT_DIR) test.c cosmoaudio.h
$(CC) $(CFLAGS) /c /Fo$(OUT_DIR)\ test.c
$(OUT_DIR)\cosmoaudio.obj: $(OUT_DIR) cosmoaudio.c miniaudio.h cosmoaudio.h
$(CC) $(DLL_CFLAGS) /c /Fo$(OUT_DIR)\ cosmoaudio.c
# Clean target
clean:
if exist $(OUT_DIR) rmdir /s /q $(OUT_DIR)
# Run tests (now called 'check')
check: $(TEST_TARGET)
$(TEST_TARGET)
# Phony targets
.PHONY: all clean check

View file

@ -0,0 +1,519 @@
// Copyright 2024 Justine Alexandra Roberts Tunney
//
// Permission to use, copy, modify, and/or distribute this software for
// any purpose with or without fee is hereby granted, provided that the
// above copyright notice and this permission notice appear in all copies.
//
// THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL
// WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
// WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE
// AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL
// DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR
// PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
// TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
// PERFORMANCE OF THIS SOFTWARE.
#define COSMOAUDIO_BUILD
#include "cosmoaudio.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define MA_DEBUG_OUTPUT
#define MA_DR_MP3_NO_STDIO
#define MA_NO_DECODING
#define MA_NO_ENCODING
#define MA_NO_ENGINE
#define MA_NO_GENERATION
#define MA_NO_NODE_GRAPH
#define MA_NO_RESOURCE_MANAGER
#define MA_STATIC
#define MINIAUDIO_IMPLEMENTATION
#include "miniaudio.h"
struct CosmoAudio {
enum CosmoAudioDeviceType deviceType;
ma_uint32 outputBufferFrames;
ma_uint32 inputBufferFrames;
int sampleRate;
int channels;
int isLeft;
ma_context context;
ma_device device;
ma_pcm_rb output;
ma_pcm_rb input;
ma_event event;
ma_log log;
};
static int read_ring_buffer(ma_log* log, ma_pcm_rb* rb, float* pOutput,
ma_uint32 frameCount, ma_uint32 channels) {
ma_result result;
ma_uint32 framesRead;
ma_uint32 framesToRead;
for (framesRead = 0; framesRead < frameCount; framesRead += framesToRead) {
framesToRead = frameCount - framesRead;
void* pMappedBuffer;
result = ma_pcm_rb_acquire_read(rb, &framesToRead, &pMappedBuffer);
if (result != MA_SUCCESS) {
ma_log_postf(log, MA_LOG_LEVEL_WARNING,
"ma_pcm_rb_acquire_read failed: %s\n",
ma_result_description(result));
return COSMOAUDIO_ERROR;
}
if (!framesToRead)
break;
memcpy(pOutput + framesRead * channels, pMappedBuffer,
framesToRead * channels * sizeof(float));
result = ma_pcm_rb_commit_read(rb, framesToRead);
if (result != MA_SUCCESS) {
if (result == MA_AT_END) {
framesRead += framesToRead;
break;
}
ma_log_postf(log, MA_LOG_LEVEL_WARNING,
"ma_pcm_rb_commit_read failed: %s\n",
ma_result_description(result));
return COSMOAUDIO_ERROR;
}
}
return framesRead;
}
static int write_ring_buffer(ma_log* log, ma_pcm_rb* rb, const float* pInput,
ma_uint32 frameCount, ma_uint32 channels) {
ma_result result;
ma_uint32 framesWritten;
ma_uint32 framesToWrite;
for (framesWritten = 0; framesWritten < frameCount;
framesWritten += framesToWrite) {
framesToWrite = frameCount - framesWritten;
void* pMappedBuffer;
result = ma_pcm_rb_acquire_write(rb, &framesToWrite, &pMappedBuffer);
if (result != MA_SUCCESS) {
ma_log_postf(log, MA_LOG_LEVEL_WARNING,
"ma_pcm_rb_acquire_write failed: %s\n",
ma_result_description(result));
return COSMOAUDIO_ERROR;
}
if (!framesToWrite)
break;
memcpy(pMappedBuffer, pInput + framesWritten * channels,
framesToWrite * channels * sizeof(float));
result = ma_pcm_rb_commit_write(rb, framesToWrite);
if (result != MA_SUCCESS) {
if (result == MA_AT_END) {
framesWritten += framesToWrite;
break;
}
ma_log_postf(log, MA_LOG_LEVEL_WARNING,
"ma_pcm_rb_commit_write failed: %s\n",
ma_result_description(result));
return COSMOAUDIO_ERROR;
}
}
return framesWritten;
}
static void data_callback_f32(ma_device* pDevice, float* pOutput,
const float* pInput, ma_uint32 frameCount) {
struct CosmoAudio* ca = (struct CosmoAudio*)pDevice->pUserData;
if (ca->deviceType & kCosmoAudioDeviceTypePlayback) {
//
// "By default, miniaudio will pre-silence the data callback's
// output buffer. If you know that you will always write valid data
// to the output buffer you can disable pre-silencing by setting
// the noPreSilence config option in the device config to true."
//
// —Quoth miniaudio documentation § 16.1. Low Level API
//
if (ca->isLeft) {
int framesCopied = read_ring_buffer(&ca->log, &ca->output, pOutput,
frameCount, ca->channels);
if (framesCopied < (int)frameCount)
ca->isLeft = 0;
} else {
// TODO(jart): Maybe we should stretch the audio too short?
int frameOffset;
int availableFrames = ma_pcm_rb_available_read(&ca->output);
if (availableFrames >= (int)frameCount) {
frameOffset = 0;
} else {
frameOffset = frameCount - availableFrames;
frameCount = availableFrames;
}
read_ring_buffer(&ca->log, &ca->output,
pOutput + frameOffset * ca->channels, frameCount,
ca->channels);
ca->isLeft = 1;
}
}
if (ca->deviceType & kCosmoAudioDeviceTypeCapture)
write_ring_buffer(&ca->log, &ca->input, pInput, frameCount, ca->channels);
ma_event_signal(&ca->event);
}
static void data_callback(ma_device* pDevice, void* pOutput, const void* pInput,
ma_uint32 frameCount) {
data_callback_f32(pDevice, (float*)pOutput, (const float*)pInput, frameCount);
}
/**
* Returns current version of cosmo audio library.
*/
COSMOAUDIO_ABI int cosmoaudio_version(void) {
return 1;
}
/**
* Opens access to speaker and microphone.
*
* @param out_ca will receive pointer to allocated CosmoAudio object,
* which must be freed by caller with cosmoaudio_close(); if this
* function fails, then this will receive a NULL pointer value so
* that cosmoaudio_close(), cosmoaudio_write() etc. can be called
* without crashing if no error checking is performed
* @return 0 on success, or negative error code on failure
*/
COSMOAUDIO_ABI int cosmoaudio_open( //
struct CosmoAudio** out_ca, //
const struct CosmoAudioOpenOptions* options) {
// Validate arguments.
if (!out_ca)
return COSMOAUDIO_EINVAL;
*out_ca = NULL;
if (!options)
return COSMOAUDIO_EINVAL;
if (options->sizeofThis < (int)sizeof(struct CosmoAudioOpenOptions))
return COSMOAUDIO_EINVAL;
if (options->bufferFrames < 0)
return COSMOAUDIO_EINVAL;
if (options->sampleRate < 8000)
return COSMOAUDIO_EINVAL;
if (options->channels < 1)
return COSMOAUDIO_EINVAL;
if (!options->deviceType)
return COSMOAUDIO_EINVAL;
if (options->deviceType &
~(kCosmoAudioDeviceTypePlayback | kCosmoAudioDeviceTypeCapture))
return COSMOAUDIO_EINVAL;
// Allocate cosmo audio object.
struct CosmoAudio* ca;
ca = (struct CosmoAudio*)calloc(1, sizeof(struct CosmoAudio));
if (!ca)
return COSMOAUDIO_ERROR;
ca->channels = options->channels;
ca->sampleRate = options->sampleRate;
ca->deviceType = options->deviceType;
// Create win32-style condition variable.
if (ma_event_init(&ca->event) != MA_SUCCESS) {
free(ca);
return COSMOAUDIO_ERROR;
}
// Create audio log.
if (ma_log_init(NULL, &ca->log) != MA_SUCCESS) {
ma_event_uninit(&ca->event);
free(ca);
return COSMOAUDIO_ERROR;
}
if (!options->debugLog)
ca->log.callbackCount = 0;
// Create audio context.
ma_context_config contextConfig = ma_context_config_init();
contextConfig.pLog = &ca->log;
if (ma_context_init(NULL, 0, &contextConfig, &ca->context) != MA_SUCCESS) {
ma_event_uninit(&ca->event);
ma_log_uninit(&ca->log);
free(ca);
return COSMOAUDIO_ERROR;
}
// Initialize device.
ma_result result;
ma_device_config deviceConfig;
deviceConfig = ma_device_config_init(ca->deviceType);
deviceConfig.sampleRate = ca->sampleRate;
if (ca->deviceType & kCosmoAudioDeviceTypeCapture) {
deviceConfig.capture.channels = ca->channels;
deviceConfig.capture.format = ma_format_f32;
deviceConfig.capture.shareMode = ma_share_mode_shared;
}
if (ca->deviceType & kCosmoAudioDeviceTypePlayback) {
deviceConfig.playback.channels = ca->channels;
deviceConfig.playback.format = ma_format_f32;
}
deviceConfig.dataCallback = data_callback;
deviceConfig.pUserData = ca;
result = ma_device_init(&ca->context, &deviceConfig, &ca->device);
if (result != MA_SUCCESS) {
ma_context_uninit(&ca->context);
ma_event_uninit(&ca->event);
ma_log_uninit(&ca->log);
free(ca);
return COSMOAUDIO_ERROR;
}
// Initialize the speaker ring buffer.
int period = ca->device.playback.internalPeriodSizeInFrames;
if (!options->bufferFrames) {
ca->outputBufferFrames = period * 10;
} else if (options->bufferFrames < period * 2) {
ca->outputBufferFrames = period * 2;
} else {
ca->outputBufferFrames = options->bufferFrames;
}
if (ca->deviceType & kCosmoAudioDeviceTypePlayback) {
result = ma_pcm_rb_init(ma_format_f32, ca->channels, ca->outputBufferFrames,
NULL, NULL, &ca->output);
if (result != MA_SUCCESS) {
ma_device_uninit(&ca->device);
ma_context_uninit(&ca->context);
ma_event_uninit(&ca->event);
ma_log_uninit(&ca->log);
free(ca);
return COSMOAUDIO_ERROR;
}
ma_pcm_rb_set_sample_rate(&ca->output, ca->sampleRate);
}
// Initialize the microphone ring buffer.
period = ca->device.capture.internalPeriodSizeInFrames;
if (!options->bufferFrames) {
ca->inputBufferFrames = period * 10;
} else if (options->bufferFrames < period * 2) {
ca->inputBufferFrames = period * 2;
} else {
ca->inputBufferFrames = options->bufferFrames;
}
if (ca->deviceType & kCosmoAudioDeviceTypeCapture) {
result = ma_pcm_rb_init(ma_format_f32, ca->channels, ca->inputBufferFrames,
NULL, NULL, &ca->input);
if (result != MA_SUCCESS) {
ma_device_uninit(&ca->device);
if (ca->deviceType & kCosmoAudioDeviceTypePlayback)
ma_pcm_rb_uninit(&ca->output);
ma_context_uninit(&ca->context);
ma_event_uninit(&ca->event);
ma_log_uninit(&ca->log);
free(ca);
return COSMOAUDIO_ERROR;
}
ma_pcm_rb_set_sample_rate(&ca->output, ca->sampleRate);
}
// Start audio playback.
if (ma_device_start(&ca->device) != MA_SUCCESS) {
ma_device_uninit(&ca->device);
if (ca->deviceType & kCosmoAudioDeviceTypePlayback)
ma_pcm_rb_uninit(&ca->output);
if (ca->deviceType & kCosmoAudioDeviceTypeCapture)
ma_pcm_rb_uninit(&ca->input);
ma_context_uninit(&ca->context);
ma_event_uninit(&ca->event);
ma_log_uninit(&ca->log);
free(ca);
return COSMOAUDIO_ERROR;
}
*out_ca = ca;
return COSMOAUDIO_SUCCESS;
}
/**
* Closes audio device and frees all associated resources.
*
* This function is non-blocking and will drop buffered audio. In
* playback mode, you need to call cosmoaudio_flush() to ensure data
* supplied by cosmoaudio_write() gets played on your speaker.
*
* Calling this function twice on the same object will result in
* undefined behavior. Even if this function fails, the `ca` will be
* freed to the greatest extent possible.
*
* @param ca is CosmoAudio object returned earlier by cosmoaudio_open()
* @return 0 on success, or negative error code on failure
*/
COSMOAUDIO_ABI int cosmoaudio_close(struct CosmoAudio* ca) {
if (!ca)
return COSMOAUDIO_EINVAL;
ma_device_uninit(&ca->device); // do this first
if (ca->deviceType & kCosmoAudioDeviceTypePlayback)
ma_pcm_rb_uninit(&ca->output);
if (ca->deviceType & kCosmoAudioDeviceTypeCapture)
ma_pcm_rb_uninit(&ca->input);
ma_context_uninit(&ca->context);
ma_event_uninit(&ca->event);
ma_log_uninit(&ca->log);
free(ca);
return COSMOAUDIO_SUCCESS;
}
/**
* Writes raw audio data to speaker.
*
* The data is written to a ring buffer in real-time, which is then
* played back very soon on the audio device. This has tolerence for
* a certain amount of buffering, but expects that this function is
* repeatedly called at a regular time interval. The caller should
* have its own sleep loop for this purpose.
*
* This function never blocks. Programs that don't have their own timer
* can use cosmoaudio_poll() to wait until audio may be written.
*
* For any given CosmoAudio object, it's assumed that only a single
* thread will call this function.
*
* @param ca is CosmoAudio object returned earlier by cosmoaudio_open()
* @param data is pointer to raw audio samples, expected to be in the range
* -1.0 to 1.0, where channels are interleaved
* @param frames is the number of frames (i.e. number of samples divided by
* number of channels) from `data` to write to audio device
* @return number of frames written, or negative error code on failure
*/
COSMOAUDIO_ABI int cosmoaudio_write(struct CosmoAudio* ca, const float* data,
int frames) {
if (!ca)
return COSMOAUDIO_EINVAL;
if (frames < 0)
return COSMOAUDIO_EINVAL;
if (!(ca->deviceType & kCosmoAudioDeviceTypePlayback))
return COSMOAUDIO_EINVAL;
if (1u + frames > ca->outputBufferFrames)
return COSMOAUDIO_ENOBUF;
if (!frames)
return 0;
if (!data)
return COSMOAUDIO_EINVAL;
return write_ring_buffer(&ca->log, &ca->output, data, frames, ca->channels);
}
/**
* Reads raw audio data from microphone.
*
* The data is read from a ring buffer in real-time, which is then
* played back on the audio device. This has tolerence for a certain
* amount of buffering (based on the `bufferFrames` parameter passed to
* cosmoaudio_open(), which by default assumes this function will be
* called at at a regular time interval.
*
* This function never blocks. Programs that don't have their own timer
* can use cosmoaudio_poll() to wait until audio may be read.
*
* For any given CosmoAudio object, it's assumed that only a single
* thread will call this function.
*
* @param ca is CosmoAudio object returned earlier by cosmoaudio_open()
* @param data is pointer to raw audio samples, expected to be in the range
* -1.0 to 1.0, where channels are interleaved
* @param frames is the number of frames (i.e. number of samples divided by
* number of channels) from `data` to read from microphone
* @return number of frames read, or negative error code on failure
*/
COSMOAUDIO_ABI int cosmoaudio_read(struct CosmoAudio* ca, float* data,
int frames) {
if (!ca)
return COSMOAUDIO_EINVAL;
if (frames < 0)
return COSMOAUDIO_EINVAL;
if (!(ca->deviceType & kCosmoAudioDeviceTypeCapture))
return COSMOAUDIO_EINVAL;
if (!frames)
return 0;
if (!data)
return COSMOAUDIO_EINVAL;
return read_ring_buffer(&ca->log, &ca->input, data, frames, ca->channels);
}
/**
* Waits until it's possible to read/write audio.
*
* This function is uninterruptible. All signals are masked throughout
* the duration of time this function may block, including cancelation
* signals, because this is not a cancelation point. Cosmopolitan Libc
* applies this masking in its dlopen wrapper.
*
* @param ca is CosmoAudio object returned earlier by cosmoaudio_open()
* @param in_out_readFrames if non-NULL specifies how many frames of
* capture data be immediately readable by cosmoaudio_read() before
* this can return; it must not exceed the buffer size; on return
* this will be set to the actual number of frames in the buffer;
* if the caller supplies a zero then this call is a non-blocking
* way to query buffer sizes
* @param in_out_writeFrames if non-NULL specifies how many frames of
* capture data be immediately writable by cosmoaudio_write() before
* this can return; it must not exceed the buffer size; on return
* this will be set to the actual number of frames in the buffer;
* if the caller supplies a zero then this call is a non-blocking
* way to query buffer sizes
* @return 0 on success, or negative error code on error
*/
COSMOAUDIO_ABI int cosmoaudio_poll(struct CosmoAudio* ca,
int* in_out_readFrames,
int* in_out_writeFrames) {
if (!ca)
return COSMOAUDIO_EINVAL;
if (!in_out_readFrames && !in_out_writeFrames)
return COSMOAUDIO_EINVAL;
if (in_out_readFrames && !(ca->deviceType & kCosmoAudioDeviceTypeCapture))
return COSMOAUDIO_EINVAL;
if (in_out_writeFrames && !(ca->deviceType & kCosmoAudioDeviceTypePlayback))
return COSMOAUDIO_EINVAL;
if (in_out_readFrames && 1u + *in_out_readFrames > ca->inputBufferFrames)
return COSMOAUDIO_ENOBUF;
if (in_out_writeFrames && 1u + *in_out_writeFrames > ca->outputBufferFrames)
return COSMOAUDIO_ENOBUF;
for (;;) {
int done = 1;
ma_uint32 readable = 0;
ma_uint32 writable = 0;
if (in_out_readFrames) {
readable = ma_pcm_rb_available_read(&ca->input);
done &= readable >= (ma_uint32)*in_out_readFrames;
}
if (in_out_writeFrames) {
writable = ma_pcm_rb_available_write(&ca->output);
done &= writable >= (ma_uint32)*in_out_writeFrames;
}
if (done) {
if (in_out_readFrames)
*in_out_readFrames = readable;
if (in_out_writeFrames)
*in_out_writeFrames = writable;
return COSMOAUDIO_SUCCESS;
}
if (ma_event_wait(&ca->event) != MA_SUCCESS)
return COSMOAUDIO_ERROR;
}
}
/**
* Waits for written samples to be sent to device.
*
* This function is only valid to call in playback or duplex mode.
*
* This function is uninterruptible. All signals are masked throughout
* the duration of time this function may block, including cancelation
* signals, because this is not a cancelation point. Cosmopolitan Libc
* applies this masking in its dlopen wrapper.
*
* @param ca is CosmoAudio object returned earlier by cosmoaudio_open()
* @return 0 on success, or negative error code on failure
*/
COSMOAUDIO_ABI int cosmoaudio_flush(struct CosmoAudio* ca) {
if (!ca)
return COSMOAUDIO_EINVAL;
if (!(ca->deviceType & kCosmoAudioDeviceTypePlayback))
return COSMOAUDIO_EINVAL;
for (;;) {
if (!ma_pcm_rb_available_read(&ca->output))
return COSMOAUDIO_SUCCESS;
if (ma_event_wait(&ca->event) != MA_SUCCESS)
return COSMOAUDIO_ERROR;
}
}

Binary file not shown.

View file

@ -0,0 +1,104 @@
#ifndef COSMOAUDIO_H_
#define COSMOAUDIO_H_
#ifdef _MSC_VER
#define COSMOAUDIO_ABI
#ifdef COSMOAUDIO_BUILD
#define COSMOAUDIO_API __declspec(dllexport)
#else
#define COSMOAUDIO_API __declspec(dllimport)
#endif
#else
#define COSMOAUDIO_API
#ifdef __x86_64__
#define COSMOAUDIO_ABI __attribute__((__ms_abi__, __visibility__("default")))
#else
#define COSMOAUDIO_ABI __attribute__((__visibility__("default")))
#endif
#endif
#define COSMOAUDIO_SUCCESS -0 // no error or nothing written
#define COSMOAUDIO_ERROR -1 // unspecified error
#define COSMOAUDIO_EINVAL -2 // invalid parameters passed to api
#define COSMOAUDIO_ELINK -3 // loading cosmoaudio dso failed
#define COSMOAUDIO_ENOBUF -4 // invalid buffering parameters
#ifdef __cplusplus
extern "C" {
#endif
struct CosmoAudio;
enum CosmoAudioDeviceType {
kCosmoAudioDeviceTypePlayback = 1,
kCosmoAudioDeviceTypeCapture = 2,
kCosmoAudioDeviceTypeDuplex =
kCosmoAudioDeviceTypePlayback | kCosmoAudioDeviceTypeCapture,
};
struct CosmoAudioOpenOptions {
// This field must be set to sizeof(struct CosmoAudioOpenOptions) or
// cosmoaudio_open() will return COSMOAUDIO_EINVAL.
int sizeofThis;
// Whether you want this object to open the speaker or microphone.
// Please note that asking for microphone access may cause some OSes
// like MacOS to show a popup asking the user for permission.
enum CosmoAudioDeviceType deviceType;
// The sample rate can be 44100 for CD quality, 8000 for telephone
// quality, etc. Values below 8000 are currently not supported.
int sampleRate;
// The number of audio channels in each interleaved frame. Should be 1
// for mono or 2 for stereo.
int channels;
// Number of frames in each ring buffer. A frame consists of a PCM
// sample for each channel. Set to 0 for default. If this is less than
// the device period size times two, it'll be increased to that value.
int bufferFrames;
// Enables debug logging if non-zero.
int debugLog;
};
COSMOAUDIO_API int cosmoaudio_version(void) COSMOAUDIO_ABI;
COSMOAUDIO_API int cosmoaudio_open( //
struct CosmoAudio **out_ca, //
const struct CosmoAudioOpenOptions *options //
) COSMOAUDIO_ABI;
COSMOAUDIO_API int cosmoaudio_close( //
struct CosmoAudio *ca //
) COSMOAUDIO_ABI;
COSMOAUDIO_API int cosmoaudio_write( //
struct CosmoAudio *ca, //
const float *samples, //
int frameCount //
) COSMOAUDIO_ABI;
COSMOAUDIO_API int cosmoaudio_flush( //
struct CosmoAudio *ca //
) COSMOAUDIO_ABI;
COSMOAUDIO_API int cosmoaudio_read( //
struct CosmoAudio *ca, //
float *out_samples, //
int frameCount //
) COSMOAUDIO_ABI;
COSMOAUDIO_API int cosmoaudio_poll( //
struct CosmoAudio *ca, //
int *in_out_readFrames, //
int *in_out_writeFrames //
) COSMOAUDIO_ABI;
#ifdef __cplusplus
}
#endif
#endif /* COSMOAUDIO_H_ */

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,76 @@
#if 0
/*─────────────────────────────────────────────────────────────────╗
To the extent possible under law, Justine Tunney has waived
all copyright and related or neighboring rights to this file,
as it is written in the following disclaimers:
http://unlicense.org/ │
http://creativecommons.org/publicdomain/zero/1.0/ │
*/
#endif
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include "cosmoaudio.h"
#define SAMPLING_RATE 44100
#define WAVE_INTERVAL 440
#define CHANNELS 2
#ifndef M_PIf
#define M_PIf 3.14159265358979323846f
#endif
int main() {
struct CosmoAudioOpenOptions cao = {0};
cao.sizeofThis = sizeof(struct CosmoAudioOpenOptions);
cao.deviceType = kCosmoAudioDeviceTypePlayback;
cao.sampleRate = SAMPLING_RATE;
cao.channels = CHANNELS;
int status;
struct CosmoAudio *ca;
status = cosmoaudio_open(&ca, &cao);
if (status != COSMOAUDIO_SUCCESS) {
fprintf(stderr, "failed to open audio: %d\n", status);
return 1;
}
float buf[256 * CHANNELS];
for (int g = 0; g < SAMPLING_RATE;) {
int frames = 1;
status = cosmoaudio_poll(ca, NULL, &frames);
if (status != COSMOAUDIO_SUCCESS) {
fprintf(stderr, "failed to poll output: %d\n", status);
return 2;
}
if (frames > 256)
frames = 256;
if (frames > SAMPLING_RATE - g)
frames = SAMPLING_RATE - g;
for (int f = 0; f < frames; ++f) {
float t = (float)g++ / SAMPLING_RATE;
float s = sinf(2 * M_PIf * WAVE_INTERVAL * t);
for (int c = 0; c < CHANNELS; c++)
buf[f * CHANNELS + c] = s * .3f;
}
status = cosmoaudio_write(ca, buf, frames);
if (status != frames) {
fprintf(stderr, "failed to write output: %d\n", status);
return 3;
}
}
status = cosmoaudio_flush(ca);
if (status != COSMOAUDIO_SUCCESS) {
fprintf(stderr, "failed to flush output: %d\n", status);
return 4;
}
status = cosmoaudio_close(ca);
if (status != COSMOAUDIO_SUCCESS) {
fprintf(stderr, "failed to close audio: %d\n", status);
return 5;
}
}

121
dsp/audio/describe.c Normal file
View file

@ -0,0 +1,121 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
vi: set et ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi
Copyright 2024 Justine Alexandra Roberts Tunney
Permission to use, copy, modify, and/or distribute this software for
any purpose with or without fee is hereby granted, provided that the
above copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE
AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL
DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR
PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE.
*/
#include "dsp/audio/describe.h"
#include "dsp/audio/cosmoaudio/cosmoaudio.h"
#include "libc/intrin/describeflags.h"
#include "libc/intrin/kprintf.h"
#include "libc/macros.h"
#define append(...) o += ksnprintf(buf + o, n - o, __VA_ARGS__)
const char *cosmoaudio_describe_status(char *buf, int n, int status) {
switch (status) {
case COSMOAUDIO_SUCCESS:
return "COSMOAUDIO_SUCCESS";
case COSMOAUDIO_ERROR:
return "COSMOAUDIO_ERROR";
case COSMOAUDIO_EINVAL:
return "COSMOAUDIO_EINVAL";
case COSMOAUDIO_ELINK:
return "COSMOAUDIO_ELINK";
case COSMOAUDIO_ENOBUF:
return "COSMOAUDIO_ENOBUF";
default:
ksnprintf(buf, n, "%d", status);
return buf;
}
}
const char *cosmoaudio_describe_open_options(
char *buf, int n, const struct CosmoAudioOpenOptions *options) {
int o = 0;
char b128[128];
bool gotsome = false;
if (!options)
return "NULL";
if (kisdangerous(options)) {
ksnprintf(buf, n, "%p", options);
return buf;
}
append("{");
if (options->sampleRate) {
if (gotsome)
append(", ");
append(".sampleRate=%d", options->sampleRate);
gotsome = true;
}
if (options->channels) {
if (gotsome)
append(", ");
append(".channels=%d", options->channels);
gotsome = true;
}
if (options->deviceType) {
if (gotsome)
append(", ");
static struct DescribeFlags kDeviceType[] = {
{kCosmoAudioDeviceTypeDuplex, "Duplex"}, //
{kCosmoAudioDeviceTypeCapture, "Capture"}, //
{kCosmoAudioDeviceTypePlayback, "Playback"}, //
};
append(".deviceType=%s",
_DescribeFlags(b128, 128, kDeviceType, ARRAYLEN(kDeviceType),
"kCosmoAudioDeviceType", options->deviceType));
gotsome = true;
}
if (options->bufferFrames) {
if (gotsome)
append(", ");
append(".bufferFrames=%d", options->bufferFrames);
gotsome = true;
}
if (options->debugLog) {
if (gotsome)
append(", ");
append(".debugLog=%d", options->debugLog);
gotsome = true;
}
if (options->sizeofThis) {
if (gotsome)
append(", ");
append(".sizeofThis=%d", options->sizeofThis);
gotsome = true;
}
append("}");
return buf;
}
const char *cosmoaudio_describe_poll_frames(char *buf, int n,
int *in_out_frames) {
if (!in_out_frames)
return "NULL";
if (kisdangerous(in_out_frames)) {
ksnprintf(buf, n, "%p", in_out_frames);
return buf;
}
ksnprintf(buf, n, "[%d]", *in_out_frames);
return buf;
}

12
dsp/audio/describe.h Normal file
View file

@ -0,0 +1,12 @@
#ifndef COSMOPOLITAN_DSP_AUDIO_DESCRIBE_H_
#define COSMOPOLITAN_DSP_AUDIO_DESCRIBE_H_
#include "dsp/audio/cosmoaudio/cosmoaudio.h"
COSMOPOLITAN_C_START_
const char *cosmoaudio_describe_status(char *, int, int);
const char *cosmoaudio_describe_open_options(
char *, int, const struct CosmoAudioOpenOptions *);
const char *cosmoaudio_describe_poll_frames(char *, int, int *);
COSMOPOLITAN_C_END_
#endif /* COSMOPOLITAN_DSP_AUDIO_DESCRIBE_H_ */

View file

@ -1,6 +1,6 @@
#ifndef COSMOPOLITAN_DSP_CORE_C161_H_
#define COSMOPOLITAN_DSP_CORE_C161_H_
#include "libc/macros.internal.h"
#include "libc/macros.h"
#define EXTRA_SHARP 2

View file

@ -1,7 +1,7 @@
#ifndef COSMOPOLITAN_DSP_CORE_C161S_H_
#define COSMOPOLITAN_DSP_CORE_C161S_H_
#include "dsp/core/c161.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
__funline signed char C161S(signed char al, signed char bl, signed char cl) {
short ax, bx, cx;

View file

@ -9,14 +9,13 @@ int mulaw(int);
int unmulaw(int);
void *double2byte(long, const void *, double, double) vallocesque;
void *byte2double(long, const void *, double, double) vallocesque;
void *dct(float[restrict hasatleast 8][8], unsigned, float, float, float, float,
float);
void *dctjpeg(float[restrict hasatleast 8][8], unsigned);
void *dct(float[hasatleast 8][8], unsigned, float, float, float, float, float);
void *dctjpeg(float[hasatleast 8][8], unsigned);
double det3(const double[3][3]) nosideeffect;
void *inv3(double[restrict 3][3], const double[restrict 3][3], double);
void *matmul3(double[restrict 3][3], const double[3][3], const double[3][3]);
void *vmatmul3(double[restrict 3], const double[3], const double[3][3]);
void *matvmul3(double[restrict 3], const double[3][3], const double[3]);
void *inv3(double[3][3], const double[3][3], double);
void *matmul3(double[3][3], const double[3][3], const double[3][3]);
void *vmatmul3(double[3], const double[3], const double[3][3]);
void *matvmul3(double[3], const double[3][3], const double[3]);
double rgb2stdtv(double) pureconst;
double rgb2lintv(double) pureconst;
double rgb2stdpc(double, double) pureconst;

View file

@ -65,8 +65,8 @@
*
* @cost ~100ns
*/
void *dct(float M[restrict hasatleast 8][8], unsigned stride, float c0,
float c1, float c2, float c3, float c4) {
void *dct(float M[hasatleast 8][8], unsigned stride, float c0, float c1,
float c2, float c3, float c4) {
unsigned y, x;
for (y = 0; y < stride * 8; y += stride) {
DCT(M[y][0], M[y][1], M[y][2], M[y][3], M[y][4], M[y][5], M[y][6], M[y][7],
@ -79,7 +79,7 @@ void *dct(float M[restrict hasatleast 8][8], unsigned stride, float c0,
return M;
}
void *dctjpeg(float M[restrict hasatleast 8][8], unsigned stride) {
void *dctjpeg(float M[hasatleast 8][8], unsigned stride) {
return dct(M, stride, .707106781f, .382683433f, .541196100f, 1.306562965f,
.707106781f);
}

View file

@ -17,7 +17,7 @@
PERFORMANCE OF THIS SOFTWARE.
*/
#include "dsp/core/core.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
#include "libc/math.h"
#include "libc/mem/mem.h"

View file

@ -17,7 +17,7 @@
PERFORMANCE OF THIS SOFTWARE.
*/
#include "libc/limits.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
#include "libc/math.h"
#include "libc/str/str.h"

View file

@ -20,7 +20,7 @@
#include "libc/assert.h"
#include "libc/dce.h"
#include "libc/limits.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
#include "libc/math.h"
#include "libc/str/str.h"

View file

@ -19,7 +19,7 @@
#include "dsp/core/core.h"
#include "dsp/core/q.h"
#include "libc/dce.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
#include "libc/math.h"
#include "libc/str/str.h"

View file

@ -1,6 +1,6 @@
#ifndef COSMOPOLITAN_DSP_CORE_HALF_H_
#define COSMOPOLITAN_DSP_CORE_HALF_H_
#include "libc/macros.internal.h"
#include "libc/macros.h"
/**
* Divides integer in half w/ rounding.

View file

@ -30,7 +30,7 @@
* @define 𝐀¹=𝐁 such that 𝐀×𝐁=𝐁×𝐀=𝐈
* @see det3()
*/
void *inv3(double B[restrict 3][3], const double A[restrict 3][3], double d) {
void *inv3(double B[3][3], const double A[3][3], double d) {
d = d ? 1 / d : NAN;
B[0][0] = (A[1][1] * A[2][2] - A[2][1] * A[1][2]) * d;
B[0][1] = (A[2][1] * A[0][2] - A[0][1] * A[2][2]) * d;

View file

@ -1,6 +1,6 @@
#ifndef COSMOPOLITAN_DSP_CORE_KS8_H_
#define COSMOPOLITAN_DSP_CORE_KS8_H_
#include "libc/macros.internal.h"
#include "libc/macros.h"
/**
* Performs 16-bit scaled rounded madd w/ eight coefficients or fewer.

View file

@ -1,7 +1,7 @@
#ifndef COSMOPOLITAN_DSP_CORE_KSS8_H_
#define COSMOPOLITAN_DSP_CORE_KSS8_H_
#include "libc/limits.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
/**
* Performs 16-bit scaled rounded saturated madd w/ eight coefficients or fewer.

View file

@ -22,8 +22,7 @@
/**
* Multiplies 3×3 matrices.
*/
void *matmul3(double R[restrict 3][3], const double A[3][3],
const double B[3][3]) {
void *matmul3(double R[3][3], const double A[3][3], const double B[3][3]) {
int i, j, k;
memset(R, 0, sizeof(double) * 3 * 3);
for (i = 0; i < 3; ++i) {

View file

@ -23,7 +23,7 @@
*
* @see vmatmul3() for noncommutative corollary
*/
void *matvmul3(double R[restrict 3], const double M[3][3], const double V[3]) {
void *matvmul3(double R[3], const double M[3][3], const double V[3]) {
R[0] = V[0] * M[0][0] + V[1] * M[0][1] + V[2] * M[0][2];
R[1] = V[0] * M[1][0] + V[1] * M[1][1] + V[2] * M[1][2];
R[2] = V[0] * M[2][0] + V[1] * M[2][1] + V[2] * M[2][2];

View file

@ -1,7 +1,7 @@
#ifndef COSMOPOLITAN_DSP_CORE_Q_H_
#define COSMOPOLITAN_DSP_CORE_Q_H_
#include "libc/limits.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
#include "libc/math.h"
/**

View file

@ -18,7 +18,7 @@
*/
#include "dsp/core/core.h"
#include "libc/limits.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
#include "third_party/aarch64/arm_neon.internal.h"
#include "third_party/intel/emmintrin.internal.h"

View file

@ -23,7 +23,7 @@
*
* @see matvmul3() for noncommutative corollary
*/
void *vmatmul3(double R[restrict 3], const double V[3], const double M[3][3]) {
void *vmatmul3(double R[3], const double V[3], const double M[3][3]) {
R[0] = V[0] * M[0][0] + V[1] * M[1][0] + V[2] * M[2][0];
R[1] = V[0] * M[0][1] + V[1] * M[1][1] + V[2] * M[2][1];
R[2] = V[0] * M[0][2] + V[1] * M[1][2] + V[2] * M[2][2];

2
dsp/mpeg/.clang-format Normal file
View file

@ -0,0 +1,2 @@
DisableFormat: true
SortIncludes: Never

View file

@ -25,18 +25,13 @@ DSP_MPEG_A_CHECKS = \
DSP_MPEG_A_DIRECTDEPS = \
LIBC_CALLS \
LIBC_FMT \
LIBC_INTRIN \
LIBC_LOG \
LIBC_LOG \
LIBC_MEM \
LIBC_NEXGEN32E \
LIBC_RUNTIME \
LIBC_STDIO \
LIBC_STR \
LIBC_SYSV \
LIBC_TINYMATH \
THIRD_PARTY_COMPILER_RT
THIRD_PARTY_COMPILER_RT \
DSP_MPEG_A_DEPS := \
$(call uniq,$(foreach x,$(DSP_MPEG_A_DIRECTDEPS),$($(x))))
@ -49,9 +44,10 @@ $(DSP_MPEG_A).pkg: \
$(DSP_MPEG_A_OBJS) \
$(foreach x,$(DSP_MPEG_A_DIRECTDEPS),$($(x)_A).pkg)
o/$(MODE)/dsp/mpeg/clamp4int256-k8.o: private \
o/$(MODE)/dsp/mpeg/pl_mpeg.o: private \
CFLAGS += \
-Os
-ffunction-sections \
-fdata-sections
DSP_MPEG_LIBS = $(foreach x,$(DSP_MPEG_ARTIFACTS),$($(x)))
DSP_MPEG_SRCS = $(foreach x,$(DSP_MPEG_ARTIFACTS),$($(x)_SRCS))

17
dsp/mpeg/README.cosmo Normal file
View file

@ -0,0 +1,17 @@
DESCRIPTION
pl_mpeg lets you decode .mpg files
ORIGIN
https://github.com/phoboslab/pl_mpeg/
9e40dd6536269d788728e32c39bfacf2ab7a0866
LICENSE
MIT
LOCAL CHANGES
- Added API for extracting pixel aspect ratio
https://github.com/phoboslab/pl_mpeg/pull/42

68
dsp/mpeg/README.md Executable file
View file

@ -0,0 +1,68 @@
# PL_MPEG - MPEG1 Video decoder, MP2 Audio decoder, MPEG-PS demuxer
Single-file MIT licensed library for C/C++
See [pl_mpeg.h](https://github.com/phoboslab/pl_mpeg/blob/master/pl_mpeg.h) for
the documentation.
## Why?
This is meant as a simple way to get video playback into your app or game. Other
solutions, such as ffmpeg require huge libraries and a lot of glue code.
MPEG1 is an old and inefficient codec, but it's still good enough for many use
cases. All patents related to MPEG1 and MP2 have expired, so it's completely
free now.
This library does not make use of any SIMD instructions, but because of
the relative simplicity of the codec it still manages to decode 4k60fps video
on a single CPU core (on my i7-6700k at least).
## Compilation on Linux
Use a GCC invocation like the following to build the example `pl_mpeg_player`
program:
```shell
gcc -o pl_mpeg_player pl_mpeg_player.c $(pkg-config --cflags --libs sdl2 glew)
```
## Example Usage
- [pl_mpeg_extract_frames.c](https://github.com/phoboslab/pl_mpeg/blob/master/pl_mpeg_extract_frames.c)
extracts all frames from a video and saves them as PNG.
- [pl_mpeg_player.c](https://github.com/phoboslab/pl_mpeg/blob/master/pl_mpeg_player.c)
implements a video player using SDL2 and OpenGL for rendering.
## Encoding for PL_MPEG
Most [MPEG-PS](https://en.wikipedia.org/wiki/MPEG_program_stream) (`.mpg`) files
containing MPEG1 Video ("mpeg1") and MPEG1 Audio Layer II ("mp2") streams should
work with PL_MPEG. Note that `.mpg` files can also contain MPEG2 Video, which is
not supported by this library.
You can encode video in a suitable format using ffmpeg:
```
ffmpeg -i input.mp4 -c:v mpeg1video -q:v 0 -c:a mp2 -format mpeg output.mpg
```
`-q:v` sets a fixed video quality with a variable bitrate, where `0` is the
highest. You may use `-b:v` to set a fixed bitrate instead; e.g.
`-b:v 2000k` for 2000 kbit/s. Please refer to the
[ffmpeg documentation](http://ffmpeg.org/ffmpeg.html#Options) for more details.
If you just want to quickly test the library, try this file:
https://phoboslab.org/files/bjork-all-is-full-of-love.mpg
## Limitations
- no error reporting. PL_MPEG will silently ignore any invalid data.
- the pts (presentation time stamp) for packets in the MPEG-PS container is
ignored. This may cause sync issues with some files.
- bugs, probably.

View file

@ -1,92 +0,0 @@
PL_MPEG - MPEG1 Video decoder, MP2 Audio decoder, MPEG-PS demuxer
Dominic Szablewski - https://phoboslab.org
-- Synopsis
// This function gets called for each decoded video frame
void my_video_callback(plm_t *plm, plm_frame_t *frame, void *user) {
// Do something with frame->y.data, frame->cr.data, frame->cb.data
}
// This function gets called for each decoded audio frame
void my_audio_callback(plm_t *plm, plm_samples_t *frame, void *user) {
// Do something with samples->interleaved
}
// Load a .mpg (MPEG Program Stream) file
plm_t *plm = plm_create_with_filename("some-file.mpg");
// Install the video & audio decode callbacks
plm_set_video_decode_callback(plm, my_video_callback, my_data);
plm_set_audio_decode_callback(plm, my_audio_callback, my_data);
// Decode
do {
plm_decode(plm, time_since_last_call);
} while (!plm_has_ended(plm));
// All done
plm_destroy(plm);
-- Documentation
This library provides several interfaces to load, demux and decode MPEG video
and audio data. A high-level API combines the demuxer, video & audio decoders
in an easy to use wrapper.
Lower-level APIs for accessing the demuxer, video decoder and audio decoder,
as well as providing different data sources are also available.
Interfaces are written in an object orientet style, meaning you create object
instances via various different constructor functions (plm_*create()),
do some work on them and later dispose them via plm_*destroy().
plm_* -- the high-level interface, combining demuxer and decoders
plm_buffer_* -- the data source used by all interfaces
plm_demux_* -- the MPEG-PS demuxer
plm_video_* -- the MPEG1 Video ("mpeg1") decoder
plm_audio_* -- the MPEG1 Audio Layer II ("mp2") decoder
This library uses malloc(), realloc() and free() to manage memory. Typically
all allocation happens up-front when creating the interface. However, the
default buffer size may be too small for certain inputs. In these cases plmpeg
will realloc() the buffer with a larger size whenever needed. You can configure
the default buffer size by defining PLM_BUFFER_DEFAULT_SIZE *before*
including this library.
With the high-level interface you have two options to decode video & audio:
1) Use plm_decode() and just hand over the delta time since the last call.
It will decode everything needed and call your callbacks (specified through
plm_set_{video|audio}_decode_callback()) any number of times.
2) Use plm_decode_video() and plm_decode_audio() to decode exactly one
frame of video or audio data at a time. How you handle the synchronization of
both streams is up to you.
If you only want to decode video *or* audio through these functions, you should
disable the other stream (plm_set_{video|audio}_enabled(false))
Video data is decoded into a struct with all 3 planes (Y, Cr, Cb) stored in
separate buffers. You can either convert this to RGB on the CPU (slow) via the
plm_frame_to_rgb() function or do it on the GPU with the following matrix:
mat4 rec601 = mat4(
1.16438, 0.00000, 1.59603, -0.87079,
1.16438, -0.39176, -0.81297, 0.52959,
1.16438, 2.01723, 0.00000, -1.08139,
0, 0, 0, 1
);
gl_FragColor = vec4(y, cb, cr, 1.0) * rec601;
Audio data is decoded into a struct with either one single float array with the
samples for the left and right channel interleaved, or if the
PLM_AUDIO_SEPARATE_CHANNELS is defined *before* including this library, into
two separate float arrays - one for each channel.
See below for detailed the API documentation.

View file

@ -1,20 +0,0 @@
#ifndef COSMOPOLITAN_DSP_MPEG_BLOCKSET_H_
#define COSMOPOLITAN_DSP_MPEG_BLOCKSET_H_
#define PLM_BLOCK_SET(DEST, DEST_INDEX, DEST_WIDTH, SOURCE_INDEX, \
SOURCE_WIDTH, BLOCK_SIZE, OP) \
do { \
int dest_scan = DEST_WIDTH - BLOCK_SIZE; \
int source_scan = SOURCE_WIDTH - BLOCK_SIZE; \
for (int y = 0; y < BLOCK_SIZE; y++) { \
for (int x = 0; x < BLOCK_SIZE; x++) { \
DEST[DEST_INDEX] = OP; \
SOURCE_INDEX++; \
DEST_INDEX++; \
} \
SOURCE_INDEX += source_scan; \
DEST_INDEX += dest_scan; \
} \
} while (false)
#endif /* COSMOPOLITAN_DSP_MPEG_BLOCKSET_H_ */

View file

@ -1,153 +0,0 @@
/*-*- mode:c;indent-tabs-mode:t;c-basic-offset:4;tab-width:4;coding:utf-8 -*-│
vi: set noet ft=c ts=4 sw=4 fenc=utf-8 :vi
PL_MPEG - MPEG1 Video decoder, MP2 Audio decoder, MPEG-PS demuxer
Dominic Szablewski - https://phoboslab.org │
The MIT License(MIT)
Copyright(c) 2019 Dominic Szablewski
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files(the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and / or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT.IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include "dsp/mpeg/buffer.h"
#include "dsp/mpeg/mpeg.h"
#include "libc/calls/calls.h"
#include "libc/log/check.h"
#include "libc/mem/mem.h"
#include "libc/stdio/stdio.h"
#include "libc/str/str.h"
#include "libc/sysv/consts/madv.h"
__static_yoink("pl_mpeg_notice");
/* clang-format off */
// -----------------------------------------------------------------------------
// plm_buffer implementation
plm_buffer_t *plm_buffer_create_with_filename(const char *filename) {
FILE *fh = fopen(filename, "rb");
if (!fh) {
return NULL;
}
fadvise(fileno(fh), 0, 0, MADV_SEQUENTIAL);
return plm_buffer_create_with_file(fh, true);
}
plm_buffer_t *plm_buffer_create_with_file(FILE *fh, int close_when_done) {
plm_buffer_t *b;
b = plm_buffer_create_with_capacity(PLM_BUFFER_DEFAULT_SIZE);
b->fh = fh;
b->close_when_done = close_when_done;
b->mode = PLM_BUFFER_MODE_FILE;
plm_buffer_set_load_callback(b, plm_buffer_load_file_callback, NULL);
return b;
}
plm_buffer_t *plm_buffer_create_with_memory(unsigned char *bytes, size_t length, int free_when_done) {
plm_buffer_t *b;
b = memalign(_Alignof(plm_buffer_t), sizeof(plm_buffer_t));
memset(b, 0, sizeof(plm_buffer_t));
b->capacity = length;
b->length = length;
b->free_when_done = free_when_done;
b->bytes = bytes;
b->mode = PLM_BUFFER_MODE_FIXED_MEM;
return b;
}
plm_buffer_t * plm_buffer_create_with_capacity(size_t capacity) {
plm_buffer_t *b;
b = memalign(_Alignof(plm_buffer_t), sizeof(plm_buffer_t));
memset(b, 0, sizeof(plm_buffer_t));
b->capacity = capacity;
b->free_when_done = true;
b->bytes = (unsigned char *)malloc(capacity);
b->mode = PLM_BUFFER_MODE_DYNAMIC_MEM;
return b;
}
void plm_buffer_destroy(plm_buffer_t *self) {
if (self->fh && self->close_when_done) {
fclose(self->fh);
}
if (self->free_when_done) {
free(self->bytes);
}
free(self);
}
size_t plm_buffer_write(plm_buffer_t *self, unsigned char *bytes, size_t length) {
if (self->mode == PLM_BUFFER_MODE_FIXED_MEM) {
return 0;
}
// This should be a ring buffer, but instead it just shifts all unread data
// to the beginning of the buffer and appends new data at the end. Seems
// to be good enough.
plm_buffer_discard_read_bytes(self);
// Do we have to resize to fit the new data?
size_t bytes_available = self->capacity - self->length;
if (bytes_available < length) {
size_t new_size = self->capacity;
do {
new_size *= 2;
} while (new_size - self->length < length);
self->bytes = (unsigned char *)realloc(self->bytes, new_size);
self->capacity = new_size;
}
memcpy(self->bytes + self->length, bytes, length);
self->length += length;
return length;
}
void plm_buffer_set_load_callback(plm_buffer_t *self, plm_buffer_load_callback fp, void *user) {
self->load_callback = fp;
self->load_callback_user_data = user;
}
void plm_buffer_rewind(plm_buffer_t *self) {
if (self->fh) {
fseek(self->fh, 0, SEEK_SET);
self->length = 0;
}
if (self->mode != PLM_BUFFER_MODE_FIXED_MEM) {
self->length = 0;
}
self->bit_index = 0;
}
void plm_buffer_discard_read_bytes(plm_buffer_t *self) {
size_t byte_pos = self->bit_index >> 3;
if (byte_pos == self->length) {
self->bit_index = 0;
self->length = 0;
}
else if (byte_pos > 0) {
memmove(self->bytes, self->bytes + byte_pos, self->length - byte_pos);
self->bit_index -= byte_pos << 3;
self->length -= byte_pos;
}
}
void plm_buffer_load_file_callback(plm_buffer_t *self, void *user) {
plm_buffer_discard_read_bytes(self);
unsigned bytes_available = self->capacity - self->length;
unsigned bytes_read = fread(self->bytes + self->length, 1, bytes_available, self->fh);
self->length += bytes_read;
}

View file

@ -1,160 +0,0 @@
#ifndef COSMOPOLITAN_DSP_MPEG_BUFFER_H_
#define COSMOPOLITAN_DSP_MPEG_BUFFER_H_
#include "dsp/mpeg/mpeg.h"
COSMOPOLITAN_C_START_
enum plm_buffer_mode {
PLM_BUFFER_MODE_FILE,
PLM_BUFFER_MODE_FIXED_MEM,
PLM_BUFFER_MODE_DYNAMIC_MEM
};
typedef struct plm_buffer_t {
unsigned bit_index;
unsigned capacity;
unsigned length;
int free_when_done;
int close_when_done;
FILE *fh;
plm_buffer_load_callback load_callback;
void *load_callback_user_data;
unsigned char *bytes;
enum plm_buffer_mode mode;
} plm_buffer_t;
typedef struct {
int16_t index;
int16_t value;
} plm_vlc_t;
typedef struct {
int16_t index;
uint16_t value;
} plm_vlc_uint_t;
/* bool plm_buffer_has(plm_buffer_t *, size_t); */
/* int plm_buffer_read(plm_buffer_t *, int); */
/* void plm_buffer_align(plm_buffer_t *); */
/* void plm_buffer_skip(plm_buffer_t *, size_t); */
/* int plm_buffer_skip_bytes(plm_buffer_t *, unsigned char); */
/* int plm_buffer_next_start_code(plm_buffer_t *); */
/* int plm_buffer_find_start_code(plm_buffer_t *, int); */
/* int plm_buffer_no_start_code(plm_buffer_t *); */
/* int16_t plm_buffer_read_vlc(plm_buffer_t *, const plm_vlc_t *); */
/* uint16_t plm_buffer_read_vlc_uint(plm_buffer_t *, const plm_vlc_uint_t *); */
void plm_buffer_discard_read_bytes(plm_buffer_t *);
relegated void plm_buffer_load_file_callback(plm_buffer_t *, void *);
forceinline bool plm_buffer_has(plm_buffer_t *b, size_t bits) {
unsigned have;
have = b->length;
have <<= 3;
have -= b->bit_index;
if (bits <= have) {
return true;
} else {
if (b->load_callback) {
b->load_callback(b, b->load_callback_user_data);
return ((b->length << 3) - b->bit_index) >= bits;
} else {
return false;
}
}
}
forceinline int plm_buffer_read(plm_buffer_t *self, int count) {
if (!plm_buffer_has(self, count))
return 0;
int value = 0;
while (count) {
int current_byte = self->bytes[self->bit_index >> 3];
int remaining = 8 - (self->bit_index & 7); // Remaining bits in byte
int read = remaining < count ? remaining : count; // Bits in self run
int shift = remaining - read;
int mask = (0xff >> (8 - read));
value = (value << read) | ((current_byte & (mask << shift)) >> shift);
self->bit_index += read;
count -= read;
}
return value;
}
forceinline void plm_buffer_align(plm_buffer_t *self) {
self->bit_index = ((self->bit_index + 7) >> 3) << 3;
}
forceinline void plm_buffer_skip(plm_buffer_t *self, size_t count) {
if (plm_buffer_has(self, count)) {
self->bit_index += count;
}
}
forceinline int plm_buffer_skip_bytes(plm_buffer_t *self, unsigned char v) {
unsigned skipped;
plm_buffer_align(self);
skipped = 0;
while (plm_buffer_has(self, 8)) {
if (v == self->bytes[self->bit_index >> 3]) {
self->bit_index += 8;
++skipped;
} else {
break;
}
}
return skipped;
}
forceinline int plm_buffer_next_start_code(plm_buffer_t *self) {
plm_buffer_align(self);
while (plm_buffer_has(self, (5 << 3))) {
size_t byte_index = (self->bit_index) >> 3;
if (self->bytes[byte_index] == 0x00 &&
self->bytes[byte_index + 1] == 0x00 &&
self->bytes[byte_index + 2] == 0x01) {
self->bit_index = (byte_index + 4) << 3;
return self->bytes[byte_index + 3];
}
self->bit_index += 8;
}
self->bit_index = (self->length << 3);
return -1;
}
forceinline int plm_buffer_find_start_code(plm_buffer_t *self, int code) {
int current = 0;
while (true) {
current = plm_buffer_next_start_code(self);
if (current == code || current == -1) {
return current;
}
}
return -1;
}
forceinline int plm_buffer_no_start_code(plm_buffer_t *self) {
if (!plm_buffer_has(self, (5 << 3))) {
return false;
}
size_t byte_index = ((self->bit_index + 7) >> 3);
return !(self->bytes[byte_index] == 0x00 &&
self->bytes[byte_index + 1] == 0x00 &&
self->bytes[byte_index + 2] == 0x01);
}
forceinline int16_t plm_buffer_read_vlc(plm_buffer_t *self,
const plm_vlc_t *table) {
plm_vlc_t state = {0, 0};
do {
state = table[state.index + plm_buffer_read(self, 1)];
} while (state.index > 0);
return state.value;
}
forceinline uint16_t plm_buffer_read_vlc_uint(plm_buffer_t *self,
const plm_vlc_uint_t *table) {
return (uint16_t)plm_buffer_read_vlc(self, (plm_vlc_t *)table);
}
COSMOPOLITAN_C_END_
#endif /* COSMOPOLITAN_DSP_MPEG_BUFFER_H_ */

View file

@ -1,203 +0,0 @@
/*-*- mode:c;indent-tabs-mode:t;c-basic-offset:4;tab-width:4;coding:utf-8 -*-│
vi: set noet ft=c ts=4 sw=4 fenc=utf-8 :vi
PL_MPEG - MPEG1 Video decoder, MP2 Audio decoder, MPEG-PS demuxer
Dominic Szablewski - https://phoboslab.org │
The MIT License(MIT)
Copyright(c) 2019 Dominic Szablewski
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files(the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and / or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT.IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include "dsp/mpeg/demux.h"
#include "dsp/mpeg/buffer.h"
#include "dsp/mpeg/mpeg.h"
#include "libc/mem/mem.h"
#include "libc/str/str.h"
__static_yoink("pl_mpeg_notice");
/* clang-format off */
// ----------------------------------------------------------------------------
// plm_demux implementation
plm_demux_t *plm_demux_create(plm_buffer_t *buffer, int destroy_when_done) {
plm_demux_t *self = (plm_demux_t *)malloc(sizeof(plm_demux_t));
memset(self, 0, sizeof(plm_demux_t));
self->buffer = buffer;
self->destroy_buffer_when_done = destroy_when_done;
if (plm_buffer_find_start_code(self->buffer, START_PACK) != -1) {
plm_demux_decode_pack_header(self);
}
if (plm_buffer_find_start_code(self->buffer, START_SYSTEM) != -1) {
plm_demux_decode_system_header(self);
}
return self;
}
void plm_demux_destroy(plm_demux_t *self) {
if (self->destroy_buffer_when_done) {
plm_buffer_destroy(self->buffer);
}
free(self);
}
int plm_demux_get_num_video_streams(plm_demux_t *self) {
return self->num_video_streams;
}
int plm_demux_get_num_audio_streams(plm_demux_t *self) {
return self->num_audio_streams;
}
void plm_demux_rewind(plm_demux_t *self) {
plm_buffer_rewind(self->buffer);
}
plm_packet_t *plm_demux_decode(plm_demux_t *self) {
if (self->current_packet.length) {
size_t bits_till_next_packet = self->current_packet.length << 3;
if (!plm_buffer_has(self->buffer, bits_till_next_packet)) {
return NULL;
}
plm_buffer_skip(self->buffer, bits_till_next_packet);
self->current_packet.length = 0;
}
if (!self->has_pack_header) {
if (plm_buffer_find_start_code(self->buffer, START_PACK) != -1) {
plm_demux_decode_pack_header(self);
}
else {
return NULL;
}
}
if (!self->has_system_header) {
if (plm_buffer_find_start_code(self->buffer, START_SYSTEM) != -1) {
plm_demux_decode_system_header(self);
}
else {
return NULL;
}
}
// pending packet just waiting for data?
if (self->next_packet.length) {
return plm_demux_get_packet(self);
}
int code;
do {
code = plm_buffer_next_start_code(self->buffer);
if (
code == PLM_DEMUX_PACKET_VIDEO_1 ||
code == PLM_DEMUX_PACKET_PRIVATE ||
(code >= PLM_DEMUX_PACKET_AUDIO_1 && code <= PLM_DEMUX_PACKET_AUDIO_4)
) {
return plm_demux_decode_packet(self, code);
}
} while (code != -1);
return NULL;
}
double plm_demux_read_time(plm_demux_t *self) {
int64_t clock = plm_buffer_read(self->buffer, 3) << 30;
plm_buffer_skip(self->buffer, 1);
clock |= plm_buffer_read(self->buffer, 15) << 15;
plm_buffer_skip(self->buffer, 1);
clock |= plm_buffer_read(self->buffer, 15);
plm_buffer_skip(self->buffer, 1);
return (double)clock / 90000.0;
}
void plm_demux_decode_pack_header(plm_demux_t *self) {
if (plm_buffer_read(self->buffer, 4) != 0x02) {
return; // invalid
}
self->system_clock_ref = plm_demux_read_time(self);
plm_buffer_skip(self->buffer, 1);
plm_buffer_skip(self->buffer, 22); // mux_rate * 50
plm_buffer_skip(self->buffer, 1);
self->has_pack_header = true;
}
void plm_demux_decode_system_header(plm_demux_t *self) {
plm_buffer_skip(self->buffer, 16); // header_length
plm_buffer_skip(self->buffer, 24); // rate bound
self->num_audio_streams = plm_buffer_read(self->buffer, 6);
plm_buffer_skip(self->buffer, 5); // misc flags
self->num_video_streams = plm_buffer_read(self->buffer, 5);
self->has_system_header = true;
}
plm_packet_t *plm_demux_decode_packet(plm_demux_t *self, int start_code) {
if (!plm_buffer_has(self->buffer, 8 << 3)) {
return NULL;
}
self->next_packet.type = start_code;
self->next_packet.length = plm_buffer_read(self->buffer, 16);
self->next_packet.length -= plm_buffer_skip_bytes(self->buffer, 0xff); // stuffing
// skip P-STD
if (plm_buffer_read(self->buffer, 2) == 0x01) {
plm_buffer_skip(self->buffer, 16);
self->next_packet.length -= 2;
}
int pts_dts_marker = plm_buffer_read(self->buffer, 2);
if (pts_dts_marker == 0x03) {
self->next_packet.pts = plm_demux_read_time(self);
plm_buffer_skip(self->buffer, 40); // skip dts
self->next_packet.length -= 10;
}
else if (pts_dts_marker == 0x02) {
self->next_packet.pts = plm_demux_read_time(self);
self->next_packet.length -= 5;
}
else if (pts_dts_marker == 0x00) {
self->next_packet.pts = 0;
plm_buffer_skip(self->buffer, 4);
self->next_packet.length -= 1;
}
else {
return NULL; // invalid
}
return plm_demux_get_packet(self);
}
plm_packet_t *plm_demux_get_packet(plm_demux_t *self) {
if (!plm_buffer_has(self->buffer, self->next_packet.length << 3)) {
return NULL;
}
self->current_packet.data = self->buffer->bytes + (self->buffer->bit_index >> 3);
self->current_packet.length = self->next_packet.length;
self->current_packet.type = self->next_packet.type;
self->current_packet.pts = self->next_packet.pts;
self->next_packet.length = 0;
return &self->current_packet;
}

View file

@ -1,29 +0,0 @@
#ifndef COSMOPOLITAN_DSP_MPEG_DEMUX_H_
#define COSMOPOLITAN_DSP_MPEG_DEMUX_H_
#include "dsp/mpeg/mpeg.h"
COSMOPOLITAN_C_START_
#define START_PACK 0xBA
#define START_END 0xB9
#define START_SYSTEM 0xBB
typedef struct plm_demux_t {
plm_buffer_t *buffer;
int destroy_buffer_when_done;
double system_clock_ref;
int has_pack_header;
int has_system_header;
int num_audio_streams;
int num_video_streams;
plm_packet_t current_packet;
plm_packet_t next_packet;
} plm_demux_t;
double plm_demux_read_time(plm_demux_t *self);
void plm_demux_decode_pack_header(plm_demux_t *self);
void plm_demux_decode_system_header(plm_demux_t *self);
plm_packet_t *plm_demux_decode_packet(plm_demux_t *self, int start_code);
plm_packet_t *plm_demux_get_packet(plm_demux_t *self);
COSMOPOLITAN_C_END_
#endif /* COSMOPOLITAN_DSP_MPEG_DEMUX_H_ */

View file

@ -1,101 +0,0 @@
/*-*- mode:c;indent-tabs-mode:t;c-basic-offset:4;tab-width:4;coding:utf-8 -*-│
vi: set et ft=c ts=4 sw=4 fenc=utf-8 :vi
PL_MPEG - MPEG1 Video decoder, MP2 Audio decoder, MPEG-PS demuxer
Dominic Szablewski - https://phoboslab.org │
The MIT License(MIT)
Copyright(c) 2019 Dominic Szablewski
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files(the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and / or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT.IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include "dsp/core/half.h"
__static_yoink("pl_mpeg_notice");
/**
* Computes Fixed-Point 8x8 Inverse Discrete Cosine Transform.
*
* @note discovered by Nasir Ahmed
*/
void plm_video_idct(int block[8][8]) {
int i, t1, t2, m0;
int b1, b3, b4, b6, b7;
int y3, y4, y5, y6, y7;
int x0, x1, x2, x3, x4;
for (i = 0; i < 8; ++i) {
b1 = block[4][i];
b3 = block[2][i] + block[6][i];
b4 = block[5][i] - block[3][i];
t1 = block[1][i] + block[7][i];
t2 = block[3][i] + block[5][i];
b6 = block[1][i] - block[7][i];
b7 = t1 + t2;
m0 = block[0][i];
x4 = ((b6 * 473 - b4 * 196 + 128) >> 8) - b7;
x0 = x4 - (((t1 - t2) * 362 + 128) >> 8);
x1 = m0 - b1;
x2 = (((block[2][i] - block[6][i]) * 362 + 128) >> 8) - b3;
x3 = m0 + b1;
y3 = x1 + x2;
y4 = x3 + b3;
y5 = x1 - x2;
y6 = x3 - b3;
y7 = -x0 - ((b4 * 473 + b6 * 196 + 128) >> 8);
block[0][i] = b7 + y4;
block[1][i] = x4 + y3;
block[2][i] = y5 - x0;
block[3][i] = y6 - y7;
block[4][i] = y6 + y7;
block[5][i] = x0 + y5;
block[6][i] = y3 - x4;
block[7][i] = y4 - b7;
}
for (i = 0; i < 8; ++i) {
b1 = block[i][4];
b3 = block[i][2] + block[i][6];
b4 = block[i][5] - block[i][3];
t1 = block[i][1] + block[i][7];
t2 = block[i][3] + block[i][5];
b6 = block[i][1] - block[i][7];
b7 = t1 + t2;
m0 = block[i][0];
x4 = ((b6 * 473 - b4 * 196 + 128) >> 8) - b7;
x0 = x4 - (((t1 - t2) * 362 + 128) >> 8);
x1 = m0 - b1;
x2 = (((block[i][2] - block[i][6]) * 362 + 128) >> 8) - b3;
x3 = m0 + b1;
y3 = x1 + x2;
y4 = x3 + b3;
y5 = x1 - x2;
y6 = x3 - b3;
y7 = -x0 - ((b4 * 473 + b6 * 196 + 128) >> 8);
block[i][0] = (b7 + y4 + 128) >> 8;
block[i][1] = (x4 + y3 + 128) >> 8;
block[i][2] = (y5 - x0 + 128) >> 8;
block[i][3] = (y6 - y7 + 128) >> 8;
block[i][4] = (y6 + y7 + 128) >> 8;
block[i][5] = (x0 + y5 + 128) >> 8;
block[i][6] = (y3 - x4 + 128) >> 8;
block[i][7] = (y4 - b7 + 128) >> 8;
}
}

View file

@ -1,8 +0,0 @@
#ifndef COSMOPOLITAN_DSP_MPEG_IDCT_H_
#define COSMOPOLITAN_DSP_MPEG_IDCT_H_
COSMOPOLITAN_C_START_
void plm_video_idct(int *);
COSMOPOLITAN_C_END_
#endif /* COSMOPOLITAN_DSP_MPEG_IDCT_H_ */

View file

@ -1,172 +0,0 @@
/*-*- mode:c;indent-tabs-mode:t;c-basic-offset:4;tab-width:4;coding:utf-8 -*-│
vi: set et ft=c ts=4 sw=4 fenc=utf-8 :vi
PL_MPEG - MPEG1 Video decoder, MP2 Audio decoder, MPEG-PS demuxer
Dominic Szablewski - https://phoboslab.org │
The MIT License(MIT)
Copyright(c) 2019 Dominic Szablewski
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files(the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and / or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT.IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include "dsp/mpeg/mpeg.h"
#include "dsp/mpeg/video.h"
#include "libc/log/check.h"
forceinline void plm_video_process_macroblock(plm_video_t *self,
uint8_t *restrict d,
uint8_t *restrict s, int motion_h,
int motion_v, bool interpolate,
unsigned BW) {
unsigned si, di, max_address;
int y, x, dest_scan, source_scan, dw, hp, vp, odd_h, odd_v;
dw = self->mb_width * BW;
hp = motion_h >> 1;
vp = motion_v >> 1;
odd_h = (motion_h & 1) == 1;
odd_v = (motion_v & 1) == 1;
si = ((self->mb_row * BW) + vp) * dw + (self->mb_col * BW) + hp;
di = (self->mb_row * dw + self->mb_col) * BW;
max_address = (dw * (self->mb_height * BW - BW + 1) - BW);
if (si > max_address || di > max_address)
return;
d += di;
s += si;
switch (((interpolate << 2) | (odd_h << 1) | (odd_v)) & 7) {
case 0:
dest_scan = dw - BW;
source_scan = dw - BW;
for (y = 0; y < BW; y++) {
for (x = 0; x < BW; x++) {
*d++ = *s++;
}
s += source_scan;
d += dest_scan;
}
break;
case 1:
dest_scan = dw - BW;
source_scan = dw - BW;
for (y = 0; y < BW; y++) {
for (x = 0; x < BW; x++) {
*d++ = (s[0] + s[dw] + 1) >> 1;
s++;
}
s += source_scan;
d += dest_scan;
}
break;
case 2:
dest_scan = dw - BW;
source_scan = dw - BW;
for (y = 0; y < BW; y++) {
for (x = 0; x < BW; x++) {
*d++ = (s[0] + s[1] + 1) >> 1;
s++;
}
s += source_scan;
d += dest_scan;
}
break;
case 3:
dest_scan = dw - BW;
source_scan = dw - BW;
for (y = 0; y < BW; y++) {
for (x = 0; x < BW; x++) {
*d++ = (s[0] + s[1] + s[dw] + s[dw + 1] + 2) >> 2;
s++;
}
s += source_scan;
d += dest_scan;
}
break;
case 4:
dest_scan = dw - BW;
source_scan = dw - BW;
for (y = 0; y < BW; y++) {
for (x = 0; x < BW; x++) {
d[0] = (d[0] + (s[0]) + 1) >> 1;
d++;
s++;
}
s += source_scan;
d += dest_scan;
}
break;
case 5:
dest_scan = dw - BW;
source_scan = dw - BW;
for (y = 0; y < BW; y++) {
for (x = 0; x < BW; x++) {
d[0] = (d[0] + ((s[0] + s[dw] + 1) >> 1) + 1) >> 1;
d++;
s++;
}
s += source_scan;
d += dest_scan;
}
break;
case 6:
dest_scan = dw - BW;
source_scan = dw - BW;
for (y = 0; y < BW; y++) {
for (x = 0; x < BW; x++) {
d[0] = (d[0] + ((s[0] + s[1] + 1) >> 1) + 1) >> 1;
d++;
s++;
}
s += source_scan;
d += dest_scan;
}
break;
case 7:
dest_scan = dw - BW;
source_scan = dw - BW;
for (y = 0; y < BW; y++) {
for (x = 0; x < BW; x++) {
d[0] = (d[0] + ((s[0] + s[1] + s[dw] + s[dw + 1] + 2) >> 2) + 1) >> 1;
d++;
s++;
}
s += source_scan;
d += dest_scan;
}
break;
default:
break;
}
}
void plm_video_process_macroblock_8(plm_video_t *self, uint8_t *restrict d,
uint8_t *restrict s, int motion_h,
int motion_v, bool interpolate) {
DCHECK_ALIGNED(8, d);
DCHECK_ALIGNED(8, s);
plm_video_process_macroblock(self, d, s, motion_h, motion_v, interpolate, 8);
}
void plm_video_process_macroblock_16(plm_video_t *self, uint8_t *restrict d,
uint8_t *restrict s, int motion_h,
int motion_v, bool interpolate) {
DCHECK_ALIGNED(16, d);
DCHECK_ALIGNED(16, s);
plm_video_process_macroblock(self, d, s, motion_h, motion_v, interpolate, 16);
}

View file

@ -1,769 +0,0 @@
/*-*- mode:c;indent-tabs-mode:t;c-basic-offset:4;tab-width:4;coding:utf-8 -*-│
vi: set noet ft=c ts=4 sw=4 fenc=utf-8 :vi
PL_MPEG - MPEG1 Video decoder, MP2 Audio decoder, MPEG-PS demuxer
Dominic Szablewski - https://phoboslab.org │
The MIT License(MIT)
Copyright(c) 2019 Dominic Szablewski
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files(the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and / or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT.IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include "dsp/mpeg/buffer.h"
#include "dsp/mpeg/mpeg.h"
#include "libc/log/log.h"
#include "libc/mem/mem.h"
#include "libc/str/str.h"
/* clang-format off */
// -----------------------------------------------------------------------------
// plm_audio implementation
// Based on kjmp2 by Martin J. Fiedler
// http://keyj.emphy.de/kjmp2/
#define PLM_AUDIO_FRAME_SYNC 0x7ff
#define PLM_AUDIO_MPEG_2_5 0x0
#define PLM_AUDIO_MPEG_2 0x2
#define PLM_AUDIO_MPEG_1 0x3
#define PLM_AUDIO_LAYER_III 0x1
#define PLM_AUDIO_LAYER_II 0x2
#define PLM_AUDIO_LAYER_I 0x3
#define PLM_AUDIO_MODE_STEREO 0x0
#define PLM_AUDIO_MODE_JOINT_STEREO 0x1
#define PLM_AUDIO_MODE_DUAL_CHANNEL 0x2
#define PLM_AUDIO_MODE_MONO 0x3
static const unsigned short PLM_AUDIO_SAMPLE_RATE[] = {
44100, 48000, 32000, 0, // MPEG-1
22050, 24000, 16000, 0 // MPEG-2
};
static const short PLM_AUDIO_BIT_RATE[] = {
32, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, 384, // MPEG-1
8, 16, 24, 32, 40, 48, 56, 64, 80, 96, 112, 128, 144, 160 // MPEG-2
};
static const int PLM_AUDIO_SCALEFACTOR_BASE[] = {
0x02000000, 0x01965FEA, 0x01428A30
};
static const float PLM_AUDIO_SYNTHESIS_WINDOW[] = {
0.0, -0.5, -0.5, -0.5, -0.5, -0.5,
-0.5, -1.0, -1.0, -1.0, -1.0, -1.5,
-1.5, -2.0, -2.0, -2.5, -2.5, -3.0,
-3.5, -3.5, -4.0, -4.5, -5.0, -5.5,
-6.5, -7.0, -8.0, -8.5, -9.5, -10.5,
-12.0, -13.0, -14.5, -15.5, -17.5, -19.0,
-20.5, -22.5, -24.5, -26.5, -29.0, -31.5,
-34.0, -36.5, -39.5, -42.5, -45.5, -48.5,
-52.0, -55.5, -58.5, -62.5, -66.0, -69.5,
-73.5, -77.0, -80.5, -84.5, -88.0, -91.5,
-95.0, -98.0, -101.0, -104.0, 106.5, 109.0,
111.0, 112.5, 113.5, 114.0, 114.0, 113.5,
112.0, 110.5, 107.5, 104.0, 100.0, 94.5,
88.5, 81.5, 73.0, 63.5, 53.0, 41.5,
28.5, 14.5, -1.0, -18.0, -36.0, -55.5,
-76.5, -98.5, -122.0, -147.0, -173.5, -200.5,
-229.5, -259.5, -290.5, -322.5, -355.5, -389.5,
-424.0, -459.5, -495.5, -532.0, -568.5, -605.0,
-641.5, -678.0, -714.0, -749.0, -783.5, -817.0,
-849.0, -879.5, -908.5, -935.0, -959.5, -981.0,
-1000.5, -1016.0, -1028.5, -1037.5, -1042.5, -1043.5,
-1040.0, -1031.5, 1018.5, 1000.0, 976.0, 946.5,
911.0, 869.5, 822.0, 767.5, 707.0, 640.0,
565.5, 485.0, 397.0, 302.5, 201.0, 92.5,
-22.5, -144.0, -272.5, -407.0, -547.5, -694.0,
-846.0, -1003.0, -1165.0, -1331.5, -1502.0, -1675.5,
-1852.5, -2031.5, -2212.5, -2394.0, -2576.5, -2758.5,
-2939.5, -3118.5, -3294.5, -3467.5, -3635.5, -3798.5,
-3955.0, -4104.5, -4245.5, -4377.5, -4499.0, -4609.5,
-4708.0, -4792.5, -4863.5, -4919.0, -4958.0, -4979.5,
-4983.0, -4967.5, -4931.5, -4875.0, -4796.0, -4694.5,
-4569.5, -4420.0, -4246.0, -4046.0, -3820.0, -3567.0,
3287.0, 2979.5, 2644.0, 2280.5, 1888.0, 1467.5,
1018.5, 541.0, 35.0, -499.0, -1061.0, -1650.0,
-2266.5, -2909.0, -3577.0, -4270.0, -4987.5, -5727.5,
-6490.0, -7274.0, -8077.5, -8899.5, -9739.0, -10594.5,
-11464.5, -12347.0, -13241.0, -14144.5, -15056.0, -15973.5,
-16895.5, -17820.0, -18744.5, -19668.0, -20588.0, -21503.0,
-22410.5, -23308.5, -24195.0, -25068.5, -25926.5, -26767.0,
-27589.0, -28389.0, -29166.5, -29919.0, -30644.5, -31342.0,
-32009.5, -32645.0, -33247.0, -33814.5, -34346.0, -34839.5,
-35295.0, -35710.0, -36084.5, -36417.5, -36707.5, -36954.0,
-37156.5, -37315.0, -37428.0, -37496.0, 37519.0, 37496.0,
37428.0, 37315.0, 37156.5, 36954.0, 36707.5, 36417.5,
36084.5, 35710.0, 35295.0, 34839.5, 34346.0, 33814.5,
33247.0, 32645.0, 32009.5, 31342.0, 30644.5, 29919.0,
29166.5, 28389.0, 27589.0, 26767.0, 25926.5, 25068.5,
24195.0, 23308.5, 22410.5, 21503.0, 20588.0, 19668.0,
18744.5, 17820.0, 16895.5, 15973.5, 15056.0, 14144.5,
13241.0, 12347.0, 11464.5, 10594.5, 9739.0, 8899.5,
8077.5, 7274.0, 6490.0, 5727.5, 4987.5, 4270.0,
3577.0, 2909.0, 2266.5, 1650.0, 1061.0, 499.0,
-35.0, -541.0, -1018.5, -1467.5, -1888.0, -2280.5,
-2644.0, -2979.5, 3287.0, 3567.0, 3820.0, 4046.0,
4246.0, 4420.0, 4569.5, 4694.5, 4796.0, 4875.0,
4931.5, 4967.5, 4983.0, 4979.5, 4958.0, 4919.0,
4863.5, 4792.5, 4708.0, 4609.5, 4499.0, 4377.5,
4245.5, 4104.5, 3955.0, 3798.5, 3635.5, 3467.5,
3294.5, 3118.5, 2939.5, 2758.5, 2576.5, 2394.0,
2212.5, 2031.5, 1852.5, 1675.5, 1502.0, 1331.5,
1165.0, 1003.0, 846.0, 694.0, 547.5, 407.0,
272.5, 144.0, 22.5, -92.5, -201.0, -302.5,
-397.0, -485.0, -565.5, -640.0, -707.0, -767.5,
-822.0, -869.5, -911.0, -946.5, -976.0, -1000.0,
1018.5, 1031.5, 1040.0, 1043.5, 1042.5, 1037.5,
1028.5, 1016.0, 1000.5, 981.0, 959.5, 935.0,
908.5, 879.5, 849.0, 817.0, 783.5, 749.0,
714.0, 678.0, 641.5, 605.0, 568.5, 532.0,
495.5, 459.5, 424.0, 389.5, 355.5, 322.5,
290.5, 259.5, 229.5, 200.5, 173.5, 147.0,
122.0, 98.5, 76.5, 55.5, 36.0, 18.0,
1.0, -14.5, -28.5, -41.5, -53.0, -63.5,
-73.0, -81.5, -88.5, -94.5, -100.0, -104.0,
-107.5, -110.5, -112.0, -113.5, -114.0, -114.0,
-113.5, -112.5, -111.0, -109.0, 106.5, 104.0,
101.0, 98.0, 95.0, 91.5, 88.0, 84.5,
80.5, 77.0, 73.5, 69.5, 66.0, 62.5,
58.5, 55.5, 52.0, 48.5, 45.5, 42.5,
39.5, 36.5, 34.0, 31.5, 29.0, 26.5,
24.5, 22.5, 20.5, 19.0, 17.5, 15.5,
14.5, 13.0, 12.0, 10.5, 9.5, 8.5,
8.0, 7.0, 6.5, 5.5, 5.0, 4.5,
4.0, 3.5, 3.5, 3.0, 2.5, 2.5,
2.0, 2.0, 1.5, 1.5, 1.0, 1.0,
1.0, 1.0, 0.5, 0.5, 0.5, 0.5,
0.5, 0.5
};
// Quantizer lookup, step 1: bitrate classes
static const uint8_t PLM_AUDIO_QUANT_LUT_STEP_1[2][16] = {
// 32, 48, 56, 64, 80, 96,112,128,160,192,224,256,320,384 <- bitrate
{ 0, 0, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2 }, // mono
// 16, 24, 28, 32, 40, 48, 56, 64, 80, 96,112,128,160,192 <- bitrate / chan
{ 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2, 2 } // stereo
};
// Quantizer lookup, step 2: bitrate class, sample rate -> B2 table idx, sblimit
static const uint8_t PLM_AUDIO_QUANT_TAB_A = (27 | 64); // Table 3-B.2a: high-rate, sblimit = 27
static const uint8_t PLM_AUDIO_QUANT_TAB_B = (30 | 64); // Table 3-B.2b: high-rate, sblimit = 30
static const uint8_t PLM_AUDIO_QUANT_TAB_C = 8; // Table 3-B.2c: low-rate, sblimit = 8
static const uint8_t PLM_AUDIO_QUANT_TAB_D = 12; // Table 3-B.2d: low-rate, sblimit = 12
static const uint8_t QUANT_LUT_STEP_2[3][3] = {
// 44.1 kHz, 48 kHz, 32 kHz
{ PLM_AUDIO_QUANT_TAB_C, PLM_AUDIO_QUANT_TAB_C, PLM_AUDIO_QUANT_TAB_D }, // 32 - 48 kbit/sec/ch
{ PLM_AUDIO_QUANT_TAB_A, PLM_AUDIO_QUANT_TAB_A, PLM_AUDIO_QUANT_TAB_A }, // 56 - 80 kbit/sec/ch
{ PLM_AUDIO_QUANT_TAB_B, PLM_AUDIO_QUANT_TAB_A, PLM_AUDIO_QUANT_TAB_B } // 96+ kbit/sec/ch
};
// Quantizer lookup, step 3: B2 table, subband -> nbal, row index
// (upper 4 bits: nbal, lower 4 bits: row index)
static const uint8_t PLM_AUDIO_QUANT_LUT_STEP_3[3][32] = {
// Low-rate table (3-B.2c and 3-B.2d)
{
0x44,0x44,
0x34,0x34,0x34,0x34,0x34,0x34,0x34,0x34,0x34,0x34
},
// High-rate table (3-B.2a and 3-B.2b)
{
0x43,0x43,0x43,
0x42,0x42,0x42,0x42,0x42,0x42,0x42,0x42,
0x31,0x31,0x31,0x31,0x31,0x31,0x31,0x31,0x31,0x31,0x31,0x31,
0x20,0x20,0x20,0x20,0x20,0x20,0x20
},
// MPEG-2 LSR table (B.2 in ISO 13818-3)
{
0x45,0x45,0x45,0x45,
0x34,0x34,0x34,0x34,0x34,0x34,0x34,
0x24,0x24,0x24,0x24,0x24,0x24,0x24,0x24,0x24,0x24,
0x24,0x24,0x24,0x24,0x24,0x24,0x24,0x24,0x24
}
};
// Quantizer lookup, step 4: table row, allocation[] value -> quant table index
static const uint8_t PLM_AUDIO_QUANT_LUT_STEP4[6][16] = {
{ 0, 1, 2, 17 },
{ 0, 1, 2, 3, 4, 5, 6, 17 },
{ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17 },
{ 0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17 },
{ 0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17 },
{ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 }
};
typedef struct plm_quantizer_spec_t {
unsigned short levels;
unsigned char group;
unsigned char bits;
} plm_quantizer_spec_t;
static const plm_quantizer_spec_t PLM_AUDIO_QUANT_TAB[] = {
{ 3, 1, 5 }, // 1
{ 5, 1, 7 }, // 2
{ 7, 0, 3 }, // 3
{ 9, 1, 10 }, // 4
{ 15, 0, 4 }, // 5
{ 31, 0, 5 }, // 6
{ 63, 0, 6 }, // 7
{ 127, 0, 7 }, // 8
{ 255, 0, 8 }, // 9
{ 511, 0, 9 }, // 10
{ 1023, 0, 10 }, // 11
{ 2047, 0, 11 }, // 12
{ 4095, 0, 12 }, // 13
{ 8191, 0, 13 }, // 14
{ 16383, 0, 14 }, // 15
{ 32767, 0, 15 }, // 16
{ 65535, 0, 16 } // 17
};
struct plm_audio_t {
double time;
int samples_decoded;
int samplerate_index;
int bitrate_index;
int version;
int layer;
int mode;
int bound;
int v_pos;
int next_frame_data_size;
plm_buffer_t *buffer;
int destroy_buffer_when_done;
const plm_quantizer_spec_t *allocation[2][32];
uint8_t scale_factor_info[2][32];
int scale_factor[2][32][3];
int sample[2][32][3];
plm_samples_t samples;
float D[1024];
float V[1024];
float U[32];
} forcealign(64);
typedef plm_audio_t plm_audio_t;
int plm_audio_decode_header(plm_audio_t *self);
void plm_audio_decode_frame(plm_audio_t *self);
const plm_quantizer_spec_t *plm_audio_read_allocation(plm_audio_t *self, int sb, int tab3);
void plm_audio_read_samples(plm_audio_t *self, int ch, int sb, int part);
void plm_audio_matrix_transform(int s[32][3], int ss, float *d, int dp);
plm_audio_t *plm_audio_create_with_buffer(plm_buffer_t *buffer, int destroy_when_done) {
plm_audio_t *self = (plm_audio_t *)memalign(_Alignof(plm_audio_t), sizeof(plm_audio_t));
memset(self, 0, sizeof(plm_audio_t));
self->samples.count = PLM_AUDIO_SAMPLES_PER_FRAME;
self->buffer = buffer;
self->destroy_buffer_when_done = destroy_when_done;
self->samplerate_index = 3; // indicates 0 samplerate
memcpy(self->D, PLM_AUDIO_SYNTHESIS_WINDOW, 512 * sizeof(float));
memcpy(self->D + 512, PLM_AUDIO_SYNTHESIS_WINDOW, 512 * sizeof(float));
// Decode first header
if (plm_buffer_has(self->buffer, 48)) {
self->next_frame_data_size = plm_audio_decode_header(self);
}
return self;
}
void plm_audio_destroy(plm_audio_t *self) {
if (self->destroy_buffer_when_done) {
plm_buffer_destroy(self->buffer);
}
free(self);
}
int plm_audio_get_samplerate(plm_audio_t *self) {
return PLM_AUDIO_SAMPLE_RATE[self->samplerate_index];
}
double plm_audio_get_time(plm_audio_t *self) {
return self->time;
}
void plm_audio_rewind(plm_audio_t *self) {
plm_buffer_rewind(self->buffer);
self->time = 0;
self->samples_decoded = 0;
self->next_frame_data_size = 0;
// TODO: needed?
memset(self->V, 0, sizeof(self->V));
memset(self->U, 0, sizeof(self->U));
}
plm_samples_t *plm_audio_decode(plm_audio_t *self) {
DEBUGF("%s", "plm_audio_decode");
// Do we have at least enough information to decode the frame header?
if (!self->next_frame_data_size) {
if (!plm_buffer_has(self->buffer, 48)) {
return NULL;
}
self->next_frame_data_size = plm_audio_decode_header(self);
}
if (
self->next_frame_data_size == 0 ||
!plm_buffer_has(self->buffer, self->next_frame_data_size << 3)
) {
return NULL;
}
plm_audio_decode_frame(self);
self->next_frame_data_size = 0;
self->samples.time = self->time;
self->samples_decoded += PLM_AUDIO_SAMPLES_PER_FRAME;
self->time = (double)self->samples_decoded /
(double)PLM_AUDIO_SAMPLE_RATE[self->samplerate_index];
return &self->samples;
}
int plm_audio_decode_header(plm_audio_t *self) {
// Check for valid header: syncword OK, MPEG-Audio Layer 2
plm_buffer_skip_bytes(self->buffer, 0x00);
int sync = plm_buffer_read(self->buffer, 11);
self->version = plm_buffer_read(self->buffer, 2);
self->layer = plm_buffer_read(self->buffer, 2);
int hasCRC = !plm_buffer_read(self->buffer, 1);
if (
sync != PLM_AUDIO_FRAME_SYNC ||
self->version != PLM_AUDIO_MPEG_1 ||
self->layer != PLM_AUDIO_LAYER_II
) {
return false; // Invalid header or unsupported version
}
self->bitrate_index = plm_buffer_read(self->buffer, 4) - 1;
if (self->bitrate_index > 13) {
return false; // Invalid bit rate or 'free format'
}
self->samplerate_index = plm_buffer_read(self->buffer, 2);
if (self->samplerate_index == 3) {
return false; // Invalid sample rate
}
if (self->version == PLM_AUDIO_MPEG_2) {
self->samplerate_index += 4;
self->bitrate_index += 14;
}
int padding = plm_buffer_read(self->buffer, 1);
plm_buffer_skip(self->buffer, 1); // f_private
self->mode = plm_buffer_read(self->buffer, 2);
// Parse the mode_extension, set up the stereo bound
self->bound = 0;
if (self->mode == PLM_AUDIO_MODE_JOINT_STEREO) {
self->bound = (plm_buffer_read(self->buffer, 2) + 1) << 2;
}
else {
plm_buffer_skip(self->buffer, 2);
self->bound = (self->mode == PLM_AUDIO_MODE_MONO) ? 0 : 32;
}
// Discard the last 4 bits of the header and the CRC value, if present
plm_buffer_skip(self->buffer, 4);
if (hasCRC) {
plm_buffer_skip(self->buffer, 16);
}
// Compute frame size, check if we have enough data to decode the whole
// frame.
int bitrate = PLM_AUDIO_BIT_RATE[self->bitrate_index];
int samplerate = PLM_AUDIO_SAMPLE_RATE[self->samplerate_index];
int frame_size = (144000 * bitrate / samplerate) + padding;
return frame_size - (hasCRC ? 6 : 4);
}
void plm_audio_decode_frame(plm_audio_t *self) {
// Prepare the quantizer table lookups
int tab3 = 0;
int sblimit = 0;
if (self->version == PLM_AUDIO_MPEG_2) {
// MPEG-2 (LSR)
tab3 = 2;
sblimit = 30;
}
else {
// MPEG-1
int tab1 = (self->mode == PLM_AUDIO_MODE_MONO) ? 0 : 1;
int tab2 = PLM_AUDIO_QUANT_LUT_STEP_1[tab1][self->bitrate_index];
tab3 = QUANT_LUT_STEP_2[tab2][self->samplerate_index];
sblimit = tab3 & 63;
tab3 >>= 6;
}
if (self->bound > sblimit) {
self->bound = sblimit;
}
// Read the allocation information
for (int sb = 0; sb < self->bound; sb++) {
self->allocation[0][sb] = plm_audio_read_allocation(self, sb, tab3);
self->allocation[1][sb] = plm_audio_read_allocation(self, sb, tab3);
}
for (int sb = self->bound; sb < sblimit; sb++) {
self->allocation[0][sb] =
self->allocation[1][sb] =
plm_audio_read_allocation(self, sb, tab3);
}
// Read scale factor selector information
int channels = (self->mode == PLM_AUDIO_MODE_MONO) ? 1 : 2;
for (int sb = 0; sb < sblimit; sb++) {
for (int ch = 0; ch < channels; ch++) {
if (self->allocation[ch][sb]) {
self->scale_factor_info[ch][sb] = plm_buffer_read(self->buffer, 2);
}
}
if (self->mode == PLM_AUDIO_MODE_MONO) {
self->scale_factor_info[1][sb] = self->scale_factor_info[0][sb];
}
}
// Read scale factors
for (int sb = 0; sb < sblimit; sb++) {
for (int ch = 0; ch < channels; ch++) {
if (self->allocation[ch][sb]) {
int *sf = self->scale_factor[ch][sb];
switch (self->scale_factor_info[ch][sb]) {
case 0:
sf[0] = plm_buffer_read(self->buffer, 6);
sf[1] = plm_buffer_read(self->buffer, 6);
sf[2] = plm_buffer_read(self->buffer, 6);
break;
case 1:
sf[0] =
sf[1] = plm_buffer_read(self->buffer, 6);
sf[2] = plm_buffer_read(self->buffer, 6);
break;
case 2:
sf[0] =
sf[1] =
sf[2] = plm_buffer_read(self->buffer, 6);
break;
case 3:
sf[0] = plm_buffer_read(self->buffer, 6);
sf[1] =
sf[2] = plm_buffer_read(self->buffer, 6);
break;
}
}
}
if (self->mode == PLM_AUDIO_MODE_MONO) {
self->scale_factor[1][sb][0] = self->scale_factor[0][sb][0];
self->scale_factor[1][sb][1] = self->scale_factor[0][sb][1];
self->scale_factor[1][sb][2] = self->scale_factor[0][sb][2];
}
}
// Coefficient input and reconstruction
int out_pos = 0;
for (int part = 0; part < 3; part++) {
for (int granule = 0; granule < 4; granule++) {
// Read the samples
for (int sb = 0; sb < self->bound; sb++) {
plm_audio_read_samples(self, 0, sb, part);
plm_audio_read_samples(self, 1, sb, part);
}
for (int sb = self->bound; sb < sblimit; sb++) {
plm_audio_read_samples(self, 0, sb, part);
self->sample[1][sb][0] = self->sample[0][sb][0];
self->sample[1][sb][1] = self->sample[0][sb][1];
self->sample[1][sb][2] = self->sample[0][sb][2];
}
for (int sb = sblimit; sb < 32; sb++) {
self->sample[0][sb][0] = 0;
self->sample[0][sb][1] = 0;
self->sample[0][sb][2] = 0;
self->sample[1][sb][0] = 0;
self->sample[1][sb][1] = 0;
self->sample[1][sb][2] = 0;
}
// Synthesis loop
for (int p = 0; p < 3; p++) {
// Shifting step
self->v_pos = (self->v_pos - 64) & 1023;
for (int ch = 0; ch < 2; ch++) {
plm_audio_matrix_transform(self->sample[ch], p, self->V, self->v_pos);
// Build U, windowing, calculate output
memset(self->U, 0, sizeof(self->U));
int d_index = 512 - (self->v_pos >> 1);
int v_index = (self->v_pos % 128) >> 1;
while (v_index < 1024) {
for (int i = 0; i < 32; ++i) {
self->U[i] += self->D[d_index++] * self->V[v_index++];
}
v_index += 128 - 32;
d_index += 64 - 32;
}
d_index -= (512 - 32);
v_index = (128 - 32 + 1024) - v_index;
while (v_index < 1024) {
for (int i = 0; i < 32; ++i) {
self->U[i] += self->D[d_index++] * self->V[v_index++];
}
v_index += 128 - 32;
d_index += 64 - 32;
}
// Output samples
#ifdef PLM_AUDIO_SEPARATE_CHANNELS
float *out_channel = ch == 0
? self->samples.left
: self->samples.right;
for (int j = 0; j < 32; j++) {
out_channel[out_pos + j] = self->U[j] / 2147418112.0f;
}
#else
for (int j = 0; j < 32; j++) {
self->samples.interleaved[((out_pos + j) << 1) + ch] =
self->U[j] / 2147418112.0f;
}
#endif
} // End of synthesis channel loop
out_pos += 32;
} // End of synthesis sub-block loop
} // Decoding of the granule finished
}
plm_buffer_align(self->buffer);
}
const plm_quantizer_spec_t *plm_audio_read_allocation(plm_audio_t *self, int sb, int tab3) {
int tab4 = PLM_AUDIO_QUANT_LUT_STEP_3[tab3][sb];
int qtab = PLM_AUDIO_QUANT_LUT_STEP4[tab4 & 15][plm_buffer_read(self->buffer, tab4 >> 4)];
return qtab ? (&PLM_AUDIO_QUANT_TAB[qtab - 1]) : 0;
}
void plm_audio_read_samples(plm_audio_t *self, int ch, int sb, int part) {
const plm_quantizer_spec_t *q = self->allocation[ch][sb];
int sf = self->scale_factor[ch][sb][part];
int *sample = self->sample[ch][sb];
int val = 0;
if (!q) {
// No bits allocated for this subband
sample[0] = sample[1] = sample[2] = 0;
return;
}
// Resolve scalefactor
if (sf == 63) {
sf = 0;
}
else {
int shift = (sf / 3) | 0;
sf = (PLM_AUDIO_SCALEFACTOR_BASE[sf % 3] + ((1u << shift) >> 1)) >> shift;
}
// Decode samples
int adj = q->levels;
if (q->group) {
// Decode grouped samples
val = plm_buffer_read(self->buffer, q->bits);
sample[0] = val % adj;
val /= adj;
sample[1] = val % adj;
sample[2] = val / adj;
}
else {
// Decode direct samples
sample[0] = plm_buffer_read(self->buffer, q->bits);
sample[1] = plm_buffer_read(self->buffer, q->bits);
sample[2] = plm_buffer_read(self->buffer, q->bits);
}
// Postmultiply samples
int scale = 65536 / (adj + 1);
adj = ((adj + 1) >> 1) - 1;
val = (adj - sample[0]) * scale;
sample[0] = (val * (sf >> 12) + ((val * (sf & 4095) + 2048) >> 12)) >> 12;
val = (adj - sample[1]) * scale;
sample[1] = (val * (sf >> 12) + ((val * (sf & 4095) + 2048) >> 12)) >> 12;
val = (adj - sample[2]) * scale;
sample[2] = (val * (sf >> 12) + ((val * (sf & 4095) + 2048) >> 12)) >> 12;
}
void plm_audio_matrix_transform(int s[32][3], int ss, float *d, int dp) {
float t01, t02, t03, t04, t05, t06, t07, t08, t09, t10, t11, t12,
t13, t14, t15, t16, t17, t18, t19, t20, t21, t22, t23, t24,
t25, t26, t27, t28, t29, t30, t31, t32, t33;
t01 = (float)(s[0][ss] + s[31][ss]); t02 = (float)(s[0][ss] - s[31][ss]) * 0.500602998235f;
t03 = (float)(s[1][ss] + s[30][ss]); t04 = (float)(s[1][ss] - s[30][ss]) * 0.505470959898f;
t05 = (float)(s[2][ss] + s[29][ss]); t06 = (float)(s[2][ss] - s[29][ss]) * 0.515447309923f;
t07 = (float)(s[3][ss] + s[28][ss]); t08 = (float)(s[3][ss] - s[28][ss]) * 0.53104259109f;
t09 = (float)(s[4][ss] + s[27][ss]); t10 = (float)(s[4][ss] - s[27][ss]) * 0.553103896034f;
t11 = (float)(s[5][ss] + s[26][ss]); t12 = (float)(s[5][ss] - s[26][ss]) * 0.582934968206f;
t13 = (float)(s[6][ss] + s[25][ss]); t14 = (float)(s[6][ss] - s[25][ss]) * 0.622504123036f;
t15 = (float)(s[7][ss] + s[24][ss]); t16 = (float)(s[7][ss] - s[24][ss]) * 0.674808341455f;
t17 = (float)(s[8][ss] + s[23][ss]); t18 = (float)(s[8][ss] - s[23][ss]) * 0.744536271002f;
t19 = (float)(s[9][ss] + s[22][ss]); t20 = (float)(s[9][ss] - s[22][ss]) * 0.839349645416f;
t21 = (float)(s[10][ss] + s[21][ss]); t22 = (float)(s[10][ss] - s[21][ss]) * 0.972568237862f;
t23 = (float)(s[11][ss] + s[20][ss]); t24 = (float)(s[11][ss] - s[20][ss]) * 1.16943993343f;
t25 = (float)(s[12][ss] + s[19][ss]); t26 = (float)(s[12][ss] - s[19][ss]) * 1.48416461631f;
t27 = (float)(s[13][ss] + s[18][ss]); t28 = (float)(s[13][ss] - s[18][ss]) * 2.05778100995f;
t29 = (float)(s[14][ss] + s[17][ss]); t30 = (float)(s[14][ss] - s[17][ss]) * 3.40760841847f;
t31 = (float)(s[15][ss] + s[16][ss]); t32 = (float)(s[15][ss] - s[16][ss]) * 10.1900081235f;
t33 = t01 + t31; t31 = (t01 - t31) * 0.502419286188f;
t01 = t03 + t29; t29 = (t03 - t29) * 0.52249861494f;
t03 = t05 + t27; t27 = (t05 - t27) * 0.566944034816f;
t05 = t07 + t25; t25 = (t07 - t25) * 0.64682178336f;
t07 = t09 + t23; t23 = (t09 - t23) * 0.788154623451f;
t09 = t11 + t21; t21 = (t11 - t21) * 1.06067768599f;
t11 = t13 + t19; t19 = (t13 - t19) * 1.72244709824f;
t13 = t15 + t17; t17 = (t15 - t17) * 5.10114861869f;
t15 = t33 + t13; t13 = (t33 - t13) * 0.509795579104f;
t33 = t01 + t11; t01 = (t01 - t11) * 0.601344886935f;
t11 = t03 + t09; t09 = (t03 - t09) * 0.899976223136f;
t03 = t05 + t07; t07 = (t05 - t07) * 2.56291544774f;
t05 = t15 + t03; t15 = (t15 - t03) * 0.541196100146f;
t03 = t33 + t11; t11 = (t33 - t11) * 1.30656296488f;
t33 = t05 + t03; t05 = (t05 - t03) * 0.707106781187f;
t03 = t15 + t11; t15 = (t15 - t11) * 0.707106781187f;
t03 += t15;
t11 = t13 + t07; t13 = (t13 - t07) * 0.541196100146f;
t07 = t01 + t09; t09 = (t01 - t09) * 1.30656296488f;
t01 = t11 + t07; t07 = (t11 - t07) * 0.707106781187f;
t11 = t13 + t09; t13 = (t13 - t09) * 0.707106781187f;
t11 += t13; t01 += t11;
t11 += t07; t07 += t13;
t09 = t31 + t17; t31 = (t31 - t17) * 0.509795579104f;
t17 = t29 + t19; t29 = (t29 - t19) * 0.601344886935f;
t19 = t27 + t21; t21 = (t27 - t21) * 0.899976223136f;
t27 = t25 + t23; t23 = (t25 - t23) * 2.56291544774f;
t25 = t09 + t27; t09 = (t09 - t27) * 0.541196100146f;
t27 = t17 + t19; t19 = (t17 - t19) * 1.30656296488f;
t17 = t25 + t27; t27 = (t25 - t27) * 0.707106781187f;
t25 = t09 + t19; t19 = (t09 - t19) * 0.707106781187f;
t25 += t19;
t09 = t31 + t23; t31 = (t31 - t23) * 0.541196100146f;
t23 = t29 + t21; t21 = (t29 - t21) * 1.30656296488f;
t29 = t09 + t23; t23 = (t09 - t23) * 0.707106781187f;
t09 = t31 + t21; t31 = (t31 - t21) * 0.707106781187f;
t09 += t31; t29 += t09; t09 += t23; t23 += t31;
t17 += t29; t29 += t25; t25 += t09; t09 += t27;
t27 += t23; t23 += t19; t19 += t31;
t21 = t02 + t32; t02 = (t02 - t32) * 0.502419286188f;
t32 = t04 + t30; t04 = (t04 - t30) * 0.52249861494f;
t30 = t06 + t28; t28 = (t06 - t28) * 0.566944034816f;
t06 = t08 + t26; t08 = (t08 - t26) * 0.64682178336f;
t26 = t10 + t24; t10 = (t10 - t24) * 0.788154623451f;
t24 = t12 + t22; t22 = (t12 - t22) * 1.06067768599f;
t12 = t14 + t20; t20 = (t14 - t20) * 1.72244709824f;
t14 = t16 + t18; t16 = (t16 - t18) * 5.10114861869f;
t18 = t21 + t14; t14 = (t21 - t14) * 0.509795579104f;
t21 = t32 + t12; t32 = (t32 - t12) * 0.601344886935f;
t12 = t30 + t24; t24 = (t30 - t24) * 0.899976223136f;
t30 = t06 + t26; t26 = (t06 - t26) * 2.56291544774f;
t06 = t18 + t30; t18 = (t18 - t30) * 0.541196100146f;
t30 = t21 + t12; t12 = (t21 - t12) * 1.30656296488f;
t21 = t06 + t30; t30 = (t06 - t30) * 0.707106781187f;
t06 = t18 + t12; t12 = (t18 - t12) * 0.707106781187f;
t06 += t12;
t18 = t14 + t26; t26 = (t14 - t26) * 0.541196100146f;
t14 = t32 + t24; t24 = (t32 - t24) * 1.30656296488f;
t32 = t18 + t14; t14 = (t18 - t14) * 0.707106781187f;
t18 = t26 + t24; t24 = (t26 - t24) * 0.707106781187f;
t18 += t24; t32 += t18;
t18 += t14; t26 = t14 + t24;
t14 = t02 + t16; t02 = (t02 - t16) * 0.509795579104f;
t16 = t04 + t20; t04 = (t04 - t20) * 0.601344886935f;
t20 = t28 + t22; t22 = (t28 - t22) * 0.899976223136f;
t28 = t08 + t10; t10 = (t08 - t10) * 2.56291544774f;
t08 = t14 + t28; t14 = (t14 - t28) * 0.541196100146f;
t28 = t16 + t20; t20 = (t16 - t20) * 1.30656296488f;
t16 = t08 + t28; t28 = (t08 - t28) * 0.707106781187f;
t08 = t14 + t20; t20 = (t14 - t20) * 0.707106781187f;
t08 += t20;
t14 = t02 + t10; t02 = (t02 - t10) * 0.541196100146f;
t10 = t04 + t22; t22 = (t04 - t22) * 1.30656296488f;
t04 = t14 + t10; t10 = (t14 - t10) * 0.707106781187f;
t14 = t02 + t22; t02 = (t02 - t22) * 0.707106781187f;
t14 += t02; t04 += t14; t14 += t10; t10 += t02;
t16 += t04; t04 += t08; t08 += t14; t14 += t28;
t28 += t10; t10 += t20; t20 += t02; t21 += t16;
t16 += t32; t32 += t04; t04 += t06; t06 += t08;
t08 += t18; t18 += t14; t14 += t30; t30 += t28;
t28 += t26; t26 += t10; t10 += t12; t12 += t20;
t20 += t24; t24 += t02;
d[dp + 48] = -t33;
d[dp + 49] = d[dp + 47] = -t21;
d[dp + 50] = d[dp + 46] = -t17;
d[dp + 51] = d[dp + 45] = -t16;
d[dp + 52] = d[dp + 44] = -t01;
d[dp + 53] = d[dp + 43] = -t32;
d[dp + 54] = d[dp + 42] = -t29;
d[dp + 55] = d[dp + 41] = -t04;
d[dp + 56] = d[dp + 40] = -t03;
d[dp + 57] = d[dp + 39] = -t06;
d[dp + 58] = d[dp + 38] = -t25;
d[dp + 59] = d[dp + 37] = -t08;
d[dp + 60] = d[dp + 36] = -t11;
d[dp + 61] = d[dp + 35] = -t18;
d[dp + 62] = d[dp + 34] = -t09;
d[dp + 63] = d[dp + 33] = -t14;
d[dp + 32] = -t05;
d[dp + 0] = t05; d[dp + 31] = -t30;
d[dp + 1] = t30; d[dp + 30] = -t27;
d[dp + 2] = t27; d[dp + 29] = -t28;
d[dp + 3] = t28; d[dp + 28] = -t07;
d[dp + 4] = t07; d[dp + 27] = -t26;
d[dp + 5] = t26; d[dp + 26] = -t23;
d[dp + 6] = t23; d[dp + 25] = -t10;
d[dp + 7] = t10; d[dp + 24] = -t15;
d[dp + 8] = t15; d[dp + 23] = -t12;
d[dp + 9] = t12; d[dp + 22] = -t19;
d[dp + 10] = t19; d[dp + 21] = -t20;
d[dp + 11] = t20; d[dp + 20] = -t13;
d[dp + 12] = t13; d[dp + 19] = -t24;
d[dp + 13] = t24; d[dp + 18] = -t31;
d[dp + 14] = t31; d[dp + 17] = -t02;
d[dp + 15] = t02; d[dp + 16] = 0.0;
};

View file

@ -1,447 +0,0 @@
#ifndef COSMOPOLITAN_DSP_MPEG_MPEG_H_
#define COSMOPOLITAN_DSP_MPEG_MPEG_H_
#include "libc/stdio/stdio.h"
COSMOPOLITAN_C_START_
typedef struct plm_t plm_t;
typedef struct plm_buffer_t plm_buffer_t;
typedef struct plm_demux_t plm_demux_t;
typedef struct plm_video_t plm_video_t;
typedef struct plm_audio_t plm_audio_t;
/**
* Demuxed MPEG PS packet
*
* The type maps directly to the various MPEG-PES start codes. pts is
* the presentation time stamp of the packet in seconds. Not all packets
* have a pts value.
*/
typedef struct plm_packet_t {
int type;
double pts;
size_t length;
uint8_t *data;
} plm_packet_t;
/**
* Decoded Video Plane
*
* The byte length of the data is width * height. Note that different
* planes have different sizes: the Luma plane (Y) is double the size of
* each of the two Chroma planes (Cr, Cb) - i.e. 4 times the byte
* length. Also note that the size of the plane does *not* denote the
* size of the displayed frame. The sizes of planes are always rounded
* up to the nearest macroblock (16px).
*/
typedef struct plm_plane_t {
unsigned int width;
unsigned int height;
uint8_t *data;
} plm_plane_t;
/**
* Decoded Video Frame
*
* Width and height denote the desired display size of the frame. This
* may be different from the internal size of the 3 planes.
*/
typedef struct plm_frame_t {
double time;
unsigned int width;
unsigned int height;
plm_plane_t y;
plm_plane_t cr;
plm_plane_t cb;
} plm_frame_t;
/**
* Callback function type for decoded video frames used by the high-level
* plm_* interface
*/
typedef void (*plm_video_decode_callback)(plm_t *self, plm_frame_t *frame,
void *user);
/**
* Decoded Audio Samples
*
* Samples are stored as normalized (-1, 1) float either interleaved, or if
* PLM_AUDIO_SEPARATE_CHANNELS is defined, in two separate arrays.
* The `count` is always PLM_AUDIO_SAMPLES_PER_FRAME and just there for
* convenience.
*/
#define PLM_AUDIO_SAMPLES_PER_FRAME 1152
struct plm_samples_t {
double time;
unsigned int count;
#ifdef PLM_AUDIO_SEPARATE_CHANNELS
float left[PLM_AUDIO_SAMPLES_PER_FRAME] forcealign(32);
float right[PLM_AUDIO_SAMPLES_PER_FRAME] forcealign(32);
#else
float interleaved[PLM_AUDIO_SAMPLES_PER_FRAME * 2] forcealign(32);
#endif
} forcealign(32);
typedef struct plm_samples_t plm_samples_t;
/**
* Callback function type for decoded audio samples used by the high-level
* plm_* interface
*/
typedef void (*plm_audio_decode_callback)(plm_t *self, plm_samples_t *samples,
void *user);
/**
* Callback function for plm_buffer when it needs more data
*/
typedef void (*plm_buffer_load_callback)(plm_buffer_t *self, void *user);
/**
* -----------------------------------------------------------------------------
* plm_* public API
* High-Level API for loading/demuxing/decoding MPEG-PS data
*
* Create a plmpeg instance with a filename. Returns NULL if the file could not
* be opened.
*/
plm_t *plm_create_with_filename(const char *filename);
/**
* Create a plmpeg instance with file handle. Pass true to close_when_done
* to let plmpeg call fclose() on the handle when plm_destroy() is
* called.
*/
plm_t *plm_create_with_file(FILE *fh, int close_when_done);
/**
* Create a plmpeg instance with pointer to memory as source. This assumes the
* whole file is in memory. Pass true to free_when_done to let plmpeg call
* free() on the pointer when plm_destroy() is called.
*/
plm_t *plm_create_with_memory(uint8_t *bytes, size_t length,
int free_when_done);
/**
* Create a plmpeg instance with a plm_buffer as source. This is also
* called internally by all the above constructor functions.
*/
plm_t *plm_create_with_buffer(plm_buffer_t *buffer, int destroy_when_done);
/**
* Destroy a plmpeg instance and free all data
*/
void plm_destroy(plm_t *self);
/**
* Get or set whether video decoding is enabled.
*/
int plm_get_video_enabled(plm_t *self);
void plm_set_video_enabled(plm_t *self, int enabled);
/**
* Get or set whether audio decoding is enabled. When enabling, you can set the
* desired audio stream (0-3) to decode.
*/
int plm_get_audio_enabled(plm_t *self);
void plm_set_audio_enabled(plm_t *self, int enabled, int stream_index);
/**
* Get the display width/height of the video stream
*/
int plm_get_width(plm_t *self);
int plm_get_height(plm_t *self);
double plm_get_pixel_aspect_ratio(plm_t *);
/**
* Get the framerate of the video stream in frames per second
*/
double plm_get_framerate(plm_t *self);
/**
* Get the number of available audio streams in the file
*/
int plm_get_num_audio_streams(plm_t *self);
/**
* Get the samplerate of the audio stream in samples per second
*/
int plm_get_samplerate(plm_t *self);
/**
* Get or set the audio lead time in seconds - the time in which audio samples
* are decoded in advance (or behind) the video decode time. Default 0.
*/
double plm_get_audio_lead_time(plm_t *self);
void plm_set_audio_lead_time(plm_t *self, double lead_time);
/**
* Get the current internal time in seconds
*/
double plm_get_time(plm_t *self);
/**
* Rewind all buffers back to the beginning.
*/
void plm_rewind(plm_t *self);
/**
* Get or set looping. Default false.
*/
int plm_get_loop(plm_t *self);
void plm_set_loop(plm_t *self, int loop);
/**
* Get whether the file has ended. If looping is enabled, this will always
* return false.
*/
int plm_has_ended(plm_t *self);
/**
* Set the callback for decoded video frames used with plm_decode(). If no
* callback is set, video data will be ignored and not be decoded.
*/
void plm_set_video_decode_callback(plm_t *self, plm_video_decode_callback fp,
void *user);
/**
* Set the callback for decoded audio samples used with plm_decode(). If no
* callback is set, audio data will be ignored and not be decoded.
*/
void plm_set_audio_decode_callback(plm_t *self, plm_audio_decode_callback fp,
void *user);
/**
* Advance the internal timer by seconds and decode video/audio up to
* this time. Returns true/false whether anything was decoded.
*/
int plm_decode(plm_t *self, double seconds);
/**
* Decode and return one video frame. Returns NULL if no frame could be decoded
* (either because the source ended or data is corrupt). If you only want to
* decode video, you should disable audio via plm_set_audio_enabled().
* The returned plm_frame_t is valid until the next call to
* plm_decode_video call or until the plm_destroy is called.
*/
plm_frame_t *plm_decode_video(plm_t *self);
/**
* Decode and return one audio frame. Returns NULL if no frame could be decoded
* (either because the source ended or data is corrupt). If you only want to
* decode audio, you should disable video via plm_set_video_enabled().
* The returned plm_samples_t is valid until the next call to
* plm_decode_video or until the plm_destroy is called.
*/
plm_samples_t *plm_decode_audio(plm_t *self);
/* -----------------------------------------------------------------------------
* plm_buffer public API
* Provides the data source for all other plm_* interfaces
*
* The default size for buffers created from files or by the high-level API
*/
#ifndef PLM_BUFFER_DEFAULT_SIZE
#define PLM_BUFFER_DEFAULT_SIZE (128 * 1024)
#endif
/**
* Create a buffer instance with a filename. Returns NULL if the file could not
* be opened.
*/
plm_buffer_t *plm_buffer_create_with_filename(const char *filename);
/**
* Create a buffer instance with file handle. Pass true to close_when_done
* to let plmpeg call fclose() on the handle when plm_destroy() is
* called.
*/
plm_buffer_t *plm_buffer_create_with_file(FILE *fh, int close_when_done);
/**
* Create a buffer instance with a pointer to memory as source. This assumes
* the whole file is in memory. Pass 1 to free_when_done to let plmpeg call
* free() on the pointer when plm_destroy() is called.
*/
plm_buffer_t *plm_buffer_create_with_memory(uint8_t *bytes, size_t length,
int free_when_done);
/**
* Create an empty buffer with an initial capacity. The buffer will grow
* as needed.
*/
plm_buffer_t *plm_buffer_create_with_capacity(size_t capacity);
/**
* Destroy a buffer instance and free all data
*/
void plm_buffer_destroy(plm_buffer_t *self);
/**
* Copy data into the buffer. If the data to be written is larger than the
* available space, the buffer will realloc() with a larger capacity.
* Returns the number of bytes written. This will always be the same as the
* passed in length, except when the buffer was created _with_memory() for
* which _write() is forbidden.
*/
size_t plm_buffer_write(plm_buffer_t *self, uint8_t *bytes, size_t length);
/**
* Set a callback that is called whenever the buffer needs more data
*/
void plm_buffer_set_load_callback(plm_buffer_t *self,
plm_buffer_load_callback fp, void *user);
/**
* Rewind the buffer back to the beginning. When loading from a file handle,
* this also seeks to the beginning of the file.
*/
void plm_buffer_rewind(plm_buffer_t *self);
/**
* -----------------------------------------------------------------------------
* plm_demux public API
* Demux an MPEG Program Stream (PS) data into separate packages
*
* Various Packet Types
*/
#define PLM_DEMUX_PACKET_PRIVATE 0xBD
#define PLM_DEMUX_PACKET_AUDIO_1 0xC0
#define PLM_DEMUX_PACKET_AUDIO_2 0xC1
#define PLM_DEMUX_PACKET_AUDIO_3 0xC2
#define PLM_DEMUX_PACKET_AUDIO_4 0xC2
#define PLM_DEMUX_PACKET_VIDEO_1 0xE0
/**
* Create a demuxer with a plm_buffer as source
*/
plm_demux_t *plm_demux_create(plm_buffer_t *buffer, int destroy_when_done);
/**
* Destroy a demuxer and free all data
*/
void plm_demux_destroy(plm_demux_t *self);
/**
* Returns the number of video streams found in the system header.
*/
int plm_demux_get_num_video_streams(plm_demux_t *self);
/**
* Returns the number of audio streams found in the system header.
*/
int plm_demux_get_num_audio_streams(plm_demux_t *self);
/**
* Rewinds the internal buffer. See plm_buffer_rewind().
*/
void plm_demux_rewind(plm_demux_t *self);
/**
* Decode and return the next packet. The returned packet_t is valid until
* the next call to plm_demux_decode() or until the demuxer is destroyed.
*/
plm_packet_t *plm_demux_decode(plm_demux_t *self);
/* -----------------------------------------------------------------------------
* plm_video public API
* Decode MPEG1 Video ("mpeg1") data into raw YCrCb frames
*/
/**
* Create a video decoder with a plm_buffer as source
*/
plm_video_t *plm_video_create_with_buffer(plm_buffer_t *buffer,
int destroy_when_done);
/**
* Destroy a video decoder and free all data
*/
void plm_video_destroy(plm_video_t *self);
/**
* Get the framerate in frames per second
*/
double plm_video_get_framerate(plm_video_t *);
double plm_video_get_pixel_aspect_ratio(plm_video_t *);
/**
* Get the display width/height
*/
int plm_video_get_width(plm_video_t *);
int plm_video_get_height(plm_video_t *);
/**
* Set "no delay" mode. When enabled, the decoder assumes that the video does
* *not* contain any B-Frames. This is useful for reducing lag when streaming.
*/
void plm_video_set_no_delay(plm_video_t *self, int no_delay);
/**
* Get the current internal time in seconds
*/
double plm_video_get_time(plm_video_t *self);
/**
* Rewinds the internal buffer. See plm_buffer_rewind().
*/
void plm_video_rewind(plm_video_t *self);
/**
* Decode and return one frame of video and advance the internal time by
* 1/framerate seconds. The returned frame_t is valid until the next call of
* plm_video_decode() or until the video decoder is destroyed.
*/
plm_frame_t *plm_video_decode(plm_video_t *self);
/**
* Convert the YCrCb data of a frame into an interleaved RGB buffer. The buffer
* pointed to by *rgb must have a size of (frame->width * frame->height * 3)
* bytes.
*/
void plm_frame_to_rgb(plm_frame_t *frame, uint8_t *rgb);
/* -----------------------------------------------------------------------------
* plm_audio public API
* Decode MPEG-1 Audio Layer II ("mp2") data into raw samples
*/
/**
* Create an audio decoder with a plm_buffer as source.
*/
plm_audio_t *plm_audio_create_with_buffer(plm_buffer_t *buffer,
int destroy_when_done);
/**
* Destroy an audio decoder and free all data
*/
void plm_audio_destroy(plm_audio_t *self);
/**
* Get the samplerate in samples per second
*/
int plm_audio_get_samplerate(plm_audio_t *self);
/**
* Get the current internal time in seconds
*/
double plm_audio_get_time(plm_audio_t *self);
/**
* Rewinds the internal buffer. See plm_buffer_rewind().
*/
void plm_audio_rewind(plm_audio_t *self);
/**
* Decode and return one "frame" of audio and advance the internal time by
* (PLM_AUDIO_SAMPLES_PER_FRAME/samplerate) seconds. The returned samples_t
* is valid until the next call of plm_audio_decode() or until the audio
* decoder is destroyed.
*/
plm_samples_t *plm_audio_decode(plm_audio_t *self);
extern long plmpegdecode_latency_;
COSMOPOLITAN_C_END_
#endif /* COSMOPOLITAN_DSP_MPEG_MPEG_H_ */

File diff suppressed because it is too large Load diff

View file

@ -2,3 +2,8 @@ __notice(pl_mpeg_notice, "\
PL_MPEG (MIT License)\n\
Copyright(c) 2019 Dominic Szablewski\n\
https://phoboslab.org");
long plmpegdecode_latency_;
#define PL_MPEG_IMPLEMENTATION
#include "pl_mpeg.h"

4379
dsp/mpeg/pl_mpeg.h Executable file

File diff suppressed because it is too large Load diff

View file

@ -1,332 +0,0 @@
/*-*- mode:c;indent-tabs-mode:t;c-basic-offset:4;tab-width:4;coding:utf-8 -*-│
vi: set noet ft=c ts=4 sw=4 fenc=utf-8 :vi
PL_MPEG - MPEG1 Video decoder, MP2 Audio decoder, MPEG-PS demuxer
Dominic Szablewski - https://phoboslab.org │
The MIT License(MIT)
Copyright(c) 2019 Dominic Szablewski
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files(the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and / or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT.IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include "dsp/mpeg/mpeg.h"
#include "libc/log/log.h"
#include "libc/mem/mem.h"
#include "libc/stdio/stdio.h"
#include "libc/str/str.h"
__static_yoink("pl_mpeg_notice");
/* clang-format off */
// -----------------------------------------------------------------------------
// plm (high-level interface) implementation
typedef struct plm_t {
plm_demux_t *demux;
double time;
int has_ended;
int loop;
int video_packet_type;
plm_buffer_t *video_buffer;
plm_video_t *video_decoder;
int audio_packet_type;
double audio_lead_time;
plm_buffer_t *audio_buffer;
plm_audio_t *audio_decoder;
plm_video_decode_callback video_decode_callback;
void *video_decode_callback_user_data;
plm_audio_decode_callback audio_decode_callback;
void *audio_decode_callback_user_data;
} plm_t;
void plm_handle_end(plm_t *self);
void plm_read_video_packet(plm_buffer_t *buffer, void *user);
void plm_read_audio_packet(plm_buffer_t *buffer, void *user);
void plm_read_packets(plm_t *self, int requested_type);
plm_t *plm_create_with_filename(const char *filename) {
plm_buffer_t *buffer = plm_buffer_create_with_filename(filename);
if (!buffer) {
return NULL;
}
return plm_create_with_buffer(buffer, true);
}
plm_t *plm_create_with_file(FILE *fh, int close_when_done) {
plm_buffer_t *buffer = plm_buffer_create_with_file(fh, close_when_done);
return plm_create_with_buffer(buffer, true);
}
plm_t *plm_create_with_memory(uint8_t *bytes, size_t length, int free_when_done) {
plm_buffer_t *buffer = plm_buffer_create_with_memory(bytes, length, free_when_done);
return plm_create_with_buffer(buffer, true);
}
plm_t *plm_create_with_buffer(plm_buffer_t *buffer, int destroy_when_done) {
plm_t *self = (plm_t *)malloc(sizeof(plm_t));
memset(self, 0, sizeof(plm_t));
self->demux = plm_demux_create(buffer, destroy_when_done);
// In theory we should check plm_demux_get_num_video_streams() and
// plm_demux_get_num_audio_streams() here, but older files typically
// do not specify these correctly. So we just assume we have a video and
// audio stream and create the decoders.
self->video_packet_type = PLM_DEMUX_PACKET_VIDEO_1;
self->video_buffer = plm_buffer_create_with_capacity(PLM_BUFFER_DEFAULT_SIZE);
plm_buffer_set_load_callback(self->video_buffer, plm_read_video_packet, self);
self->audio_packet_type = PLM_DEMUX_PACKET_AUDIO_1;
self->audio_buffer = plm_buffer_create_with_capacity(PLM_BUFFER_DEFAULT_SIZE);
plm_buffer_set_load_callback(self->audio_buffer, plm_read_audio_packet, self);
self->video_decoder = plm_video_create_with_buffer(self->video_buffer, true);
self->audio_decoder = plm_audio_create_with_buffer(self->audio_buffer, true);
return self;
}
void plm_destroy(plm_t *self) {
plm_video_destroy(self->video_decoder);
plm_audio_destroy(self->audio_decoder);
plm_demux_destroy(self->demux);
free(self);
}
int plm_get_audio_enabled(plm_t *self) {
return (self->audio_packet_type != 0);
}
void plm_set_audio_enabled(plm_t *self, int enabled, int stream_index) {
/* int num_streams = plm_demux_get_num_audio_streams(self->demux); */
self->audio_packet_type = (enabled && stream_index >= 0 && stream_index < 4)
? PLM_DEMUX_PACKET_AUDIO_1 + stream_index
: 0;
}
int plm_get_video_enabled(plm_t *self) {
return (self->video_packet_type != 0);
}
void plm_set_video_enabled(plm_t *self, int enabled) {
self->video_packet_type = (enabled)
? PLM_DEMUX_PACKET_VIDEO_1
: 0;
}
int plm_get_width(plm_t *self) {
return plm_video_get_width(self->video_decoder);
}
double plm_get_pixel_aspect_ratio(plm_t *self) {
return plm_video_get_pixel_aspect_ratio(self->video_decoder);
}
int plm_get_height(plm_t *self) {
return plm_video_get_height(self->video_decoder);
}
double plm_get_framerate(plm_t *self) {
return plm_video_get_framerate(self->video_decoder);
}
int plm_get_num_audio_streams(plm_t *self) {
// Some files do not specify the number of audio streams in the system header.
// If the reported number of streams is 0, we check if we have a samplerate,
// indicating at least one audio stream.
int num_streams = plm_demux_get_num_audio_streams(self->demux);
return num_streams == 0 && plm_get_samplerate(self) ? 1 : num_streams;
}
int plm_get_samplerate(plm_t *self) {
return plm_audio_get_samplerate(self->audio_decoder);
}
double plm_get_audio_lead_time(plm_t *self) {
return self->audio_lead_time;
}
void plm_set_audio_lead_time(plm_t *self, double lead_time) {
self->audio_lead_time = lead_time;
}
double plm_get_time(plm_t *self) {
return self->time;
}
void plm_rewind(plm_t *self) {
plm_video_rewind(self->video_decoder);
plm_audio_rewind(self->audio_decoder);
plm_demux_rewind(self->demux);
self->time = 0;
}
int plm_get_loop(plm_t *self) {
return self->loop;
}
void plm_set_loop(plm_t *self, int loop) {
self->loop = loop;
}
int plm_has_ended(plm_t *self) {
return self->has_ended;
}
void plm_set_video_decode_callback(plm_t *self, plm_video_decode_callback fp, void *user) {
self->video_decode_callback = fp;
self->video_decode_callback_user_data = user;
}
void plm_set_audio_decode_callback(plm_t *self, plm_audio_decode_callback fp, void *user) {
self->audio_decode_callback = fp;
self->audio_decode_callback_user_data = user;
}
int plm_decode(plm_t *self, double tick) {
DEBUGF("%s", "plm_decode");
int decode_video = (self->video_decode_callback && self->video_packet_type);
int decode_audio = (self->audio_decode_callback && self->audio_packet_type);
if (!decode_video && !decode_audio) {
// Nothing to do here
return false;
}
int did_decode = false;
int video_ended = false;
int audio_ended = false;
double video_target_time = self->time + tick;
double audio_target_time = self->time + tick;
if (self->audio_lead_time > 0 && decode_audio) {
video_target_time -= self->audio_lead_time;
}
else {
audio_target_time -= self->audio_lead_time;
}
do {
did_decode = false;
if (decode_video && plm_video_get_time(self->video_decoder) < video_target_time) {
plm_frame_t *frame = plm_video_decode(self->video_decoder);
if (frame) {
self->video_decode_callback(self, frame, self->video_decode_callback_user_data);
did_decode = true;
}
else {
video_ended = true;
}
}
if (decode_audio && plm_audio_get_time(self->audio_decoder) < audio_target_time) {
plm_samples_t *samples = plm_audio_decode(self->audio_decoder);
if (samples) {
self->audio_decode_callback(self, samples, self->audio_decode_callback_user_data);
did_decode = true;
}
else {
audio_ended = true;
}
}
} while (did_decode);
// We wanted to decode something but failed -> the source must have ended
if ((!decode_video || video_ended) && (!decode_audio || audio_ended)) {
plm_handle_end(self);
}
else {
self->time += tick;
}
return did_decode ? true : false;
}
plm_frame_t *plm_decode_video(plm_t *self) {
if (!self->video_packet_type) {
return NULL;
}
plm_frame_t *frame = plm_video_decode(self->video_decoder);
if (frame) {
self->time = frame->time;
}
else {
plm_handle_end(self);
}
return frame;
}
plm_samples_t *plm_decode_audio(plm_t *self) {
if (!self->audio_packet_type) {
return NULL;
}
plm_samples_t *samples = plm_audio_decode(self->audio_decoder);
if (samples) {
self->time = samples->time;
}
else {
plm_handle_end(self);
}
return samples;
}
void plm_handle_end(plm_t *self) {
if (self->loop) {
plm_rewind(self);
}
else {
self->has_ended = true;
}
}
void plm_read_video_packet(plm_buffer_t *buffer, void *user) {
plm_t *self = (plm_t *)user;
plm_read_packets(self, self->video_packet_type);
}
void plm_read_audio_packet(plm_buffer_t *buffer, void *user) {
plm_t *self = (plm_t *)user;
plm_read_packets(self, self->audio_packet_type);
}
void plm_read_packets(plm_t *self, int requested_type) {
plm_packet_t *packet;
while ((packet = plm_demux_decode(self->demux))) {
if (packet->type == self->video_packet_type) {
plm_buffer_write(self->video_buffer, packet->data, packet->length);
}
else if (packet->type == self->audio_packet_type) {
plm_buffer_write(self->audio_buffer, packet->data, packet->length);
}
if (packet->type == requested_type) {
return;
}
}
}

View file

@ -1,83 +0,0 @@
/*-*- mode:c;indent-tabs-mode:t;c-basic-offset:4;tab-width:4;coding:utf-8 -*-│
vi: set et ft=c ts=4 sw=4 fenc=utf-8 :vi
PL_MPEG - MPEG1 Video decoder, MP2 Audio decoder, MPEG-PS demuxer
Dominic Szablewski - https://phoboslab.org │
The MIT License(MIT)
Copyright(c) 2019 Dominic Szablewski
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files(the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and / or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT.IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include "dsp/mpeg/mpeg.h"
#include "libc/macros.internal.h"
__static_yoink("pl_mpeg_notice");
/**
* @see YCbCr2RGB() in tool/viz/lib/ycbcr2rgb.c
*/
void plm_frame_to_rgb(plm_frame_t *frame, uint8_t *rgb) {
// Chroma values are the same for each block of 4 pixels, so we process
// 2 lines at a time, 2 neighboring pixels each.
int w = frame->y.width, w2 = w >> 1;
int y_index1 = 0, y_index2 = w, y_next_2_lines = w + (w - frame->width);
int c_index = 0, c_next_line = w2 - (frame->width >> 1);
int rgb_index1 = 0, rgb_index2 = frame->width * 3,
rgb_next_2_lines = frame->width * 3;
int cols = frame->width >> 1, rows = frame->height >> 1;
int ccb, ccr, r, g, b;
uint8_t *y = frame->y.data, *cb = frame->cb.data, *cr = frame->cr.data;
for (int row = 0; row < rows; row++) {
for (int col = 0; col < cols; col++) {
ccb = cb[c_index];
ccr = cr[c_index];
c_index++;
r = (ccr + ((ccr * 103) >> 8)) - 179;
g = ((ccb * 88) >> 8) - 44 + ((ccr * 183) >> 8) - 91;
b = (ccb + ((ccb * 198) >> 8)) - 227;
// Line 1
int y1 = y[y_index1++];
int y2 = y[y_index1++];
rgb[rgb_index1 + 0] = MAX(0, MIN(255, y1 + r));
rgb[rgb_index1 + 1] = MAX(0, MIN(255, y1 - g));
rgb[rgb_index1 + 2] = MAX(0, MIN(255, y1 + b));
rgb[rgb_index1 + 3] = MAX(0, MIN(255, y2 + r));
rgb[rgb_index1 + 4] = MAX(0, MIN(255, y2 - g));
rgb[rgb_index1 + 5] = MAX(0, MIN(255, y2 + b));
rgb_index1 += 6;
// Line 2
int y3 = y[y_index2++];
int y4 = y[y_index2++];
rgb[rgb_index2 + 0] = MAX(0, MIN(255, y3 + r));
rgb[rgb_index2 + 1] = MAX(0, MIN(255, y3 - g));
rgb[rgb_index2 + 2] = MAX(0, MIN(255, y3 + b));
rgb[rgb_index2 + 3] = MAX(0, MIN(255, y4 + r));
rgb[rgb_index2 + 4] = MAX(0, MIN(255, y4 - g));
rgb[rgb_index2 + 5] = MAX(0, MIN(255, y4 + b));
rgb_index2 += 6;
}
y_index1 += y_next_2_lines;
y_index2 += y_next_2_lines;
rgb_index1 += rgb_next_2_lines;
rgb_index2 += rgb_next_2_lines;
c_index += c_next_line;
}
}

View file

@ -1,60 +0,0 @@
#ifndef COSMOPOLITAN_DSP_MPEG_VIDEO_H_
#define COSMOPOLITAN_DSP_MPEG_VIDEO_H_
#include "dsp/mpeg/mpeg.h"
COSMOPOLITAN_C_START_
typedef struct {
int full_px;
int is_set;
int r_size;
int h;
int v;
} plm_video_motion_t;
typedef struct plm_video_t {
double framerate;
double time;
double pixel_aspect_ratio;
int frames_decoded;
int width;
int height;
int mb_width;
int mb_height;
int mb_size;
int luma_width;
int luma_height;
int chroma_width;
int chroma_height;
int start_code;
int picture_type;
plm_video_motion_t motion_forward;
plm_video_motion_t motion_backward;
int has_sequence_header;
int quantizer_scale;
int slice_begin;
int macroblock_address;
int mb_row;
int mb_col;
int macroblock_type;
int macroblock_intra;
int dc_predictor[3];
plm_buffer_t *buffer;
int destroy_buffer_when_done;
plm_frame_t frame_current;
plm_frame_t frame_forward;
plm_frame_t frame_backward;
uint8_t *frames_data;
int block_data[64];
uint8_t intra_quant_matrix[64];
uint8_t non_intra_quant_matrix[64];
int has_reference_frame;
int assume_no_b_frames;
} plm_video_t;
void plm_video_process_macroblock_8(plm_video_t *, uint8_t *restrict,
uint8_t *restrict, int, int, bool);
void plm_video_process_macroblock_16(plm_video_t *, uint8_t *restrict,
uint8_t *restrict, int, int, bool);
COSMOPOLITAN_C_END_
#endif /* COSMOPOLITAN_DSP_MPEG_VIDEO_H_ */

43
dsp/prog/BUILD.mk Normal file
View file

@ -0,0 +1,43 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#── vi: set noet ft=make ts=8 sw=8 fenc=utf-8 :vi ────────────────────┘
PKGS += DSP_PROG
DSP_PROG_FILES := $(wildcard dsp/prog/*)
DSP_PROG_HDRS = $(filter %.h,$(DSP_PROG_FILES))
DSP_PROG_SRCS = $(filter %.c,$(DSP_PROG_FILES))
DSP_PROG_OBJS = $(DSP_PROG_SRCS:%.c=o/$(MODE)/%.o)
DSP_PROG_COMS = $(DSP_PROG_SRCS:%.c=o/$(MODE)/%)
DSP_PROG_BINS = $(DSP_PROG_COMS) $(DSP_PROG_COMS:%=%.dbg)
DSP_PROG_DIRECTDEPS = \
DSP_AUDIO \
LIBC_CALLS \
LIBC_INTRIN \
LIBC_NEXGEN32E \
LIBC_RUNTIME \
LIBC_SOCK \
LIBC_STDIO \
LIBC_SYSV \
LIBC_TINYMATH \
THIRD_PARTY_MUSL \
DSP_PROG_DEPS := \
$(call uniq,$(foreach x,$(DSP_PROG_DIRECTDEPS),$($(x))))
o/$(MODE)/dsp/prog/prog.pkg: \
$(DSP_PROG_OBJS) \
$(foreach x,$(DSP_PROG_DIRECTDEPS),$($(x)_A).pkg)
o/$(MODE)/dsp/prog/%.dbg: \
$(DSP_PROG_DEPS) \
o/$(MODE)/dsp/prog/prog.pkg \
o/$(MODE)/dsp/prog/%.o \
$(CRT) \
$(APE_NO_MODIFY_SELF)
@$(APELINK)
$(DSP_PROG_OBJS): dsp/prog/BUILD.mk
.PHONY: o/$(MODE)/dsp/prog
o/$(MODE)/dsp/prog: $(DSP_PROG_BINS)

41
dsp/prog/loudness.h Normal file
View file

@ -0,0 +1,41 @@
#ifndef COSMOPOLITAN_DSP_PROG_LOUDNESS_H_
#define COSMOPOLITAN_DSP_PROG_LOUDNESS_H_
#include <math.h>
#include <stdio.h>
#define MIN_DECIBEL -60
#define MAX_DECIBEL 0
// computes root of mean squares
static double rms(float *p, int n) {
double s = 0;
for (int i = 0; i < n; ++i)
s += p[i] * p[i];
return sqrt(s / n);
}
// converts rms to decibel
static double rms_to_db(double rms) {
double db = 20 * log10(rms);
db = fmin(db, MAX_DECIBEL);
db = fmax(db, MIN_DECIBEL);
return db;
}
// char meter[21];
// format_decibel_meter(meter, 20, rms_to_db(rms(samps, count)))
static char *format_decibel_meter(char *meter, int width, double db) {
double range = MAX_DECIBEL - MIN_DECIBEL;
int filled = (db - MIN_DECIBEL) / range * width;
for (int i = 0; i < width; ++i) {
if (i < filled) {
meter[i] = '=';
} else {
meter[i] = ' ';
}
}
meter[width] = 0;
return meter;
}
#endif /* COSMOPOLITAN_DSP_PROG_LOUDNESS_H_ */

127
dsp/prog/recvaudio.c Normal file
View file

@ -0,0 +1,127 @@
#if 0
/*─────────────────────────────────────────────────────────────────╗
To the extent possible under law, Justine Tunney has waived
all copyright and related or neighboring rights to this file,
as it is written in the following disclaimers:
http://unlicense.org/ │
http://creativecommons.org/publicdomain/zero/1.0/ │
*/
#endif
#include <arpa/inet.h>
#include <assert.h>
#include <cosmoaudio.h>
#include <errno.h>
#include <math.h>
#include <netinet/in.h>
#include <signal.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <time.h>
#include "loudness.h"
/**
* @fileoverview plays audio from remote computer on speaker
* @see dsp/prog/sendaudio.c
*/
#define SAMPLING_RATE 44100
#define FRAMES_PER_SECOND 60
#define DEBUG_LOG 0
#define PORT 9834
#define CHUNK_FRAMES (SAMPLING_RATE / FRAMES_PER_SECOND)
static_assert(CHUNK_FRAMES * sizeof(short) < 1472,
"audio chunks won't fit in udp ethernet packet");
sig_atomic_t g_done;
void onsig(int sig) {
g_done = 1;
}
short toshort(float x) {
return fmaxf(-1, fminf(1, x)) * 32767;
}
float tofloat(short x) {
return x / 32768.f;
}
int main(int argc, char* argv[]) {
// listen on udp port for audio
int server;
if ((server = socket(AF_INET, SOCK_DGRAM, 0)) == -1) {
perror("socket");
return 3;
}
struct sockaddr_in addr = {.sin_family = AF_INET, .sin_port = htons(PORT)};
if (bind(server, (struct sockaddr*)&addr, sizeof(addr))) {
perror("bind");
return 4;
}
// setup signals
struct sigaction sa;
sa.sa_flags = 0;
sa.sa_handler = onsig;
sigemptyset(&sa.sa_mask);
sigaction(SIGINT, &sa, 0);
// configure cosmo audio
struct CosmoAudioOpenOptions cao = {0};
cao.sizeofThis = sizeof(struct CosmoAudioOpenOptions);
cao.deviceType = kCosmoAudioDeviceTypePlayback;
cao.sampleRate = SAMPLING_RATE;
cao.bufferFrames = CHUNK_FRAMES * 2;
cao.debugLog = DEBUG_LOG;
cao.channels = 1;
// connect to microphone and speaker
int status;
struct CosmoAudio* ca;
status = cosmoaudio_open(&ca, &cao);
if (status != COSMOAUDIO_SUCCESS) {
fprintf(stderr, "failed to open audio: %d\n", status);
return 5;
}
while (!g_done) {
// read from network
ssize_t got;
short buf16[CHUNK_FRAMES];
if ((got = read(server, buf16, CHUNK_FRAMES * sizeof(short))) == -1) {
if (errno == EINTR)
continue;
perror("read");
return 7;
}
if (got != CHUNK_FRAMES * sizeof(short)) {
fprintf(stderr, "warning: got partial audio frame\n");
continue;
}
// write to speaker
float buf32[CHUNK_FRAMES];
for (int i = 0; i < CHUNK_FRAMES; ++i)
buf32[i] = tofloat(buf16[i]);
cosmoaudio_poll(ca, 0, (int[]){CHUNK_FRAMES});
cosmoaudio_write(ca, buf32, CHUNK_FRAMES);
// print loudness in ascii
char meter[21];
double db = rms_to_db(rms(buf32, CHUNK_FRAMES));
format_decibel_meter(meter, 20, db);
printf("\r%s| %+6.2f dB", meter, db);
fflush(stdout);
}
// clean up resources
cosmoaudio_flush(ca);
cosmoaudio_close(ca);
close(server);
}

149
dsp/prog/sendaudio.c Normal file
View file

@ -0,0 +1,149 @@
#if 0
/*─────────────────────────────────────────────────────────────────╗
To the extent possible under law, Justine Tunney has waived
all copyright and related or neighboring rights to this file,
as it is written in the following disclaimers:
http://unlicense.org/ │
http://creativecommons.org/publicdomain/zero/1.0/ │
*/
#endif
#include <arpa/inet.h>
#include <assert.h>
#include <cosmoaudio.h>
#include <errno.h>
#include <math.h>
#include <netdb.h>
#include <netinet/in.h>
#include <signal.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <time.h>
#include "loudness.h"
/**
* @fileoverview sends audio from microphone to remote computer
* @see dsp/prog/recvaudio.c
*/
#define SAMPLING_RATE 44100
#define FRAMES_PER_SECOND 60
#define DEBUG_LOG 0
#define PORT 9834
#define CHUNK_FRAMES (SAMPLING_RATE / FRAMES_PER_SECOND)
static_assert(CHUNK_FRAMES * sizeof(short) < 1472,
"audio chunks won't fit in udp ethernet packet");
sig_atomic_t g_done;
void onsig(int sig) {
g_done = 1;
}
short toshort(float x) {
return fmaxf(-1, fminf(1, x)) * 32767;
}
float tofloat(short x) {
return x / 32768.f;
}
uint32_t host2ip(const char* host) {
uint32_t ip;
if ((ip = inet_addr(host)) != -1u)
return ip;
int rc;
struct addrinfo* ai = NULL;
struct addrinfo hint = {AI_NUMERICSERV, AF_INET, SOCK_STREAM, IPPROTO_TCP};
if ((rc = getaddrinfo(host, "0", &hint, &ai))) {
fprintf(stderr, "%s: %s\n", host, gai_strerror(rc));
exit(50 + rc);
}
ip = ntohl(((struct sockaddr_in*)ai->ai_addr)->sin_addr.s_addr);
freeaddrinfo(ai);
return ip;
}
int main(int argc, char* argv[]) {
if (argc != 2) {
fprintf(stderr, "%s: missing host argument\n", argv[0]);
return 1;
}
// get host argument
const char* remote_host = argv[1];
uint32_t ip = host2ip(remote_host);
// connect to server
int client;
if ((client = socket(AF_INET, SOCK_DGRAM, 0)) == -1) {
perror(remote_host);
return 3;
}
struct sockaddr_in addr = {.sin_family = AF_INET,
.sin_port = htons(PORT),
.sin_addr.s_addr = htonl(ip)};
if (connect(client, (struct sockaddr*)&addr, sizeof(addr))) {
perror(remote_host);
return 4;
}
// setup signals
struct sigaction sa;
sa.sa_flags = 0;
sa.sa_handler = onsig;
sigemptyset(&sa.sa_mask);
sigaction(SIGINT, &sa, 0);
// configure cosmo audio
struct CosmoAudioOpenOptions cao = {0};
cao.sizeofThis = sizeof(struct CosmoAudioOpenOptions);
cao.deviceType = kCosmoAudioDeviceTypeCapture;
cao.sampleRate = SAMPLING_RATE;
cao.bufferFrames = CHUNK_FRAMES * 2;
cao.debugLog = DEBUG_LOG;
cao.channels = 1;
// connect to microphone and speaker
int status;
struct CosmoAudio* ca;
status = cosmoaudio_open(&ca, &cao);
if (status != COSMOAUDIO_SUCCESS) {
fprintf(stderr, "failed to open audio: %d\n", status);
return 5;
}
while (!g_done) {
// read from microphone
float buf32[CHUNK_FRAMES];
cosmoaudio_poll(ca, (int[]){CHUNK_FRAMES}, 0);
cosmoaudio_read(ca, buf32, CHUNK_FRAMES);
short buf16[CHUNK_FRAMES];
for (int i = 0; i < CHUNK_FRAMES; ++i)
buf16[i] = toshort(buf32[i]);
// send to server
if (write(client, buf16, CHUNK_FRAMES * sizeof(short)) == -1) {
if (errno == EINTR && g_done)
break;
perror(remote_host);
return 7;
}
// print loudness in ascii
char meter[21];
double db = rms_to_db(rms(buf32, CHUNK_FRAMES));
format_decibel_meter(meter, 20, db);
printf("\r%s| %+6.2f dB", meter, db);
fflush(stdout);
}
// clean up resources
cosmoaudio_close(ca);
close(client);
}

View file

@ -45,6 +45,12 @@ $(DSP_SCALE_A).pkg: \
$(DSP_SCALE_A_OBJS) \
$(foreach x,$(DSP_SCALE_A_DIRECTDEPS),$($(x)_A).pkg)
ifeq ($(ARCH),x86_64)
o/$(MODE)/dsp/scale/cdecimate2xuint8x8.o: private \
CFLAGS += \
-mssse3
endif
o/$(MODE)/dsp/scale/cdecimate2xuint8x8.o \
o/$(MODE)/dsp/scale/gyarados.o \
o/$(MODE)/dsp/scale/magikarp.o \

View file

@ -16,17 +16,20 @@
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE.
*/
#include "libc/assert.h"
#include "libc/intrin/packuswb.h"
#include "libc/intrin/paddw.h"
#include "libc/intrin/palignr.h"
#include "libc/intrin/pmaddubsw.h"
#include "libc/intrin/psraw.h"
#include "libc/log/check.h"
#include "libc/log/log.h"
#include "libc/macros.h"
#include "libc/nexgen32e/x86feature.h"
#include "libc/str/str.h"
#include "third_party/intel/immintrin.internal.h"
/**
* Performs 2D Motion Picture Convolution Acceleration by Leveraging SSSE3.
*
* @note H/T John Costella, Jean-Baptiste Joseph Fourier
* @note RIP Huixiang Chen
*/
void *cDecimate2xUint8x8(unsigned long n, unsigned char A[n],
const signed char K[8]) {
#ifdef __x86_64__
#define TAPS 8
#define RATIO 2
#define OFFSET 3
@ -37,62 +40,107 @@
#define LOOKAHEAD (SPREAD - LOOKBEHIND)
#define SCALE 5
#define ROUND (1 << (SCALE - 1))
/**
* Performs 2D Motion Picture Convolution Acceleration by Leveraging SSSE3.
*
* @note H/T John Costella, Jean-Baptiste Joseph Fourier
* @note RIP Huixiang Chen
*/
void *cDecimate2xUint8x8(unsigned long n, unsigned char A[n],
const signed char K[8]) {
short kRound[8] = {ROUND, ROUND, ROUND, ROUND, ROUND, ROUND, ROUND, ROUND};
signed char kMadd1[16] = {K[0], K[1], K[0], K[1], K[0], K[1], K[0], K[1],
K[0], K[1], K[0], K[1], K[0], K[1], K[0], K[1]};
signed char kMadd2[16] = {K[2], K[3], K[2], K[3], K[2], K[3], K[2], K[3],
K[2], K[3], K[2], K[3], K[2], K[3], K[2], K[3]};
signed char kMadd3[16] = {K[4], K[5], K[4], K[5], K[4], K[5], K[4], K[5],
K[4], K[5], K[4], K[5], K[4], K[5], K[4], K[5]};
signed char kMadd4[16] = {K[6], K[7], K[6], K[7], K[6], K[7], K[6], K[7],
K[6], K[7], K[6], K[7], K[6], K[7], K[6], K[7]};
unsigned char bv0[16], bv1[16], bv2[16], bv3[16];
unsigned char in1[16], in2[16], in3[16];
short wv0[8], wv1[8], wv2[8], wv3[8];
__m128i kRound = _mm_set1_epi16(ROUND);
__m128i kMadd1 = _mm_set_epi8(K[1], K[0], K[1], K[0], K[1], K[0], K[1], K[0],
K[1], K[0], K[1], K[0], K[1], K[0], K[1], K[0]);
__m128i kMadd2 = _mm_set_epi8(K[3], K[2], K[3], K[2], K[3], K[2], K[3], K[2],
K[3], K[2], K[3], K[2], K[3], K[2], K[3], K[2]);
__m128i kMadd3 = _mm_set_epi8(K[5], K[4], K[5], K[4], K[5], K[4], K[5], K[4],
K[5], K[4], K[5], K[4], K[5], K[4], K[5], K[4]);
__m128i kMadd4 = _mm_set_epi8(K[7], K[6], K[7], K[6], K[7], K[6], K[7], K[6],
K[7], K[6], K[7], K[6], K[7], K[6], K[7], K[6]);
__m128i bv0, bv1, bv2, bv3;
__m128i in1, in2, in3;
__m128i wv0, wv1, wv2, wv3;
unsigned long i, j, w;
if (n >= STRIDE) {
i = 0;
w = (n + RATIO / 2) / RATIO;
memset(in1, A[0], sizeof(in1));
memset(in2, A[n - 1], 16);
memcpy(in2, A, MIN(16, n));
in1 = _mm_set1_epi8(A[0]);
in2 = _mm_set1_epi8(A[n - 1]);
_mm_storeu_si128((__m128i *)&in2, _mm_loadu_si128((__m128i *)A));
for (; i < w; i += STRIDE) {
j = i * RATIO + 16;
if (j + 16 <= n) {
memcpy(in3, &A[j], 16);
in3 = _mm_loadu_si128((__m128i *)&A[j]);
} else {
memset(in3, A[n - 1], 16);
in3 = _mm_set1_epi8(A[n - 1]);
if (j < n) {
memcpy(in3, &A[j], n - j);
// SSSE3-compatible way to handle partial loads
__m128i mask = _mm_loadu_si128((__m128i *)&A[j]);
__m128i shuffle_mask = _mm_set_epi8(15, 14, 13, 12, 11, 10, 9, 8, 7,
6, 5, 4, 3, 2, 1, 0);
__m128i index = _mm_set1_epi8(n - j);
__m128i cmp = _mm_cmplt_epi8(shuffle_mask, index);
in3 = _mm_or_si128(_mm_and_si128(cmp, mask),
_mm_andnot_si128(cmp, in3));
}
}
palignr(bv0, in2, in1, 13);
palignr(bv1, in2, in1, 15);
palignr(bv2, in3, in2, 1);
palignr(bv3, in3, in2, 3);
pmaddubsw(wv0, bv0, kMadd1);
pmaddubsw(wv1, bv1, kMadd2);
pmaddubsw(wv2, bv2, kMadd3);
pmaddubsw(wv3, bv3, kMadd4);
paddw(wv0, wv0, kRound);
paddw(wv0, wv0, wv1);
paddw(wv0, wv0, wv2);
paddw(wv0, wv0, wv3);
psraw(wv0, wv0, SCALE);
packuswb(bv2, wv0, wv0);
memcpy(&A[i], bv2, STRIDE);
memcpy(in1, in2, 16);
memcpy(in2, in3, 16);
bv0 = _mm_alignr_epi8(in2, in1, 13);
bv1 = _mm_alignr_epi8(in2, in1, 15);
bv2 = _mm_alignr_epi8(in3, in2, 1);
bv3 = _mm_alignr_epi8(in3, in2, 3);
wv0 = _mm_maddubs_epi16(bv0, kMadd1);
wv1 = _mm_maddubs_epi16(bv1, kMadd2);
wv2 = _mm_maddubs_epi16(bv2, kMadd3);
wv3 = _mm_maddubs_epi16(bv3, kMadd4);
wv0 = _mm_add_epi16(wv0, kRound);
wv0 = _mm_add_epi16(wv0, wv1);
wv0 = _mm_add_epi16(wv0, wv2);
wv0 = _mm_add_epi16(wv0, wv3);
wv0 = _mm_srai_epi16(wv0, SCALE);
bv2 = _mm_packus_epi16(wv0, wv0);
_mm_storel_epi64((__m128i *)&A[i], bv2);
in1 = in2;
in2 = in3;
}
}
return A;
#else
long h, i;
if (n < 2)
return A;
unsigned char M[3 + n + 4];
unsigned char *q = M;
q[0] = A[0];
q[1] = A[0];
q[2] = A[0];
memcpy(q + 3, A, n);
q[3 + n + 0] = A[n - 1];
q[3 + n + 1] = A[n - 1];
q[3 + n + 2] = A[n - 1];
q[3 + n + 3] = A[n - 1];
q += 3;
h = (n + 1) >> 1;
for (i = 0; i < h; ++i) {
short x0, x1, x2, x3, x4, x5, x6, x7;
x0 = q[i * 2 - 3];
x1 = q[i * 2 - 2];
x2 = q[i * 2 - 1];
x3 = q[i * 2 + 0];
x4 = q[i * 2 + 1];
x5 = q[i * 2 + 2];
x6 = q[i * 2 + 3];
x7 = q[i * 2 + 4];
x0 *= K[0];
x1 *= K[1];
x2 *= K[2];
x3 *= K[3];
x4 *= K[4];
x5 *= K[5];
x6 *= K[6];
x7 *= K[7];
x0 += x1;
x2 += x3;
x4 += x5;
x6 += x7;
x0 += x2;
x4 += x6;
x0 += x4;
x0 += 1 << 4;
x0 >>= 5;
A[i] = MIN(255, MAX(0, x0));
}
return A;
#endif
}

View file

@ -25,7 +25,7 @@
#include "libc/limits.h"
#include "libc/log/check.h"
#include "libc/log/log.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
#include "libc/math.h"
#include "libc/mem/gc.h"
#include "libc/mem/mem.h"
@ -149,12 +149,11 @@ static int Sharpen(int ax, int bx, int cx) {
static void GyaradosImpl(long dyw, long dxw, int dst[dyw][dxw], long syw,
long sxw, const int src[syw][sxw], long dyn, long dxn,
long syn, long sxn, int tmp0[restrict dyn][sxn],
int tmp1[restrict dyn][sxn],
int tmp2[restrict dyn][dxn], long yfn, long xfn,
const short fyi[dyn][yfn], const short fyw[dyn][yfn],
const short fxi[dxn][xfn], const short fxw[dxn][xfn],
bool sharpen) {
long syn, long sxn, int tmp0[dyn][sxn],
int tmp1[dyn][sxn], int tmp2[dyn][dxn], long yfn,
long xfn, const short fyi[dyn][yfn],
const short fyw[dyn][yfn], const short fxi[dxn][xfn],
const short fxw[dxn][xfn], bool sharpen) {
long i;
int eax, dy, dx, sx;
for (sx = 0; sx < sxn; ++sx) {
@ -165,12 +164,19 @@ static void GyaradosImpl(long dyw, long dxw, int dst[dyw][dxw], long syw,
tmp0[dy][sx] = QRS(M, eax);
}
}
for (dy = 0; dy < dyn; ++dy) {
for (sx = 0; sx < sxn; ++sx) {
tmp1[dy][sx] = sharpen ? Sharpen(tmp0[MIN(dyn - 1, MAX(0, dy - 1))][sx],
tmp0[dy][sx],
tmp0[MIN(dyn - 1, MAX(0, dy + 1))][sx])
: tmp0[dy][sx];
if (sharpen) {
for (dy = 0; dy < dyn; ++dy) {
for (sx = 0; sx < sxn; ++sx) {
tmp1[dy][sx] =
Sharpen(tmp0[MIN(dyn - 1, MAX(0, dy - 1))][sx], tmp0[dy][sx],
tmp0[MIN(dyn - 1, MAX(0, dy + 1))][sx]);
}
}
} else {
for (dy = 0; dy < dyn; ++dy) {
for (sx = 0; sx < sxn; ++sx) {
tmp1[dy][sx] = tmp0[dy][sx];
}
}
}
for (dx = 0; dx < dxn; ++dx) {
@ -181,12 +187,19 @@ static void GyaradosImpl(long dyw, long dxw, int dst[dyw][dxw], long syw,
tmp2[dy][dx] = QRS(M, eax);
}
}
for (dx = 0; dx < dxn; ++dx) {
for (dy = 0; dy < dyn; ++dy) {
dst[dy][dx] = sharpen ? Sharpen(tmp2[dy][MIN(dxn - 1, MAX(0, dx - 1))],
tmp2[dy][dx],
tmp2[dy][MIN(dxn - 1, MAX(0, dx + 1))])
: tmp2[dy][dx];
if (sharpen) {
for (dx = 0; dx < dxn; ++dx) {
for (dy = 0; dy < dyn; ++dy) {
dst[dy][dx] =
Sharpen(tmp2[dy][MIN(dxn - 1, MAX(0, dx - 1))], tmp2[dy][dx],
tmp2[dy][MIN(dxn - 1, MAX(0, dx + 1))]);
}
}
} else {
for (dx = 0; dx < dxn; ++dx) {
for (dy = 0; dy < dyn; ++dy) {
dst[dy][dx] = tmp2[dy][dx];
}
}
}
}

View file

@ -20,7 +20,7 @@
#include "dsp/core/ks8.h"
#include "dsp/core/kss8.h"
#include "dsp/scale/cdecimate2xuint8x8.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
#include "libc/nexgen32e/x86feature.h"
#include "libc/str/str.h"
#include "libc/x/x.h"
@ -106,9 +106,9 @@ void *Magkern2xY(long ys, long xs, unsigned char p[ys][xs], long yn, long xn) {
return p;
}
void *MagikarpY(long dys, long dxs, unsigned char d[restrict dys][dxs],
long sys, long sxs, const unsigned char s[sys][sxs], long yn,
long xn, const signed char K[8]) {
void *MagikarpY(long dys, long dxs, unsigned char d[dys][dxs], long sys,
long sxs, const unsigned char s[sys][sxs], long yn, long xn,
const signed char K[8]) {
long y, x;
for (y = 0; y < yn; ++y) {
for (x = 0; x < xn; ++x) {

View file

@ -16,7 +16,7 @@
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE.
*/
#include "libc/macros.internal.h"
#include "libc/macros.h"
// TODO(jart): write me

View file

@ -21,7 +21,7 @@
#include "libc/assert.h"
#include "libc/limits.h"
#include "libc/log/log.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
#include "libc/math.h"
#include "libc/str/str.h"

View file

@ -18,7 +18,7 @@
*/
#include "dsp/tty/quant.h"
#include "libc/log/check.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
struct TtyRgb rgb2ttyi2f_(int r, int g, int b) {
return rgb2ttyf((ttyrgb_m128){r, g, b} / 255);

Some files were not shown because too many files have changed in this diff Show more