Find a file
Jōshin 8b3e368e9a
ctl::string small-string optimization ()
A small-string optimization is a way of reusing inline storage space for
sufficiently small strings, rather than allocating them on the heap. The
current approach takes after an old Facebook string class: it reuses the
highest-order byte for flags and small-string size, in such a way that a
maximally-sized small string will have its last byte zeroed, making it a
null terminator for the C string.

The only flag we have is in the highest-order bit, that says whether the
string is big (set) or small (cleared.) Most of the logic switches based
on the value of this bit; e.g. data() returns big()->p if it's set, else
small()->buf if it's cleared. For a small string, the capacity is always
fixed at sizeof(string) - 1 bytes; we store the length in the last byte,
but we store it as the number of remaining bytes of capacity, so that at
max size, the last byte will read zero and serve as our null terminator.

Morally speaking, our class's storage is a union over two POD C structs.
For now I gravitated towards a slightly more obtuse approach: the string
class itself contains a blob of the right size, and we alias that blob's
pointer for the two structs, taking some care not to run afoul of object
lifetime rules in C++. If anyone wants to improve on this, contributions
are welcome.

This commit also introduces the `ctl::__` namespace. It can't be legally
spelled by library users, and serves as our version of boost's "detail".

We introduced a string::swap function, and we now use that in operator=.
operator= now takes its argument by value, so we never need to check for
the case where the pointers are equal and can just swap the entire store
of the argument with our own, leaving the C++ destructor to free our old
storage afterwards.

There are probably still a few places where our capacity is slightly off
and we grow too fast, although there don't appear to be any where we are
too slow. I will leave these to be fixed in future changes.
2024-06-06 20:50:51 -04:00
.github github: add labeler action () 2024-06-06 05:51:36 -07:00
ape Run clang-format () 2024-06-01 16:30:43 -04:00
build Improve backtraces 2024-05-30 15:23:11 -07:00
ctl ctl::string small-string optimization () 2024-06-06 20:50:51 -04:00
dsp Run clang-format () 2024-06-01 16:30:43 -04:00
examples Make malloc() go 200x faster 2024-06-05 02:02:14 -07:00
libc Fix some nits 2024-06-05 04:05:49 -07:00
net Make malloc() go 200x faster 2024-06-05 02:02:14 -07:00
test ctl::string small-string optimization () 2024-06-06 20:50:51 -04:00
third_party Fix some nits 2024-06-05 04:05:49 -07:00
tool Release Cosmopolitan v3.4.0 2024-06-05 03:07:03 -07:00
usr/share Implement proper time zone support 2024-05-04 23:06:37 -07:00
.clang-format Update .clang-format 2024-04-19 17:11:08 -07:00
.git-blame-ignore-revs add formatting commit 2024-06-05 16:36:34 -07:00
.gitattributes Make improvements 2023-10-15 16:45:00 -07:00
.gitignore Release Cosmopolitan v3.2.4 2024-01-08 19:37:59 -08:00
CONTRIBUTING.md Fix broken link 2024-02-23 07:11:44 -08:00
LICENSE Add title to the LICENSE file 2021-02-27 13:25:59 -08:00
Makefile Introduce Cosmopolitan Templates Library (CTL) 2024-06-03 09:21:59 -07:00
README.md Remove .com from README 2024-03-22 19:38:38 -07:00

Cosmopolitan Honeybadger

build

Cosmopolitan

Cosmopolitan Libc makes C a build-once run-anywhere language, like Java, except it doesn't need an interpreter or virtual machine. Instead, it reconfigures stock GCC and Clang to output a POSIX-approved polyglot format that runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS with the best possible performance and the tiniest footprint imaginable.

Background

For an introduction to this project, please read the actually portable executable blog post and cosmopolitan libc website. We also have API documentation.

Getting Started

You can start by obtaining a release of our cosmocc compiler from https://cosmo.zip/pub/cosmocc/.

mkdir -p cosmocc
cd cosmocc
wget https://cosmo.zip/pub/cosmocc/cosmocc.zip
unzip cosmocc.zip

Here's an example program we can write:

// hello.c
#include <stdio.h>

int main() {
  printf("hello world\n");
}

It can be compiled as follows:

cosmocc -o hello hello.c
./hello

The Cosmopolitan Libc runtime links some heavyweight troubleshooting features by default, which are very useful for developers and admins. Here's how you can log system calls:

./hello --strace

Here's how you can get a much more verbose log of function calls:

./hello --ftrace

You can use the Cosmopolitan's toolchain to build conventional open source projects which use autotools. This strategy normally works:

export CC=x86_64-unknown-cosmo-cc
export CXX=x86_64-unknown-cosmo-c++
./configure --prefix=/opt/cosmos/x86_64
make -j
make install

Cosmopolitan Source Builds

Cosmopolitan can be compiled from source on any of our supported platforms. The Makefile will download cosmocc automatically.

It's recommended that you install a systemwide APE Loader. This command requires sudo access to copy the ape command to a system folder and register with binfmt_misc on Linux, for even more performance.

ape/apeinstall.sh

You can now build the mono repo with any modern version of GNU Make. To make life easier, we've included one in the cosmocc toolchain, which is guaranteed to be compatible and furthermore includes our extensions for doing build system sandboxing.

build/bootstrap/make -j8
o//examples/hello

Since the Cosmopolitan repository is very large, you might only want to build one particular thing. Here's an example of a target that can be compiled relatively quickly, which is a simple POSIX test that only depends on core LIBC packages.

rm -rf o//libc o//test
build/bootstrap/make o//test/posix/signal_test
o//test/posix/signal_test

Sometimes it's desirable to build a subset of targets, without having to list out each individual one. For example if you wanted to build and run all the unit tests in the TEST_POSIX package, you could say:

build/bootstrap/make o//test/posix

Cosmopolitan provides a variety of build modes. For example, if you want really tiny binaries (as small as 12kb in size) then you'd say:

build/bootstrap/make m=tiny

You can furthermore cut out the bloat of other operating systems, and have Cosmopolitan become much more similar to Musl Libc.

build/bootstrap/make m=tinylinux

For further details, see //build/config.mk.

Debugging

To print a log of system calls to stderr:

cosmocc -o hello hello.c
./hello --strace

To print a log of function calls to stderr:

cosmocc -o hello hello.c
./hello --ftrace

Both strace and ftrace use the unbreakable kprintf() facility, which is able to be sent to a file by setting an environment variable.

export KPRINTF_LOG=log
./hello --strace

GDB

Here's the recommended ~/.gdbinit config:

set host-charset UTF-8
set target-charset UTF-8
set target-wide-charset UTF-8
set osabi none
set complaints 0
set confirm off
set history save on
set history filename ~/.gdb_history
define asm
  layout asm
  layout reg
end
define src
  layout src
  layout reg
end
src

You normally run the .dbg file under gdb. If you need to debug the `` file itself, then you can load the debug symbols independently as

gdb foo -ex 'add-symbol-file foo.dbg 0x401000'

Platform Notes

Shells

If you use zsh and have trouble running APE programs try sh -c ./prog or simply upgrade to zsh 5.9+ (since we patched it two years ago). The same is the case for Python subprocess, old versions of fish, etc.

Linux

Some Linux systems are configured to launch MZ executables under WINE. Other distros configure their stock installs so that APE programs will print "run-detectors: unable to find an interpreter". For example:

jart@ubuntu:~$ wget https://cosmo.zip/pub/cosmos/bin/dash
jart@ubuntu:~$ chmod +x dash
jart@ubuntu:~$ ./dash
run-detectors: unable to find an interpreter for ./dash

You can fix that by registering APE with binfmt_misc:

sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf
sudo chmod +x /usr/bin/ape
sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
sudo sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"

You should be good now. APE will not only work, it'll launch executables 400µs faster now too. However if things still didn't work out, it's also possible to disable binfmt_misc as follows:

sudo sh -c 'echo -1 > /proc/sys/fs/binfmt_misc/cli'     # remove Ubuntu's MZ interpreter
sudo sh -c 'echo -1 > /proc/sys/fs/binfmt_misc/status'  # remove ALL binfmt_misc entries

WSL

It's normally unsafe to use APE in a WSL environment, because it tries to run MZ executables as WIN32 binaries within the WSL environment. In order to make it safe to use Cosmopolitan software on WSL, run this:

sudo sh -c "echo -1 > /proc/sys/fs/binfmt_misc/WSLInterop"

Discord Chatroom

The Cosmopolitan development team collaborates on the Redbean Discord server. You're welcome to join us! https://discord.gg/FwAVVu7eJ4

Support Vector

Platform Min Version Circa
AMD K8 Venus 2005
Intel Core 2006
Linux 2.6.18 2007
Windows 8 [1] 2012
Darwin (macOS) 23.1.0+ 2023
OpenBSD 7 2021
FreeBSD 13 2020
NetBSD 9.2 2021

[1] See our vista branch for a community supported version of Cosmopolitan that works on Windows Vista and Windows 7.

Special Thanks

Funding for this project is crowdsourced using GitHub Sponsors and Patreon. Your support is what makes this project possible. Thank you! We'd also like to give special thanks to the following groups and individuals:

For publicly sponsoring our work at the highest tier.