ec480f5aa0
- Every unit test now passes on Apple Silicon. The final piece of this puzzle was porting our POSIX threads cancelation support, since that works differently on ARM64 XNU vs. AMD64. Our semaphore support on Apple Silicon is also superior now compared to AMD64, thanks to the grand central dispatch library which lets *NSYNC locks go faster. - The Cosmopolitan runtime is now more stable, particularly on Windows. To do this, thread local storage is mandatory at all runtime levels, and the innermost packages of the C library is no longer being built using ASAN. TLS is being bootstrapped with a 128-byte TIB during the process startup phase, and then later on the runtime re-allocates it either statically or dynamically to support code using _Thread_local. fork() and execve() now do a better job cooperating with threads. We can now check how much stack memory is left in the process or thread when functions like kprintf() / execve() etc. call alloca(), so that ENOMEM can be raised, reduce a buffer size, or just print a warning. - POSIX signal emulation is now implemented the same way kernels do it with pthread_kill() and raise(). Any thread can interrupt any other thread, regardless of what it's doing. If it's blocked on read/write then the killer thread will cancel its i/o operation so that EINTR can be returned in the mark thread immediately. If it's doing a tight CPU bound operation, then that's also interrupted by the signal delivery. Signal delivery works now by suspending a thread and pushing context data structures onto its stack, and redirecting its execution to a trampoline function, which calls SetThreadContext(GetCurrentThread()) when it's done. - We're now doing a better job managing locks and handles. On NetBSD we now close semaphore file descriptors in forked children. Semaphores on Windows can now be canceled immediately, which means mutexes/condition variables will now go faster. Apple Silicon semaphores can be canceled too. We're now using Apple's pthread_yield() funciton. Apple _nocancel syscalls are now used on XNU when appropriate to ensure pthread_cancel requests aren't lost. The MbedTLS library has been updated to support POSIX thread cancelations. See tool/build/runitd.c for an example of how it can be used for production multi-threaded tls servers. Handles on Windows now leak less often across processes. All i/o operations on Windows are now overlapped, which means file pointers can no longer be inherited across dup() and fork() for the time being. - We now spawn a thread on Windows to deliver SIGCHLD and wakeup wait4() which means, for example, that posix_spawn() now goes 3x faster. POSIX spawn is also now more correct. Like Musl, it's now able to report the failure code of execve() via a pipe although our approach favors using shared memory to do that on systems that have a true vfork() function. - We now spawn a thread to deliver SIGALRM to threads when setitimer() is used. This enables the most precise wakeups the OS makes possible. - The Cosmopolitan runtime now uses less memory. On NetBSD for example, it turned out the kernel would actually commit the PT_GNU_STACK size which caused RSS to be 6mb for every process. Now it's down to ~4kb. On Apple Silicon, we reduce the mandatory upstream thread size to the smallest possible size to reduce the memory overhead of Cosmo threads. The examples directory has a program called greenbean which can spawn a web server on Linux with 10,000 worker threads and have the memory usage of the process be ~77mb. The 1024 byte overhead of POSIX-style thread-local storage is now optional; it won't be allocated until the pthread_setspecific/getspecific functions are called. On Windows, the threads that get spawned which are internal to the libc implementation use reserve rather than commit memory, which shaves a few hundred kb. - sigaltstack() is now supported on Windows, however it's currently not able to be used to handle stack overflows, since crash signals are still generated by WIN32. However the crash handler will still switch to the alt stack, which is helpful in environments with tiny threads. - Test binaries are now smaller. Many of the mandatory dependencies of the test runner have been removed. This ensures many programs can do a better job only linking the the thing they're testing. This caused the test binaries for LIBC_FMT for example, to decrease from 200kb to 50kb - long double is no longer used in the implementation details of libc, except in the APIs that define it. The old code that used long double for time (instead of struct timespec) has now been thoroughly removed. - ShowCrashReports() is now much tinier in MODE=tiny. Instead of doing backtraces itself, it'll just print a command you can run on the shell using our new `cosmoaddr2line` program to view the backtrace. - Crash report signal handling now works in a much better way. Instead of terminating the process, it now relies on SA_RESETHAND so that the default SIG_IGN behavior can terminate the process if necessary. - Our pledge() functionality has now been fully ported to AARCH64 Linux. |
||
|---|---|---|
| .. | ||
| test | ||
| alloc.c | ||
| as.c | ||
| as.main.c | ||
| asm.c | ||
| cast.c | ||
| chibicc.c | ||
| chibicc.h | ||
| chibicc.main.c | ||
| chibicc.mk | ||
| codegen.c | ||
| dox1.c | ||
| dox2.c | ||
| file.c | ||
| file.h | ||
| fpclassify.c | ||
| hashmap.c | ||
| help.txt | ||
| kw.c | ||
| kw.gperf | ||
| kw.h | ||
| kw.inc | ||
| NOTICE | ||
| parse.c | ||
| preprocess.c | ||
| printast.c | ||
| pybind.c | ||
| README.cosmo | ||
| README.md | ||
| strarray.c | ||
| tokenize.c | ||
| type.c | ||
| unicode.c | ||
chibicc: A Small C Compiler
(The old master has moved to historical/old branch. This is a new one uploaded in September 2020.)
chibicc is yet another small C compiler that implements most C11 features. Even though it still probably falls into the "toy compilers" category just like other small compilers do, chibicc can compile several real-world programs, including Git, SQLite and libpng, without making modifications to the compiled programs. Generated executables of these programs pass their corresponding test suites. So, chibicc actually supports a wide variety of C11 features and is able to compile hundreds of thousands of lines of real-world C code correctly.
chibicc is developed as the reference implementation for a book I'm currently writing about the C compiler and the low-level programming. The book covers the vast topic with an incremental approach; in the first chapter, readers will implement a "compiler" that accepts just a single number as a "language", which will then gain one feature at a time in each section of the book until the language that the compiler accepts matches what the C11 spec specifies. I took this incremental approach from the paper by Abdulaziz Ghuloum.
Each commit of this project corresponds to a section of the book. For this purpose, not only the final state of the project but each commit was carefully written with readability in mind. Readers should be able to learn how a C language feature can be implemented just by reading one or a few commits of this project. For example, this is how while, [], ?:, and thread-local variable are implemented. If you have plenty of spare time, it might be fun to read it from the first commit.
If you like this project, please consider purchasing a copy of the book when it becomes available! 😀 I publish the source code here to give people early access to it, because I was planing to do that anyway with a permissive open-source license after publishing the book. If I don't charge for the source code, it doesn't make much sense to me to keep it private. I hope to publish the book in 2021.
I pronounce chibicc as chee bee cee cee. "chibi" means "mini" or "small" in Japanese. "cc" stands for C compiler.
Status
Features that are often missing in a small compiler but supported by chibicc include (but not limited to):
- Preprocessor
- long double (x87 80-bit floting point numbers)
- Bit-field
- alloca()
- Variable-length array
- Thread-local variable
- Atomic variable
- Common symbol
- Designated initializer
- L, u, U and u8 string literals
chibicc does not support digraphs, trigraphs, complex numbers, K&R-style function prototype, and inline assembly.
chibicc outputs a simple but nice error message when it finds an error in source code.
There's no optimization pass. chibicc emits terrible code which is probably twice or more slower than GCC's output. I have a plan to add an optimization pass once the frontend is done.
Internals
chibicc consists of the following stages:
-
Tokenize: A tokenizer takes a string as an input, breaks it into a list of tokens and returns them.
-
Preprocess: A preprocessor takes as an input a list of tokens and output a new list of macro-expanded tokens. It interprets preprocessor directives while expanding macros.
-
Parse: A recursive descendent parser constructs abstract syntax trees from the output of the preprocessor. It also adds a type to each AST node.
-
Codegen: A code generator emits an assembly text for given AST nodes.
Contributing
When I find a bug in this compiler, I go back to the original commit that introduced the bug and rewrite the commit history as if there were no such bug from the beginning. This is an unusual way of fixing bugs, but as a a part of a book, it is important to keep every commit bug-free.
Thus, I do not take pull requests in this repo. You can send me a pull request if you find a bug, but it is very likely that I will read your patch and then apply that to my previous commits by rewriting history. I'll credit your name somewhere, but your changes will be rewritten by me before submitted to this repository.
Also, please assume that I will occasionally force-push my local repository to this public one to rewrite history. If you clone this project and make local commits on top of it, your changes will have to be rebased by hand when I force-push new commits.
About the Author
I'm Rui Ueyama. I'm the creator of 8cc, which is a hobby C compiler, and also the original creator of the current version of LLVM lld linker, which is a production-quality linker used by various operating systems and large-scale build systems.
References
-
tcc: A small C compiler written by Fabrice Bellard. I learned a lot from this compiler, but the design of tcc and chibicc are different. In particular, tcc is a one-pass compiler, while chibicc is a multi-pass one.
-
lcc: Another small C compiler. The creators wrote a book about the internals of lcc, which I found a good resource to see how a compiler is implemented.