cosmopolitan/libc/x/x.h

127 lines
8.9 KiB
C
Raw Normal View History

2020-06-15 14:18:57 +00:00
#ifndef COSMOPOLITAN_LIBC_X_H_
#define COSMOPOLITAN_LIBC_X_H_
Improve ZIP filesystem and change its prefix The ZIP filesystem has a breaking change. You now need to use /zip/ to open() / opendir() / etc. assets within the ZIP structure of your APE binary, instead of the previous convention of using zip: or zip! URIs. This is needed because Python likes to use absolute paths, and having ZIP paths encoded like URIs simply broke too many things. Many more system calls have been updated to be able to operate on ZIP files and file descriptors. In particular fcntl() and ioctl() since Python would do things like ask if a ZIP file is a terminal and get confused when the old implementation mistakenly said yes, because the fastest way to guarantee native file descriptors is to dup(2). This change also improves the async signal safety of zipos and ensures it doesn't maintain any open file descriptors beyond that which the user has opened. This change makes a lot of progress towards adding magic numbers that are specific to platforms other than Linux. The philosophy here is that, if you use an operating system like FreeBSD, then you should be able to take advantage of FreeBSD exclusive features, even if we don't polyfill them on other platforms. For example, you can now open() a file with the O_VERIFY flag. If your program runs on other platforms, then Cosmo will automatically set O_VERIFY to zero. This lets you safely use it without the need for #ifdef or ifstatements which detract from readability. One of the blindspots of the ASAN memory hardening we use to offer Rust like assurances has always been that memory passed to the kernel via system calls (e.g. writev) can't be checked automatically since the kernel wasn't built with MODE=asan. This change makes more progress ensuring that each system call will verify the soundness of memory before it's passed to the kernel. The code for doing these checks is fast, particularly for buffers, where it can verify 64 bytes a cycle. - Correct O_LOOP definition on NT - Introduce program_executable_name - Add ASAN guards to more system calls - Improve termios compatibility with BSDs - Fix bug in Windows auxiliary value encoding - Add BSD and XNU specific errnos and open flags - Add check to ensure build doesn't talk to internet
2021-08-22 08:04:18 +00:00
#include "libc/calls/struct/rusage.h"
#include "libc/calls/struct/sigaction.h"
2020-06-15 14:18:57 +00:00
#include "libc/calls/struct/timespec.h"
#include "libc/calls/struct/timeval.h"
#include "libc/fmt/pflink.h"
#include "libc/stdio/stdio.h"
2020-06-15 14:18:57 +00:00
#if !(__ASSEMBLER__ + __LINKER__ + 0)
COSMOPOLITAN_C_START_
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis
Standard Library veneers for folks not building embedded RTOS */
#define _XPNN paramsnonnull()
#define _XRET nothrow nocallback nodiscard returnsnonnull
#define _XMAL returnspointerwithnoaliases _XRET
#define _XMALPG returnsaligned((PAGESIZE)) _XMAL
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » system calls
*/
int xsigaction(int, void *, uint64_t, uint64_t, struct sigaction *);
int xwrite(int, const void *, uint64_t);
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » memory
*/
void xdie(void) wontreturn;
char *xdtoa(double) _XMAL;
2021-02-21 19:06:18 +00:00
char *xdtoaf(float) _XMAL;
char *xdtoal(long double) _XMAL;
2020-06-15 14:18:57 +00:00
char *xasprintf(const char *, ...) printfesque(1) paramsnonnull((1)) _XMAL;
char *xvasprintf(const char *, va_list) _XPNN _XMAL;
char *xgetline(struct FILE *) _XPNN mallocesque;
void *xmalloc(size_t) attributeallocsize((1)) _XMAL;
Make major improvements to redbean and libraries The most exciting improvement is dynamic pages will soon be able to use the executable itself as an object store. it required a heroic technique for overcoming ETXTBSY restrictions which lets us open the executable in read/write mode, which means (1) wa can restore the APE header, and (2) we can potentially containerize redbean extension code so that modules you download for your redbean online will only impact your redbean. Here's a list of breaking changes to redbean: - Remove /tool/net/ prefix from magic ZIP paths - GetHeader() now returns NIL if header is absent Here's a list of fixes and enhancements to redbean: - Support 64-bit ZIP archives - Record User-Agent header in logs - Add twelve error handlers to accept() - Display octal st_mode on listing page - Show ZIP file comments on listing page - Restore APE MZ header on redbean startup - Track request count on redbean index page - Report server uptime on redbean index page - Don't bind server socket using SO_REUSEPORT - Fix #151 where Lua LoadAsset() could free twice - Report rusage accounting when workers exit w/ -vv - Use ZIP iattr field as text/plain vs. binary hint - Add ParseUrl() API for parsing things like a.href - Add ParseParams() API for parsing HTTP POST bodies - Add IsAcceptablePath() API for checking dots, etc. - Add IsValidHttpToken() API for validating sane ASCII - Add IsAcceptableHostPort() for validating HOST[:PORT] - Send 400 response to HTTP/1.1 requests without a Host - Send 403 response if ZIP or file isn't other readable - Add virtual hosting that tries prepending Host to path - Route requests based on Host in Request-URI if present - Host routing will attempt to remove or add the www. prefix - Sign-extend UNIX timestamps and don't adjust FileTime zone Here's some of the improvements made to Cosmopolitan Libc: - Fix ape.S indentation - Improve consts.sh magnums - Write pretty good URL parser - Improve rusage accounting apis - Bring mremap() closer to working - Added ZIP APIs which will change - Check for overflow in reallocarray() - Remove overly fancy linkage in strerror() - Fix GDB attach on crash w/ OpenBSD msyscall() - Make sigqueue() portable to most UNIX distros - Make integer serialization macros more elegant - Bring back 34x tprecode8to16() performance boost - Make malloc() more resilient to absurdly large sizes
2021-04-18 18:34:59 +00:00
void *xrealloc(void *, size_t)
attributeallocsize((2)) nothrow nocallback nodiscard;
2020-06-15 14:18:57 +00:00
void *xcalloc(size_t, size_t) attributeallocsize((1, 2)) _XMAL;
void *xvalloc(size_t) attributeallocsize((1)) _XMALPG;
void *xmemalign(size_t, size_t) attributeallocalign((1))
attributeallocsize((2)) _XMAL;
void *xmemalignzero(size_t, size_t) attributeallocalign((1))
attributeallocsize((2)) _XMAL;
2020-06-15 14:18:57 +00:00
char *xstrdup(const char *) _XPNN _XMAL;
char *xstrndup(const char *, size_t) _XPNN _XMAL;
2020-06-15 14:18:57 +00:00
char *xstrcat(const char *, ...) paramsnonnull((1)) nullterminated() _XMAL;
char *xstrmul(const char *, size_t) paramsnonnull((1)) _XMAL;
char *xinet_ntop(int, const void *) _XPNN _XMAL;
void *xunbinga(size_t, const char16_t *) attributeallocalign((1)) _XMAL _XRET;
void *xunbing(const char16_t *) _XMAL _XRET;
char16_t *utf8toutf16(const char *, size_t, size_t *) nodiscard;
char *utf16toutf8(const char16_t *, size_t, size_t *) nodiscard;
wchar_t *utf8toutf32(const char *, size_t, size_t *) nodiscard;
Make numerous improvements - Python static hello world now 1.8mb - Python static fully loaded now 10mb - Python HTTPS client now uses MbedTLS - Python REPL now completes import stmts - Increase stack size for Python for now - Begin synthesizing posixpath and ntpath - Restore Python \N{UNICODE NAME} support - Restore Python NFKD symbol normalization - Add optimized code path for Intel SHA-NI - Get more Python unit tests passing faster - Get Python help() pagination working on NT - Python hashlib now supports MbedTLS PBKDF2 - Make memcpy/memmove/memcmp/bcmp/etc. faster - Add Mersenne Twister and Vigna to LIBC_RAND - Provide privileged __printf() for error code - Fix zipos opendir() so that it reports ENOTDIR - Add basic chmod() implementation for Windows NT - Add Cosmo's best functions to Python cosmo module - Pin function trace indent depth to that of caller - Show memory diagram on invalid access in MODE=dbg - Differentiate stack overflow on crash in MODE=dbg - Add stb_truetype and tools for analyzing font files - Upgrade to UNICODE 13 and reduce its binary footprint - COMPILE.COM now logs resource usage of build commands - Start implementing basic poll() support on bare metal - Set getauxval(AT_EXECFN) to GetModuleFileName() on NT - Add descriptions to strerror() in non-TINY build modes - Add COUNTBRANCH() macro to help with micro-optimizations - Make error / backtrace / asan / memory code more unbreakable - Add fast perfect C implementation of μ-Law and a-Law audio codecs - Make strtol() functions consistent with other libc implementations - Improve Linenoise implementation (see also github.com/jart/bestline) - COMPILE.COM now suppresses stdout/stderr of successful build commands
2021-09-28 05:58:51 +00:00
wchar_t *utf16to32(const char16_t *, size_t, size_t *) nodiscard;
char *xhomedir(void) nodiscard;
Make numerous improvements - Python static hello world now 1.8mb - Python static fully loaded now 10mb - Python HTTPS client now uses MbedTLS - Python REPL now completes import stmts - Increase stack size for Python for now - Begin synthesizing posixpath and ntpath - Restore Python \N{UNICODE NAME} support - Restore Python NFKD symbol normalization - Add optimized code path for Intel SHA-NI - Get more Python unit tests passing faster - Get Python help() pagination working on NT - Python hashlib now supports MbedTLS PBKDF2 - Make memcpy/memmove/memcmp/bcmp/etc. faster - Add Mersenne Twister and Vigna to LIBC_RAND - Provide privileged __printf() for error code - Fix zipos opendir() so that it reports ENOTDIR - Add basic chmod() implementation for Windows NT - Add Cosmo's best functions to Python cosmo module - Pin function trace indent depth to that of caller - Show memory diagram on invalid access in MODE=dbg - Differentiate stack overflow on crash in MODE=dbg - Add stb_truetype and tools for analyzing font files - Upgrade to UNICODE 13 and reduce its binary footprint - COMPILE.COM now logs resource usage of build commands - Start implementing basic poll() support on bare metal - Set getauxval(AT_EXECFN) to GetModuleFileName() on NT - Add descriptions to strerror() in non-TINY build modes - Add COUNTBRANCH() macro to help with micro-optimizations - Make error / backtrace / asan / memory code more unbreakable - Add fast perfect C implementation of μ-Law and a-Law audio codecs - Make strtol() functions consistent with other libc implementations - Improve Linenoise implementation (see also github.com/jart/bestline) - COMPILE.COM now suppresses stdout/stderr of successful build commands
2021-09-28 05:58:51 +00:00
char *xstripext(const char *) nodiscard;
char *xstripexts(const char *) nodiscard;
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » files
*/
int rmrf(const char *);
2021-01-30 03:49:34 +00:00
int makedirs(const char *, unsigned);
2020-12-24 07:42:56 +00:00
char *xdirname(const char *) paramsnonnull() _XMAL;
char *xjoinpaths(const char *, const char *) paramsnonnull() _XMAL;
2021-08-26 04:35:58 +00:00
char *xreadlink(const char *) paramsnonnull() _XMAL;
char *xreadlinkat(int, const char *) paramsnonnull() _XMAL;
2020-06-15 14:18:57 +00:00
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » time
*/
char *xiso8601i(int) mallocesque;
char *xiso8601tv(struct timeval *) mallocesque;
char *xiso8601ts(struct timespec *) mallocesque;
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » input / output
*/
void *xslurp(const char *, size_t *)
Add SSL to redbean Your redbean can now interoperate with clients that require TLS crypto. This is accomplished using a protocol polyglot that lets us distinguish between HTTP and HTTPS regardless of the port number. Certificates will be generated automatically, if none are supplied by the user. Footprint increases by only a few hundred kb so redbean in MODY=tiny is now 1.0mb - Add lseek() polyfills for ZIP executable - Automatically polyfill /tmp/FOO paths on NT - Fix readdir() / ftw() / nftw() bugs on Windows - Introduce -B flag for slower SSL that's stronger - Remove mbedtls features Cosmopolitan doesn't need - Have base64 decoder support the uri-safe alternative - Remove Truncated HMAC because it's forbidden by the IETF - Add all the mbedtls test suites and make them go 3x faster - Support opendir() / readdir() / closedir() on ZIP executable - Use Everest for ECDHE-ECDSA because it's so good it's so good - Add tinier implementation of sha1 since it's not worth the rom - Add chi-square monte-carlo mean correlation tests for getrandom() - Source entropy on Windows from the proper interface everyone uses We're continuing to outperform NGINX and other servers on raw message throughput. Using SSL means that instead of 1,000,000 qps you can get around 300,000 qps. However redbean isn't as fast as NGINX yet at SSL handshakes, since redbean can do 2,627 per second and NGINX does 4.3k Right now, the SSL UX story works best if you give your redbean a key signing key since that can be easily generated by openssl using a one liner then redbean will do all the things that are impossibly hard to do like signing ecdsa and rsa certificates that'll work in chrome. We should integrate the let's encrypt acme protocol in the future. Live Demo: https://redbean.justine.lol/ Root Cert: https://redbean.justine.lol/redbean1.crt
2021-06-24 19:31:26 +00:00
paramsnonnull((1)) returnspointerwithnoaliases
returnsaligned((PAGESIZE)) nodiscard;
int xbarf(const char *, const void *, size_t);
2020-06-15 14:18:57 +00:00
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » safety
*/
#define xstrcat(...) (xstrcat)(__VA_ARGS__, NULL)
Improve ZIP filesystem and change its prefix The ZIP filesystem has a breaking change. You now need to use /zip/ to open() / opendir() / etc. assets within the ZIP structure of your APE binary, instead of the previous convention of using zip: or zip! URIs. This is needed because Python likes to use absolute paths, and having ZIP paths encoded like URIs simply broke too many things. Many more system calls have been updated to be able to operate on ZIP files and file descriptors. In particular fcntl() and ioctl() since Python would do things like ask if a ZIP file is a terminal and get confused when the old implementation mistakenly said yes, because the fastest way to guarantee native file descriptors is to dup(2). This change also improves the async signal safety of zipos and ensures it doesn't maintain any open file descriptors beyond that which the user has opened. This change makes a lot of progress towards adding magic numbers that are specific to platforms other than Linux. The philosophy here is that, if you use an operating system like FreeBSD, then you should be able to take advantage of FreeBSD exclusive features, even if we don't polyfill them on other platforms. For example, you can now open() a file with the O_VERIFY flag. If your program runs on other platforms, then Cosmo will automatically set O_VERIFY to zero. This lets you safely use it without the need for #ifdef or ifstatements which detract from readability. One of the blindspots of the ASAN memory hardening we use to offer Rust like assurances has always been that memory passed to the kernel via system calls (e.g. writev) can't be checked automatically since the kernel wasn't built with MODE=asan. This change makes more progress ensuring that each system call will verify the soundness of memory before it's passed to the kernel. The code for doing these checks is fast, particularly for buffers, where it can verify 64 bytes a cycle. - Correct O_LOOP definition on NT - Introduce program_executable_name - Add ASAN guards to more system calls - Improve termios compatibility with BSDs - Fix bug in Windows auxiliary value encoding - Add BSD and XNU specific errnos and open flags - Add check to ensure build doesn't talk to internet
2021-08-22 08:04:18 +00:00
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » processes
*/
int xspawn(struct rusage *);
int xvspawn(void (*)(void *), void *, struct rusage *);
2020-06-15 14:18:57 +00:00
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » generic typing
*/
#if __STDC_VERSION__ + 0 >= 201112
#define xiso8601(TS) \
_Generic(*(TS), struct timeval : xiso8601tv, default : xiso8601ts)(TS)
#endif /* C11 */
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » link-time optimizations
*/
#if defined(__GNUC__) && !defined(__STRICT_ANSI__)
#define xasprintf(FMT, ...) (xasprintf)(PFLINK(FMT), ##__VA_ARGS__)
#define xvasprintf(FMT, VA) (xvasprintf)(PFLINK(FMT), VA)
#endif
COSMOPOLITAN_C_END_
#endif /* !(__ASSEMBLER__ + __LINKER__ + 0) */
#endif /* COSMOPOLITAN_LIBC_X_H_ */