cosmopolitan/libc/x/x.h

135 lines
9.3 KiB
C
Raw Normal View History

2020-06-15 14:18:57 +00:00
#ifndef COSMOPOLITAN_LIBC_X_H_
#define COSMOPOLITAN_LIBC_X_H_
Improve ZIP filesystem and change its prefix The ZIP filesystem has a breaking change. You now need to use /zip/ to open() / opendir() / etc. assets within the ZIP structure of your APE binary, instead of the previous convention of using zip: or zip! URIs. This is needed because Python likes to use absolute paths, and having ZIP paths encoded like URIs simply broke too many things. Many more system calls have been updated to be able to operate on ZIP files and file descriptors. In particular fcntl() and ioctl() since Python would do things like ask if a ZIP file is a terminal and get confused when the old implementation mistakenly said yes, because the fastest way to guarantee native file descriptors is to dup(2). This change also improves the async signal safety of zipos and ensures it doesn't maintain any open file descriptors beyond that which the user has opened. This change makes a lot of progress towards adding magic numbers that are specific to platforms other than Linux. The philosophy here is that, if you use an operating system like FreeBSD, then you should be able to take advantage of FreeBSD exclusive features, even if we don't polyfill them on other platforms. For example, you can now open() a file with the O_VERIFY flag. If your program runs on other platforms, then Cosmo will automatically set O_VERIFY to zero. This lets you safely use it without the need for #ifdef or ifstatements which detract from readability. One of the blindspots of the ASAN memory hardening we use to offer Rust like assurances has always been that memory passed to the kernel via system calls (e.g. writev) can't be checked automatically since the kernel wasn't built with MODE=asan. This change makes more progress ensuring that each system call will verify the soundness of memory before it's passed to the kernel. The code for doing these checks is fast, particularly for buffers, where it can verify 64 bytes a cycle. - Correct O_LOOP definition on NT - Introduce program_executable_name - Add ASAN guards to more system calls - Improve termios compatibility with BSDs - Fix bug in Windows auxiliary value encoding - Add BSD and XNU specific errnos and open flags - Add check to ensure build doesn't talk to internet
2021-08-22 08:04:18 +00:00
#include "libc/calls/struct/rusage.h"
#include "libc/calls/struct/sigaction.h"
2020-06-15 14:18:57 +00:00
#include "libc/calls/struct/timespec.h"
#include "libc/calls/struct/timeval.h"
#include "libc/fmt/pflink.h"
#include "libc/stdio/stdio.h"
2020-06-15 14:18:57 +00:00
#if !(__ASSEMBLER__ + __LINKER__ + 0)
COSMOPOLITAN_C_START_
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis
Standard Library veneers for folks not building embedded RTOS */
#define _XPNN paramsnonnull()
#define _XRET dontthrow nocallback dontdiscard returnsnonnull
2020-06-15 14:18:57 +00:00
#define _XMAL returnspointerwithnoaliases _XRET
#define _XMALPG returnsaligned((PAGESIZE)) _XMAL
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » system calls
*/
int xsigaction(int, void *, uint64_t, uint64_t, struct sigaction *);
int xwrite(int, const void *, uint64_t);
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » memory
*/
void xdie(void) wontreturn;
char *xdtoa(double) _XMAL;
2021-02-21 19:06:18 +00:00
char *xdtoaf(float) _XMAL;
char *xdtoal(long double) _XMAL;
2020-06-15 14:18:57 +00:00
char *xasprintf(const char *, ...) printfesque(1) paramsnonnull((1)) _XMAL;
char *xvasprintf(const char *, va_list) _XPNN _XMAL;
char *xgetline(struct FILE *) _XPNN mallocesque;
void *xmalloc(size_t) attributeallocsize((1)) _XMAL;
Make major improvements to redbean and libraries The most exciting improvement is dynamic pages will soon be able to use the executable itself as an object store. it required a heroic technique for overcoming ETXTBSY restrictions which lets us open the executable in read/write mode, which means (1) wa can restore the APE header, and (2) we can potentially containerize redbean extension code so that modules you download for your redbean online will only impact your redbean. Here's a list of breaking changes to redbean: - Remove /tool/net/ prefix from magic ZIP paths - GetHeader() now returns NIL if header is absent Here's a list of fixes and enhancements to redbean: - Support 64-bit ZIP archives - Record User-Agent header in logs - Add twelve error handlers to accept() - Display octal st_mode on listing page - Show ZIP file comments on listing page - Restore APE MZ header on redbean startup - Track request count on redbean index page - Report server uptime on redbean index page - Don't bind server socket using SO_REUSEPORT - Fix #151 where Lua LoadAsset() could free twice - Report rusage accounting when workers exit w/ -vv - Use ZIP iattr field as text/plain vs. binary hint - Add ParseUrl() API for parsing things like a.href - Add ParseParams() API for parsing HTTP POST bodies - Add IsAcceptablePath() API for checking dots, etc. - Add IsValidHttpToken() API for validating sane ASCII - Add IsAcceptableHostPort() for validating HOST[:PORT] - Send 400 response to HTTP/1.1 requests without a Host - Send 403 response if ZIP or file isn't other readable - Add virtual hosting that tries prepending Host to path - Route requests based on Host in Request-URI if present - Host routing will attempt to remove or add the www. prefix - Sign-extend UNIX timestamps and don't adjust FileTime zone Here's some of the improvements made to Cosmopolitan Libc: - Fix ape.S indentation - Improve consts.sh magnums - Write pretty good URL parser - Improve rusage accounting apis - Bring mremap() closer to working - Added ZIP APIs which will change - Check for overflow in reallocarray() - Remove overly fancy linkage in strerror() - Fix GDB attach on crash w/ OpenBSD msyscall() - Make sigqueue() portable to most UNIX distros - Make integer serialization macros more elegant - Bring back 34x tprecode8to16() performance boost - Make malloc() more resilient to absurdly large sizes
2021-04-18 18:34:59 +00:00
void *xrealloc(void *, size_t)
attributeallocsize((2)) dontthrow nocallback dontdiscard;
2020-06-15 14:18:57 +00:00
void *xcalloc(size_t, size_t) attributeallocsize((1, 2)) _XMAL;
void *xvalloc(size_t) attributeallocsize((1)) _XMALPG;
void *xmemalign(size_t, size_t) attributeallocalign((1))
attributeallocsize((2)) _XMAL;
void *xmemalignzero(size_t, size_t) attributeallocalign((1))
attributeallocsize((2)) _XMAL;
2020-06-15 14:18:57 +00:00
char *xstrdup(const char *) _XPNN _XMAL;
char *xstrndup(const char *, size_t) _XPNN _XMAL;
2020-06-15 14:18:57 +00:00
char *xstrcat(const char *, ...) paramsnonnull((1)) nullterminated() _XMAL;
char *xstrmul(const char *, size_t) paramsnonnull((1)) _XMAL;
char *xinet_ntop(int, const void *) _XPNN _XMAL;
void *xunbinga(size_t, const char16_t *) attributeallocalign((1)) _XMAL _XRET;
void *xunbing(const char16_t *) _XMAL _XRET;
char16_t *utf8toutf16(const char *, size_t, size_t *) dontdiscard;
char *utf16toutf8(const char16_t *, size_t, size_t *) dontdiscard;
wchar_t *utf8toutf32(const char *, size_t, size_t *) dontdiscard;
wchar_t *utf16to32(const char16_t *, size_t, size_t *) dontdiscard;
char *xhomedir(void) dontdiscard;
char *xstripext(const char *) dontdiscard;
char *xstripexts(const char *) dontdiscard;
Productionize new APE loader and more The APE_NO_MODIFY_SELF loader payload has been moved out of the examples folder and improved so that it works on BSD systems, and permits general elf program headers. This brings its quality up enough that it should be acceptable to use by default for many programs, e.g. Python, Lua, SQLite and Python. It's the responsibility of the user to define an appropriate TMPDIR if /tmp is considered an adversarial environment. Mac OS shall be supported by APE_NO_MODIFY_SELF soon. Fixes and improvements have been made to program_executable_name as it's now the one true way to get the absolute path of the executing image. This change fixes a memory leak in linenoise history loading, introduced by performance optimizations in 51904e2687c04d7ae20410cd94c2148972d6bae6 This change fixes a longstanding regression with Mach system calls, that 23ae9dfceb6f657862f00e674a8e4dc357a9d24d back in February which impacted our sched_yield() implementation, which is why no one noticed until now. The Blinkenlights PC emulator has been improved. We now fix rendering on XNU and BSD by not making the assumption that the kernel terminal driver understands UTF8 since that seems to break its internal modeling of \r\n which is now being addressed by using \e[𝑦H instead. The paneling is now more compact in real mode so you won't need to make your font as tiny if you're only emulating an 8086 program. The CLMUL ISA is now emulated too This change also makes improvement to time. CLOCK_MONOTONIC now does the right thing on Windows NT. The nanosecond time module functions added in Python 3.7 have been backported. This change doubles the performance of Argon2 password stretching simply by not using its copy_block and xor_block helper functions, as they were trivial to inline thus resulting in us needing to iterate over each 1024 byte block four fewer times. This change makes code size improvements. _PyUnicode_ToNumeric() was 64k in size and now it's 10k. The CJK codec lookup tables now use lazy delta zigzag deflate (δzd) encoding which reduces their size from 600k to 200k plus the code bloat caused by macro abuse in _decimal.c is now addressed so our fully-loaded statically-linked hermetically-sealed Python virtual interpreter container is now 9.4 megs in the default build mode and 5.5m in MODE=tiny which leaves plenty of room for chibicc. The pydoc web server now accommodates the use case of people who work by SSH'ing into a different machine w/ python.com -m pydoc -p8080 -h0.0.0.0 Finally Python Capsulae delenda est and won't be supported in the future
2021-10-02 15:17:04 +00:00
void *xload(bool *, void **, const void *, size_t, size_t);
void *xloadzd(bool *, void **, const void *, size_t, size_t, size_t, size_t,
uint32_t);
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » files
*/
int rmrf(const char *);
2021-01-30 03:49:34 +00:00
int makedirs(const char *, unsigned);
2020-12-24 07:42:56 +00:00
char *xdirname(const char *) paramsnonnull() _XMAL;
char *xjoinpaths(const char *, const char *) paramsnonnull() _XMAL;
2021-08-26 04:35:58 +00:00
char *xreadlink(const char *) paramsnonnull() _XMAL;
char *xreadlinkat(int, const char *) paramsnonnull() _XMAL;
2020-06-15 14:18:57 +00:00
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » time
*/
char *xiso8601i(int) mallocesque;
char *xiso8601tv(struct timeval *) mallocesque;
char *xiso8601ts(struct timespec *) mallocesque;
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » input / output
*/
void *xslurp(const char *, size_t *)
Add SSL to redbean Your redbean can now interoperate with clients that require TLS crypto. This is accomplished using a protocol polyglot that lets us distinguish between HTTP and HTTPS regardless of the port number. Certificates will be generated automatically, if none are supplied by the user. Footprint increases by only a few hundred kb so redbean in MODY=tiny is now 1.0mb - Add lseek() polyfills for ZIP executable - Automatically polyfill /tmp/FOO paths on NT - Fix readdir() / ftw() / nftw() bugs on Windows - Introduce -B flag for slower SSL that's stronger - Remove mbedtls features Cosmopolitan doesn't need - Have base64 decoder support the uri-safe alternative - Remove Truncated HMAC because it's forbidden by the IETF - Add all the mbedtls test suites and make them go 3x faster - Support opendir() / readdir() / closedir() on ZIP executable - Use Everest for ECDHE-ECDSA because it's so good it's so good - Add tinier implementation of sha1 since it's not worth the rom - Add chi-square monte-carlo mean correlation tests for getrandom() - Source entropy on Windows from the proper interface everyone uses We're continuing to outperform NGINX and other servers on raw message throughput. Using SSL means that instead of 1,000,000 qps you can get around 300,000 qps. However redbean isn't as fast as NGINX yet at SSL handshakes, since redbean can do 2,627 per second and NGINX does 4.3k Right now, the SSL UX story works best if you give your redbean a key signing key since that can be easily generated by openssl using a one liner then redbean will do all the things that are impossibly hard to do like signing ecdsa and rsa certificates that'll work in chrome. We should integrate the let's encrypt acme protocol in the future. Live Demo: https://redbean.justine.lol/ Root Cert: https://redbean.justine.lol/redbean1.crt
2021-06-24 19:31:26 +00:00
paramsnonnull((1)) returnspointerwithnoaliases
returnsaligned((PAGESIZE)) dontdiscard;
int xbarf(const char *, const void *, size_t);
2020-06-15 14:18:57 +00:00
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » safety
*/
#define xstrcat(...) (xstrcat)(__VA_ARGS__, NULL)
Improve ZIP filesystem and change its prefix The ZIP filesystem has a breaking change. You now need to use /zip/ to open() / opendir() / etc. assets within the ZIP structure of your APE binary, instead of the previous convention of using zip: or zip! URIs. This is needed because Python likes to use absolute paths, and having ZIP paths encoded like URIs simply broke too many things. Many more system calls have been updated to be able to operate on ZIP files and file descriptors. In particular fcntl() and ioctl() since Python would do things like ask if a ZIP file is a terminal and get confused when the old implementation mistakenly said yes, because the fastest way to guarantee native file descriptors is to dup(2). This change also improves the async signal safety of zipos and ensures it doesn't maintain any open file descriptors beyond that which the user has opened. This change makes a lot of progress towards adding magic numbers that are specific to platforms other than Linux. The philosophy here is that, if you use an operating system like FreeBSD, then you should be able to take advantage of FreeBSD exclusive features, even if we don't polyfill them on other platforms. For example, you can now open() a file with the O_VERIFY flag. If your program runs on other platforms, then Cosmo will automatically set O_VERIFY to zero. This lets you safely use it without the need for #ifdef or ifstatements which detract from readability. One of the blindspots of the ASAN memory hardening we use to offer Rust like assurances has always been that memory passed to the kernel via system calls (e.g. writev) can't be checked automatically since the kernel wasn't built with MODE=asan. This change makes more progress ensuring that each system call will verify the soundness of memory before it's passed to the kernel. The code for doing these checks is fast, particularly for buffers, where it can verify 64 bytes a cycle. - Correct O_LOOP definition on NT - Introduce program_executable_name - Add ASAN guards to more system calls - Improve termios compatibility with BSDs - Fix bug in Windows auxiliary value encoding - Add BSD and XNU specific errnos and open flags - Add check to ensure build doesn't talk to internet
2021-08-22 08:04:18 +00:00
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » processes
*/
int xspawn(struct rusage *);
int xvspawn(void (*)(void *), void *, struct rusage *);
2020-06-15 14:18:57 +00:00
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » generic typing
*/
#if __STDC_VERSION__ + 0 >= 201112
#define xiso8601(TS) \
_Generic(*(TS), struct timeval : xiso8601tv, default : xiso8601ts)(TS)
#endif /* C11 */
/*───────────────────────────────────────────────────────────────────────────│─╗
cosmopolitan § eXtended apis » link-time optimizations
*/
#if defined(__GNUC__) && !defined(__STRICT_ANSI__)
#define xasprintf(FMT, ...) (xasprintf)(PFLINK(FMT), ##__VA_ARGS__)
#define xvasprintf(FMT, VA) (xvasprintf)(PFLINK(FMT), VA)
#define xsigaction(SIG, HANDLER, FLAGS, MASK, OLD) \
({ \
__SIGACTION_YOINK(SIG); \
xsigaction(SIG, HANDLER, FLAGS, MASK, OLD); \
})
2020-06-15 14:18:57 +00:00
#endif
COSMOPOLITAN_C_END_
#endif /* !(__ASSEMBLER__ + __LINKER__ + 0) */
#endif /* COSMOPOLITAN_LIBC_X_H_ */