cosmopolitan/libc/zipos/zipos.internal.h

63 lines
2.1 KiB
C
Raw Normal View History

2020-06-15 14:18:57 +00:00
#ifndef COSMOPOLITAN_LIBC_ZIPOS_ZIPOS_H_
#define COSMOPOLITAN_LIBC_ZIPOS_ZIPOS_H_
Undiamond Python headers This change gets the Python codebase into a state where it conforms to the conventions of this codebase. It's now possible to include headers from Python, without worrying about ordering. Python has traditionally solved that problem by "diamonding" everything in Python.h, but that's problematic since it means any change to any Python header invalidates all the build artifacts. Lastly it makes tooling not work. Since it is hard to explain to Emacs when I press C-c C-h to add an import line it shouldn't add the header that actually defines the symbol, and instead do follow the nonstandard Python convention. Progress has been made on letting Python load source code from the zip executable structure via the standard C library APIs. System calss now recognizes zip!FILENAME alternative URIs as equivalent to zip:FILENAME since Python uses colon as its delimiter. Some progress has been made on embedding the notice license terms into the Python object code. This is easier said than done since Python has an extremely complicated ownership story. - Some termios APIs have been added - Implement rewinddir() dirstream API - GetCpuCount() API added to Cosmopolitan Libc - More bugs in Cosmopolitan Libc have been fixed - zipobj.com now has flags for mangling the path - Fixed bug a priori with sendfile() on certain BSDs - Polyfill F_DUPFD and F_DUPFD_CLOEXEC across platforms - FIOCLEX / FIONCLEX now polyfilled for fast O_CLOEXEC changes - APE now supports a hybrid solution to no-self-modify for builds - Many BSD-only magnums added, e.g. O_SEARCH, O_SHLOCK, SF_NODISKIO
2021-08-12 07:42:14 +00:00
#include "libc/calls/calls.h"
#include "libc/intrin/nopl.h"
#include "libc/nexgen32e/threaded.h"
2020-06-15 14:18:57 +00:00
#if !(__ASSEMBLER__ + __LINKER__ + 0)
COSMOPOLITAN_C_START_
struct stat;
struct iovec;
struct ZiposUri {
const char *path;
size_t len;
};
struct ZiposHandle {
struct ZiposHandle *next;
size_t size; /* byte length of `mem` */
size_t mapsize; /* total size of this struct */
2020-06-15 14:18:57 +00:00
size_t pos; /* read/write byte offset state */
uint32_t cfile; /* central directory entry rva */
uint8_t *mem; /* points to inflated data or uncompressed image */
uint8_t data[]; /* uncompressed file memory */
2020-06-15 14:18:57 +00:00
};
struct Zipos {
uint8_t *map;
uint8_t *cdir;
struct ZiposHandle *freelist;
};
void __zipos_lock(void) hidden;
void __zipos_unlock(void) hidden;
int __zipos_close(int) hidden;
Undiamond Python headers This change gets the Python codebase into a state where it conforms to the conventions of this codebase. It's now possible to include headers from Python, without worrying about ordering. Python has traditionally solved that problem by "diamonding" everything in Python.h, but that's problematic since it means any change to any Python header invalidates all the build artifacts. Lastly it makes tooling not work. Since it is hard to explain to Emacs when I press C-c C-h to add an import line it shouldn't add the header that actually defines the symbol, and instead do follow the nonstandard Python convention. Progress has been made on letting Python load source code from the zip executable structure via the standard C library APIs. System calss now recognizes zip!FILENAME alternative URIs as equivalent to zip:FILENAME since Python uses colon as its delimiter. Some progress has been made on embedding the notice license terms into the Python object code. This is easier said than done since Python has an extremely complicated ownership story. - Some termios APIs have been added - Implement rewinddir() dirstream API - GetCpuCount() API added to Cosmopolitan Libc - More bugs in Cosmopolitan Libc have been fixed - zipobj.com now has flags for mangling the path - Fixed bug a priori with sendfile() on certain BSDs - Polyfill F_DUPFD and F_DUPFD_CLOEXEC across platforms - FIOCLEX / FIONCLEX now polyfilled for fast O_CLOEXEC changes - APE now supports a hybrid solution to no-self-modify for builds - Many BSD-only magnums added, e.g. O_SEARCH, O_SHLOCK, SF_NODISKIO
2021-08-12 07:42:14 +00:00
struct Zipos *__zipos_get(void) pureconst hidden;
void __zipos_free(struct Zipos *, struct ZiposHandle *) hidden;
ssize_t __zipos_parseuri(const char *, struct ZiposUri *) hidden;
ssize_t __zipos_find(struct Zipos *, const struct ZiposUri *);
int __zipos_open(const struct ZiposUri *, unsigned, int) hidden;
int __zipos_stat(const struct ZiposUri *, struct stat *) hidden;
int __zipos_fstat(const struct ZiposHandle *, struct stat *) hidden;
int __zipos_stat_impl(struct Zipos *, size_t, struct stat *) hidden;
2020-06-15 14:18:57 +00:00
ssize_t __zipos_read(struct ZiposHandle *, const struct iovec *, size_t,
ssize_t) hidden;
2020-06-15 14:18:57 +00:00
ssize_t __zipos_write(struct ZiposHandle *, const struct iovec *, size_t,
ssize_t) hidden;
Add SSL to redbean Your redbean can now interoperate with clients that require TLS crypto. This is accomplished using a protocol polyglot that lets us distinguish between HTTP and HTTPS regardless of the port number. Certificates will be generated automatically, if none are supplied by the user. Footprint increases by only a few hundred kb so redbean in MODY=tiny is now 1.0mb - Add lseek() polyfills for ZIP executable - Automatically polyfill /tmp/FOO paths on NT - Fix readdir() / ftw() / nftw() bugs on Windows - Introduce -B flag for slower SSL that's stronger - Remove mbedtls features Cosmopolitan doesn't need - Have base64 decoder support the uri-safe alternative - Remove Truncated HMAC because it's forbidden by the IETF - Add all the mbedtls test suites and make them go 3x faster - Support opendir() / readdir() / closedir() on ZIP executable - Use Everest for ECDHE-ECDSA because it's so good it's so good - Add tinier implementation of sha1 since it's not worth the rom - Add chi-square monte-carlo mean correlation tests for getrandom() - Source entropy on Windows from the proper interface everyone uses We're continuing to outperform NGINX and other servers on raw message throughput. Using SSL means that instead of 1,000,000 qps you can get around 300,000 qps. However redbean isn't as fast as NGINX yet at SSL handshakes, since redbean can do 2,627 per second and NGINX does 4.3k Right now, the SSL UX story works best if you give your redbean a key signing key since that can be easily generated by openssl using a one liner then redbean will do all the things that are impossibly hard to do like signing ecdsa and rsa certificates that'll work in chrome. We should integrate the let's encrypt acme protocol in the future. Live Demo: https://redbean.justine.lol/ Root Cert: https://redbean.justine.lol/redbean1.crt
2021-06-24 19:31:26 +00:00
int64_t __zipos_lseek(struct ZiposHandle *, int64_t, unsigned) hidden;
Improve ZIP filesystem and change its prefix The ZIP filesystem has a breaking change. You now need to use /zip/ to open() / opendir() / etc. assets within the ZIP structure of your APE binary, instead of the previous convention of using zip: or zip! URIs. This is needed because Python likes to use absolute paths, and having ZIP paths encoded like URIs simply broke too many things. Many more system calls have been updated to be able to operate on ZIP files and file descriptors. In particular fcntl() and ioctl() since Python would do things like ask if a ZIP file is a terminal and get confused when the old implementation mistakenly said yes, because the fastest way to guarantee native file descriptors is to dup(2). This change also improves the async signal safety of zipos and ensures it doesn't maintain any open file descriptors beyond that which the user has opened. This change makes a lot of progress towards adding magic numbers that are specific to platforms other than Linux. The philosophy here is that, if you use an operating system like FreeBSD, then you should be able to take advantage of FreeBSD exclusive features, even if we don't polyfill them on other platforms. For example, you can now open() a file with the O_VERIFY flag. If your program runs on other platforms, then Cosmo will automatically set O_VERIFY to zero. This lets you safely use it without the need for #ifdef or ifstatements which detract from readability. One of the blindspots of the ASAN memory hardening we use to offer Rust like assurances has always been that memory passed to the kernel via system calls (e.g. writev) can't be checked automatically since the kernel wasn't built with MODE=asan. This change makes more progress ensuring that each system call will verify the soundness of memory before it's passed to the kernel. The code for doing these checks is fast, particularly for buffers, where it can verify 64 bytes a cycle. - Correct O_LOOP definition on NT - Introduce program_executable_name - Add ASAN guards to more system calls - Improve termios compatibility with BSDs - Fix bug in Windows auxiliary value encoding - Add BSD and XNU specific errnos and open flags - Add check to ensure build doesn't talk to internet
2021-08-22 08:04:18 +00:00
int __zipos_fcntl(int, int, uintptr_t) hidden;
int __zipos_notat(int, const char *) hidden;
2020-06-15 14:18:57 +00:00
2022-06-26 09:58:36 +00:00
#ifdef _NOPL0
#define __zipos_lock() _NOPL0("__threadcalls", __zipos_lock)
#define __zipos_unlock() _NOPL0("__threadcalls", __zipos_unlock)
#else
#define __zipos_lock() (__threaded ? __zipos_lock() : 0)
#define __zipos_unlock() (__threaded ? __zipos_unlock() : 0)
#endif
2020-06-15 14:18:57 +00:00
COSMOPOLITAN_C_END_
#endif /* !(__ASSEMBLER__ + __LINKER__ + 0) */
#endif /* COSMOPOLITAN_LIBC_ZIPOS_ZIPOS_H_ */