mirrors/cosmopolitan
1
0
Fork 0
mirror of https://github.com/jart/cosmopolitan.git synced 2025-01-31 03:27:39 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
cosmopolitan/ape/ape.mk

125 lines
3.4 KiB
Makefile
Raw Normal View History

Initial import
2020-06-15 14:18:57 +00:00
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#
# OVERVIEW
#
# αcτµαlly pδrταblε εxεcµταblε
#
# DESCRIPTION
#
# This file defines the libraries, runtimes, and build rules needed to
# create executables from your Linux workstation that'll run anywhere.
# Loading this package will make certain systemic modifications to the
# build like turning off the System V "Red Zone" optimization, because
# αcτµαlly pδrταblε εxεcµταblεs need to be able to run in kernelspace.
PKGS += APE
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
APE = o/$(MODE)/ape/ape.o \
Initial import
2020-06-15 14:18:57 +00:00
o/$(MODE)/ape/ape.lds
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
APE_NO_MODIFY_SELF = \
o/$(MODE)/ape/ape.lds \
Productionize new APE loader and more The APE_NO_MODIFY_SELF loader payload has been moved out of the examples folder and improved so that it works on BSD systems, and permits general elf program headers. This brings its quality up enough that it should be acceptable to use by default for many programs, e.g. Python, Lua, SQLite and Python. It's the responsibility of the user to define an appropriate TMPDIR if /tmp is considered an adversarial environment. Mac OS shall be supported by APE_NO_MODIFY_SELF soon. Fixes and improvements have been made to program_executable_name as it's now the one true way to get the absolute path of the executing image. This change fixes a memory leak in linenoise history loading, introduced by performance optimizations in 51904e2687c04d7ae20410cd94c2148972d6bae6 This change fixes a longstanding regression with Mach system calls, that 23ae9dfceb6f657862f00e674a8e4dc357a9d24d back in February which impacted our sched_yield() implementation, which is why no one noticed until now. The Blinkenlights PC emulator has been improved. We now fix rendering on XNU and BSD by not making the assumption that the kernel terminal driver understands UTF8 since that seems to break its internal modeling of \r\n which is now being addressed by using \e[𝑦H instead. The paneling is now more compact in real mode so you won't need to make your font as tiny if you're only emulating an 8086 program. The CLMUL ISA is now emulated too This change also makes improvement to time. CLOCK_MONOTONIC now does the right thing on Windows NT. The nanosecond time module functions added in Python 3.7 have been backported. This change doubles the performance of Argon2 password stretching simply by not using its copy_block and xor_block helper functions, as they were trivial to inline thus resulting in us needing to iterate over each 1024 byte block four fewer times. This change makes code size improvements. _PyUnicode_ToNumeric() was 64k in size and now it's 10k. The CJK codec lookup tables now use lazy delta zigzag deflate (δzd) encoding which reduces their size from 600k to 200k plus the code bloat caused by macro abuse in _decimal.c is now addressed so our fully-loaded statically-linked hermetically-sealed Python virtual interpreter container is now 9.4 megs in the default build mode and 5.5m in MODE=tiny which leaves plenty of room for chibicc. The pydoc web server now accommodates the use case of people who work by SSH'ing into a different machine w/ python.com -m pydoc -p8080 -h0.0.0.0 Finally Python Capsulae delenda est and won't be supported in the future
2021-10-02 15:17:04 +00:00
o/$(MODE)/ape/ape-no-modify-self.o
Undiamond Python headers This change gets the Python codebase into a state where it conforms to the conventions of this codebase. It's now possible to include headers from Python, without worrying about ordering. Python has traditionally solved that problem by "diamonding" everything in Python.h, but that's problematic since it means any change to any Python header invalidates all the build artifacts. Lastly it makes tooling not work. Since it is hard to explain to Emacs when I press C-c C-h to add an import line it shouldn't add the header that actually defines the symbol, and instead do follow the nonstandard Python convention. Progress has been made on letting Python load source code from the zip executable structure via the standard C library APIs. System calss now recognizes zip!FILENAME alternative URIs as equivalent to zip:FILENAME since Python uses colon as its delimiter. Some progress has been made on embedding the notice license terms into the Python object code. This is easier said than done since Python has an extremely complicated ownership story. - Some termios APIs have been added - Implement rewinddir() dirstream API - GetCpuCount() API added to Cosmopolitan Libc - More bugs in Cosmopolitan Libc have been fixed - zipobj.com now has flags for mangling the path - Fixed bug a priori with sendfile() on certain BSDs - Polyfill F_DUPFD and F_DUPFD_CLOEXEC across platforms - FIOCLEX / FIONCLEX now polyfilled for fast O_CLOEXEC changes - APE now supports a hybrid solution to no-self-modify for builds - Many BSD-only magnums added, e.g. O_SEARCH, O_SHLOCK, SF_NODISKIO
2021-08-12 07:42:14 +00:00
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
APE_COPY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-copy-self.o
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
Get codebase completely working with LLVM You can now build Cosmopolitan with Clang: make -j8 MODE=llvm o/llvm/examples/hello.com The assembler and linker code is now friendly to LLVM too. So it's not needed to configure Clang to use binutils under the hood. If you love LLVM then you can now use pure LLVM.
2021-02-08 17:19:00 +00:00
$(OUTPUT_OPTION)
Initial import
2020-06-15 14:18:57 +00:00
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-mno-red-zone \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
Add scouts honor escape hatch for source embedding
2020-06-16 02:01:28 +00:00
APE_FILES := $(wildcard ape/*.*)
APE_HDRS = $(filter %.h,$(APE_FILES))
Make more functions friendly to Address Sanitizer
2021-02-02 11:45:31 +00:00
APE_INCS = $(filter %.inc,$(APE_FILES))
Productionize new APE loader and more The APE_NO_MODIFY_SELF loader payload has been moved out of the examples folder and improved so that it works on BSD systems, and permits general elf program headers. This brings its quality up enough that it should be acceptable to use by default for many programs, e.g. Python, Lua, SQLite and Python. It's the responsibility of the user to define an appropriate TMPDIR if /tmp is considered an adversarial environment. Mac OS shall be supported by APE_NO_MODIFY_SELF soon. Fixes and improvements have been made to program_executable_name as it's now the one true way to get the absolute path of the executing image. This change fixes a memory leak in linenoise history loading, introduced by performance optimizations in 51904e2687c04d7ae20410cd94c2148972d6bae6 This change fixes a longstanding regression with Mach system calls, that 23ae9dfceb6f657862f00e674a8e4dc357a9d24d back in February which impacted our sched_yield() implementation, which is why no one noticed until now. The Blinkenlights PC emulator has been improved. We now fix rendering on XNU and BSD by not making the assumption that the kernel terminal driver understands UTF8 since that seems to break its internal modeling of \r\n which is now being addressed by using \e[𝑦H instead. The paneling is now more compact in real mode so you won't need to make your font as tiny if you're only emulating an 8086 program. The CLMUL ISA is now emulated too This change also makes improvement to time. CLOCK_MONOTONIC now does the right thing on Windows NT. The nanosecond time module functions added in Python 3.7 have been backported. This change doubles the performance of Argon2 password stretching simply by not using its copy_block and xor_block helper functions, as they were trivial to inline thus resulting in us needing to iterate over each 1024 byte block four fewer times. This change makes code size improvements. _PyUnicode_ToNumeric() was 64k in size and now it's 10k. The CJK codec lookup tables now use lazy delta zigzag deflate (δzd) encoding which reduces their size from 600k to 200k plus the code bloat caused by macro abuse in _decimal.c is now addressed so our fully-loaded statically-linked hermetically-sealed Python virtual interpreter container is now 9.4 megs in the default build mode and 5.5m in MODE=tiny which leaves plenty of room for chibicc. The pydoc web server now accommodates the use case of people who work by SSH'ing into a different machine w/ python.com -m pydoc -p8080 -h0.0.0.0 Finally Python Capsulae delenda est and won't be supported in the future
2021-10-02 15:17:04 +00:00
APE_SRCS_C = ape/loader.c
APE_SRCS_S = $(filter %.S,$(APE_FILES))
APE_SRCS = $(APE_SRCS_C) $(APE_SRCS_S)
APE_OBJS = $(APE_SRCS_S:%.S=o/$(MODE)/%.o)
Add scouts honor escape hatch for source embedding
2020-06-16 02:01:28 +00:00
APE_CHECKS = $(APE_HDRS:%=o/%.ok)
Initial import
2020-06-15 14:18:57 +00:00
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
o/$(MODE)/ape/ape.lds: \
ape/ape.lds \
ape/macros.internal.h \
libc/dce.h \
Improve dead code elimination
2021-02-08 12:04:42 +00:00
libc/zip.h
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
o/ape/idata.inc: \
ape/idata.internal.h \
Add scouts honor escape hatch for source embedding
2020-06-16 02:01:28 +00:00
ape/relocations.h
Initial import
2020-06-15 14:18:57 +00:00
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
o/$(MODE)/ape/ape-no-modify-self.o: \
ape/ape.S \
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
o/$(MODE)/ape/ape.elf
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF \
-DAPE_LOADER="\"o/$(MODE)/ape/ape.elf\"" $<
Productionize new APE loader and more The APE_NO_MODIFY_SELF loader payload has been moved out of the examples folder and improved so that it works on BSD systems, and permits general elf program headers. This brings its quality up enough that it should be acceptable to use by default for many programs, e.g. Python, Lua, SQLite and Python. It's the responsibility of the user to define an appropriate TMPDIR if /tmp is considered an adversarial environment. Mac OS shall be supported by APE_NO_MODIFY_SELF soon. Fixes and improvements have been made to program_executable_name as it's now the one true way to get the absolute path of the executing image. This change fixes a memory leak in linenoise history loading, introduced by performance optimizations in 51904e2687c04d7ae20410cd94c2148972d6bae6 This change fixes a longstanding regression with Mach system calls, that 23ae9dfceb6f657862f00e674a8e4dc357a9d24d back in February which impacted our sched_yield() implementation, which is why no one noticed until now. The Blinkenlights PC emulator has been improved. We now fix rendering on XNU and BSD by not making the assumption that the kernel terminal driver understands UTF8 since that seems to break its internal modeling of \r\n which is now being addressed by using \e[𝑦H instead. The paneling is now more compact in real mode so you won't need to make your font as tiny if you're only emulating an 8086 program. The CLMUL ISA is now emulated too This change also makes improvement to time. CLOCK_MONOTONIC now does the right thing on Windows NT. The nanosecond time module functions added in Python 3.7 have been backported. This change doubles the performance of Argon2 password stretching simply by not using its copy_block and xor_block helper functions, as they were trivial to inline thus resulting in us needing to iterate over each 1024 byte block four fewer times. This change makes code size improvements. _PyUnicode_ToNumeric() was 64k in size and now it's 10k. The CJK codec lookup tables now use lazy delta zigzag deflate (δzd) encoding which reduces their size from 600k to 200k plus the code bloat caused by macro abuse in _decimal.c is now addressed so our fully-loaded statically-linked hermetically-sealed Python virtual interpreter container is now 9.4 megs in the default build mode and 5.5m in MODE=tiny which leaves plenty of room for chibicc. The pydoc web server now accommodates the use case of people who work by SSH'ing into a different machine w/ python.com -m pydoc -p8080 -h0.0.0.0 Finally Python Capsulae delenda est and won't be supported in the future
2021-10-02 15:17:04 +00:00
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
o/$(MODE)/ape/ape-copy-self.o: \
ape/ape.S
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF $<
Initial import
2020-06-15 14:18:57 +00:00
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
o/$(MODE)/ape/loader.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b01111001 -g $(APE_LOADER_FLAGS)
Productionize new APE loader and more The APE_NO_MODIFY_SELF loader payload has been moved out of the examples folder and improved so that it works on BSD systems, and permits general elf program headers. This brings its quality up enough that it should be acceptable to use by default for many programs, e.g. Python, Lua, SQLite and Python. It's the responsibility of the user to define an appropriate TMPDIR if /tmp is considered an adversarial environment. Mac OS shall be supported by APE_NO_MODIFY_SELF soon. Fixes and improvements have been made to program_executable_name as it's now the one true way to get the absolute path of the executing image. This change fixes a memory leak in linenoise history loading, introduced by performance optimizations in 51904e2687c04d7ae20410cd94c2148972d6bae6 This change fixes a longstanding regression with Mach system calls, that 23ae9dfceb6f657862f00e674a8e4dc357a9d24d back in February which impacted our sched_yield() implementation, which is why no one noticed until now. The Blinkenlights PC emulator has been improved. We now fix rendering on XNU and BSD by not making the assumption that the kernel terminal driver understands UTF8 since that seems to break its internal modeling of \r\n which is now being addressed by using \e[𝑦H instead. The paneling is now more compact in real mode so you won't need to make your font as tiny if you're only emulating an 8086 program. The CLMUL ISA is now emulated too This change also makes improvement to time. CLOCK_MONOTONIC now does the right thing on Windows NT. The nanosecond time module functions added in Python 3.7 have been backported. This change doubles the performance of Argon2 password stretching simply by not using its copy_block and xor_block helper functions, as they were trivial to inline thus resulting in us needing to iterate over each 1024 byte block four fewer times. This change makes code size improvements. _PyUnicode_ToNumeric() was 64k in size and now it's 10k. The CJK codec lookup tables now use lazy delta zigzag deflate (δzd) encoding which reduces their size from 600k to 200k plus the code bloat caused by macro abuse in _decimal.c is now addressed so our fully-loaded statically-linked hermetically-sealed Python virtual interpreter container is now 9.4 megs in the default build mode and 5.5m in MODE=tiny which leaves plenty of room for chibicc. The pydoc web server now accommodates the use case of people who work by SSH'ing into a different machine w/ python.com -m pydoc -p8080 -h0.0.0.0 Finally Python Capsulae delenda est and won't be supported in the future
2021-10-02 15:17:04 +00:00
o/$(MODE)/ape/loader-gcc.asm: ape/loader.c
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b01111001 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=0b01111001 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b00001000 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b00001000 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=0b00001000 -S -g0 $(APE_LOADER_FLAGS)
Provide option to have APE not modify itself This change introduces ape-no-modify-self.o to the amalgamated release binaries, which may be used as an alternative to ape.o to make it easier to use APE in cases where the self-modifying behavior isn't acceptable. Please note that this alternative copying behavior isn't necessarily better. It introduces a whole bunch of questions of its own, which are documented in the ape.S source comment and should be considered by both the program author as well as the end-user of programs linked this way. For example, build environments that use read-only file systems and would prefer to not have a launcher wrapper (like we use in our build) can use ape-no-modify-self.o instead of ape.o and then set the $TMPDIR environment variable to point to a sane read-write-exec location. Fixes #146 See #82
2021-04-08 04:01:57 +00:00
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
o/$(MODE)/ape/ape.macho: o/$(MODE)/ape/ape.macho.dbg
o/$(MODE)/ape/ape.elf.dbg: \
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/loader-elf.o \
Productionize new APE loader and more The APE_NO_MODIFY_SELF loader payload has been moved out of the examples folder and improved so that it works on BSD systems, and permits general elf program headers. This brings its quality up enough that it should be acceptable to use by default for many programs, e.g. Python, Lua, SQLite and Python. It's the responsibility of the user to define an appropriate TMPDIR if /tmp is considered an adversarial environment. Mac OS shall be supported by APE_NO_MODIFY_SELF soon. Fixes and improvements have been made to program_executable_name as it's now the one true way to get the absolute path of the executing image. This change fixes a memory leak in linenoise history loading, introduced by performance optimizations in 51904e2687c04d7ae20410cd94c2148972d6bae6 This change fixes a longstanding regression with Mach system calls, that 23ae9dfceb6f657862f00e674a8e4dc357a9d24d back in February which impacted our sched_yield() implementation, which is why no one noticed until now. The Blinkenlights PC emulator has been improved. We now fix rendering on XNU and BSD by not making the assumption that the kernel terminal driver understands UTF8 since that seems to break its internal modeling of \r\n which is now being addressed by using \e[𝑦H instead. The paneling is now more compact in real mode so you won't need to make your font as tiny if you're only emulating an 8086 program. The CLMUL ISA is now emulated too This change also makes improvement to time. CLOCK_MONOTONIC now does the right thing on Windows NT. The nanosecond time module functions added in Python 3.7 have been backported. This change doubles the performance of Argon2 password stretching simply by not using its copy_block and xor_block helper functions, as they were trivial to inline thus resulting in us needing to iterate over each 1024 byte block four fewer times. This change makes code size improvements. _PyUnicode_ToNumeric() was 64k in size and now it's 10k. The CJK codec lookup tables now use lazy delta zigzag deflate (δzd) encoding which reduces their size from 600k to 200k plus the code bloat caused by macro abuse in _decimal.c is now addressed so our fully-loaded statically-linked hermetically-sealed Python virtual interpreter container is now 9.4 megs in the default build mode and 5.5m in MODE=tiny which leaves plenty of room for chibicc. The pydoc web server now accommodates the use case of people who work by SSH'ing into a different machine w/ python.com -m pydoc -p8080 -h0.0.0.0 Finally Python Capsulae delenda est and won't be supported in the future
2021-10-02 15:17:04 +00:00
ape/loader.lds
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
@$(ELFLINK) -z max-page-size=0x10
Undiamond Python headers This change gets the Python codebase into a state where it conforms to the conventions of this codebase. It's now possible to include headers from Python, without worrying about ordering. Python has traditionally solved that problem by "diamonding" everything in Python.h, but that's problematic since it means any change to any Python header invalidates all the build artifacts. Lastly it makes tooling not work. Since it is hard to explain to Emacs when I press C-c C-h to add an import line it shouldn't add the header that actually defines the symbol, and instead do follow the nonstandard Python convention. Progress has been made on letting Python load source code from the zip executable structure via the standard C library APIs. System calss now recognizes zip!FILENAME alternative URIs as equivalent to zip:FILENAME since Python uses colon as its delimiter. Some progress has been made on embedding the notice license terms into the Python object code. This is easier said than done since Python has an extremely complicated ownership story. - Some termios APIs have been added - Implement rewinddir() dirstream API - GetCpuCount() API added to Cosmopolitan Libc - More bugs in Cosmopolitan Libc have been fixed - zipobj.com now has flags for mangling the path - Fixed bug a priori with sendfile() on certain BSDs - Polyfill F_DUPFD and F_DUPFD_CLOEXEC across platforms - FIOCLEX / FIONCLEX now polyfilled for fast O_CLOEXEC changes - APE now supports a hybrid solution to no-self-modify for builds - Many BSD-only magnums added, e.g. O_SEARCH, O_SHLOCK, SF_NODISKIO
2021-08-12 07:42:14 +00:00
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
o/$(MODE)/ape/ape.macho.dbg: \
o/$(MODE)/ape/loader-xnu.o \
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
o/$(MODE)/ape/loader-macho.o \
ape/loader-macho.lds
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
@$(ELFLINK) -z max-page-size=0x10
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
Initial import
2020-06-15 14:18:57 +00:00
.PHONY: o/$(MODE)/ape
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
o/$(MODE)/ape: $(APE) \
$(APE_CHECKS) \
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
o/$(MODE)/ape/ape.elf \
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
o/$(MODE)/ape/ape.macho \
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
o/$(MODE)/ape/ape-copy-self.o \
Provide option to have APE not modify itself This change introduces ape-no-modify-self.o to the amalgamated release binaries, which may be used as an alternative to ape.o to make it easier to use APE in cases where the self-modifying behavior isn't acceptable. Please note that this alternative copying behavior isn't necessarily better. It introduces a whole bunch of questions of its own, which are documented in the ape.S source comment and should be considered by both the program author as well as the end-user of programs linked this way. For example, build environments that use read-only file systems and would prefer to not have a launcher wrapper (like we use in our build) can use ape-no-modify-self.o instead of ape.o and then set the $TMPDIR environment variable to point to a sane read-write-exec location. Fixes #146 See #82
2021-04-08 04:01:57 +00:00
o/$(MODE)/ape/ape-no-modify-self.o
Reference in a new issue Copy permalink