mirrors/cosmopolitan
1
0
Fork 0
mirror of https://github.com/jart/cosmopolitan.git synced 2025-01-31 03:27:39 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
cosmopolitan/ape/ape.mk

238 lines
6.1 KiB
Makefile
Raw Normal View History

Initial import
2020-06-15 14:18:57 +00:00
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#
# OVERVIEW
#
# αcτµαlly pδrταblε εxεcµταblε
#
# DESCRIPTION
#
# This file defines the libraries, runtimes, and build rules needed to
# create executables from your Linux workstation that'll run anywhere.
# Loading this package will make certain systemic modifications to the
# build like turning off the System V "Red Zone" optimization, because
# αcτµαlly pδrταblε εxεcµταblεs need to be able to run in kernelspace.
PKGS += APE
Make more code aarch64 friendly
2023-05-02 20:38:16 +00:00
APE_FILES := $(wildcard ape/*.*)
APE_HDRS = $(filter %.h,$(APE_FILES))
APE_INCS = $(filter %.inc,$(APE_FILES))
Introduce native support for MacOS ARM64 There's a new program named ape/ape-m1.c which will be used to build an embeddable binary that can load ape and elf executables. The support is mostly working so far, but still chasing down ABI issues.
2023-05-19 02:05:08 +00:00
ifeq ($(ARCH), aarch64)
APE = o/$(MODE)/ape/aarch64.lds
Initial import
2020-06-15 14:18:57 +00:00
Get aarch64 hello world working $ m=aarch64-tiny $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64 $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com hello world $ ls -hal o/$m/tool/hello/hello.com -rwxr-xr-x 1 jart jart 4.0K May 9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-09 09:35:05 +00:00
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION)
Get LIBC_RUNTIME and LIBC_CALLS building on aarch64
2023-05-09 08:56:56 +00:00
APE_SRCS = ape/ape.S
APE_OBJS = o/$(MODE)/ape/ape.o
APE_NO_MODIFY_SELF = $(APE)
APE_COPY_SELF = $(APE)
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE)
Introduce native support for MacOS ARM64 There's a new program named ape/ape-m1.c which will be used to build an embeddable binary that can load ape and elf executables. The support is mostly working so far, but still chasing down ABI issues.
2023-05-19 02:05:08 +00:00
o/$(MODE)/ape/aarch64.lds: \
ape/aarch64.lds \
libc/zip.h \
libc/intrin/bits.h \
Make improvements This change progresses our AARCH64 support: - The AARCH64 build and tests are now passing - Add 128-bit floating-point support to printf() - Fix clone() so it initializes cosmo's x28 TLS register - Fix TLS memory layout issue with aarch64 _Alignas vars - Revamp microbenchmarking tools so they work on aarch64 - Make some subtle improvements to aarch64 crash reporting - Make kisdangerous() memory checks more accurate on aarch64 - Remove sys_open() since it's not available on Linux AARCH64 This change makes general improvements to Cosmo and Redbean: - Introduce GetHostIsa() function in Redbean - You can now feature check using pledge(0, 0) - You can now feature check using unveil("",0) - Refactor some more x86-specific asm comments - Refactor and write docs for some libm functions - Make the mmap() API behave more similar to Linux - Fix WIFSIGNALED() which wrongly returned true for zero - Rename some obscure cosmo keywords from noFOO to dontFOO
2023-06-03 15:12:13 +00:00
libc/thread/tls.h \
Introduce native support for MacOS ARM64 There's a new program named ape/ape-m1.c which will be used to build an embeddable binary that can load ape and elf executables. The support is mostly working so far, but still chasing down ABI issues.
2023-05-19 02:05:08 +00:00
libc/calls/struct/timespec.h \
libc/macros.internal.h \
libc/str/str.h
Get LIBC_RUNTIME and LIBC_CALLS building on aarch64
2023-05-09 08:56:56 +00:00
else
Introduce native support for MacOS ARM64 There's a new program named ape/ape-m1.c which will be used to build an embeddable binary that can load ape and elf executables. The support is mostly working so far, but still chasing down ABI issues.
2023-05-19 02:05:08 +00:00
APE = o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION)
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
APE_NO_MODIFY_SELF = \
o/$(MODE)/ape/ape.lds \
Productionize new APE loader and more The APE_NO_MODIFY_SELF loader payload has been moved out of the examples folder and improved so that it works on BSD systems, and permits general elf program headers. This brings its quality up enough that it should be acceptable to use by default for many programs, e.g. Python, Lua, SQLite and Python. It's the responsibility of the user to define an appropriate TMPDIR if /tmp is considered an adversarial environment. Mac OS shall be supported by APE_NO_MODIFY_SELF soon. Fixes and improvements have been made to program_executable_name as it's now the one true way to get the absolute path of the executing image. This change fixes a memory leak in linenoise history loading, introduced by performance optimizations in 51904e2687c04d7ae20410cd94c2148972d6bae6 This change fixes a longstanding regression with Mach system calls, that 23ae9dfceb6f657862f00e674a8e4dc357a9d24d back in February which impacted our sched_yield() implementation, which is why no one noticed until now. The Blinkenlights PC emulator has been improved. We now fix rendering on XNU and BSD by not making the assumption that the kernel terminal driver understands UTF8 since that seems to break its internal modeling of \r\n which is now being addressed by using \e[𝑦H instead. The paneling is now more compact in real mode so you won't need to make your font as tiny if you're only emulating an 8086 program. The CLMUL ISA is now emulated too This change also makes improvement to time. CLOCK_MONOTONIC now does the right thing on Windows NT. The nanosecond time module functions added in Python 3.7 have been backported. This change doubles the performance of Argon2 password stretching simply by not using its copy_block and xor_block helper functions, as they were trivial to inline thus resulting in us needing to iterate over each 1024 byte block four fewer times. This change makes code size improvements. _PyUnicode_ToNumeric() was 64k in size and now it's 10k. The CJK codec lookup tables now use lazy delta zigzag deflate (δzd) encoding which reduces their size from 600k to 200k plus the code bloat caused by macro abuse in _decimal.c is now addressed so our fully-loaded statically-linked hermetically-sealed Python virtual interpreter container is now 9.4 megs in the default build mode and 5.5m in MODE=tiny which leaves plenty of room for chibicc. The pydoc web server now accommodates the use case of people who work by SSH'ing into a different machine w/ python.com -m pydoc -p8080 -h0.0.0.0 Finally Python Capsulae delenda est and won't be supported in the future
2021-10-02 15:17:04 +00:00
o/$(MODE)/ape/ape-no-modify-self.o
Undiamond Python headers This change gets the Python codebase into a state where it conforms to the conventions of this codebase. It's now possible to include headers from Python, without worrying about ordering. Python has traditionally solved that problem by "diamonding" everything in Python.h, but that's problematic since it means any change to any Python header invalidates all the build artifacts. Lastly it makes tooling not work. Since it is hard to explain to Emacs when I press C-c C-h to add an import line it shouldn't add the header that actually defines the symbol, and instead do follow the nonstandard Python convention. Progress has been made on letting Python load source code from the zip executable structure via the standard C library APIs. System calss now recognizes zip!FILENAME alternative URIs as equivalent to zip:FILENAME since Python uses colon as its delimiter. Some progress has been made on embedding the notice license terms into the Python object code. This is easier said than done since Python has an extremely complicated ownership story. - Some termios APIs have been added - Implement rewinddir() dirstream API - GetCpuCount() API added to Cosmopolitan Libc - More bugs in Cosmopolitan Libc have been fixed - zipobj.com now has flags for mangling the path - Fixed bug a priori with sendfile() on certain BSDs - Polyfill F_DUPFD and F_DUPFD_CLOEXEC across platforms - FIOCLEX / FIONCLEX now polyfilled for fast O_CLOEXEC changes - APE now supports a hybrid solution to no-self-modify for builds - Many BSD-only magnums added, e.g. O_SEARCH, O_SHLOCK, SF_NODISKIO
2021-08-12 07:42:14 +00:00
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
APE_COPY_SELF = \
o/$(MODE)/ape/ape.lds \
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
o/$(MODE)/ape/ape-copy-self.o
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-mno-red-zone \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
$<
Productionize new APE loader and more The APE_NO_MODIFY_SELF loader payload has been moved out of the examples folder and improved so that it works on BSD systems, and permits general elf program headers. This brings its quality up enough that it should be acceptable to use by default for many programs, e.g. Python, Lua, SQLite and Python. It's the responsibility of the user to define an appropriate TMPDIR if /tmp is considered an adversarial environment. Mac OS shall be supported by APE_NO_MODIFY_SELF soon. Fixes and improvements have been made to program_executable_name as it's now the one true way to get the absolute path of the executing image. This change fixes a memory leak in linenoise history loading, introduced by performance optimizations in 51904e2687c04d7ae20410cd94c2148972d6bae6 This change fixes a longstanding regression with Mach system calls, that 23ae9dfceb6f657862f00e674a8e4dc357a9d24d back in February which impacted our sched_yield() implementation, which is why no one noticed until now. The Blinkenlights PC emulator has been improved. We now fix rendering on XNU and BSD by not making the assumption that the kernel terminal driver understands UTF8 since that seems to break its internal modeling of \r\n which is now being addressed by using \e[𝑦H instead. The paneling is now more compact in real mode so you won't need to make your font as tiny if you're only emulating an 8086 program. The CLMUL ISA is now emulated too This change also makes improvement to time. CLOCK_MONOTONIC now does the right thing on Windows NT. The nanosecond time module functions added in Python 3.7 have been backported. This change doubles the performance of Argon2 password stretching simply by not using its copy_block and xor_block helper functions, as they were trivial to inline thus resulting in us needing to iterate over each 1024 byte block four fewer times. This change makes code size improvements. _PyUnicode_ToNumeric() was 64k in size and now it's 10k. The CJK codec lookup tables now use lazy delta zigzag deflate (δzd) encoding which reduces their size from 600k to 200k plus the code bloat caused by macro abuse in _decimal.c is now addressed so our fully-loaded statically-linked hermetically-sealed Python virtual interpreter container is now 9.4 megs in the default build mode and 5.5m in MODE=tiny which leaves plenty of room for chibicc. The pydoc web server now accommodates the use case of people who work by SSH'ing into a different machine w/ python.com -m pydoc -p8080 -h0.0.0.0 Finally Python Capsulae delenda est and won't be supported in the future
2021-10-02 15:17:04 +00:00
APE_SRCS_C = ape/loader.c
APE_SRCS_S = $(filter %.S,$(APE_FILES))
APE_SRCS = $(APE_SRCS_C) $(APE_SRCS_S)
APE_OBJS = $(APE_SRCS_S:%.S=o/$(MODE)/%.o)
Add scouts honor escape hatch for source embedding
2020-06-16 02:01:28 +00:00
APE_CHECKS = $(APE_HDRS:%=o/%.ok)
Initial import
2020-06-15 14:18:57 +00:00
Improve open source compatibility This change tunes the default stack size for the outside world to 8mb while at the same time, reducing Cosmopolitan's default stack size to 64kb. You can override the stack size using STATIC_STACK_SIZE(). Your build scripts should point to o//ape/public/ape.lds This change also fixes the definition of SOMAXCONN and removes AF_RDS since it's not polyfilled and Python 3.11 complained.
2022-09-08 09:48:17 +00:00
o/$(MODE)/ape/public/ape.lds: OVERRIDE_CPPFLAGS += -UCOSMO
o/$(MODE)/ape/public/ape.lds: \
ape/public/ape.lds \
ape/ape.lds \
Get aarch64 hello world working $ m=aarch64-tiny $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64 $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com hello world $ ls -hal o/$m/tool/hello/hello.com -rwxr-xr-x 1 jart jart 4.0K May 9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-09 09:35:05 +00:00
ape/ape.internal.h \
Improve open source compatibility This change tunes the default stack size for the outside world to 8mb while at the same time, reducing Cosmopolitan's default stack size to 64kb. You can override the stack size using STATIC_STACK_SIZE(). Your build scripts should point to o//ape/public/ape.lds This change also fixes the definition of SOMAXCONN and removes AF_RDS since it's not polyfilled and Python 3.11 complained.
2022-09-08 09:48:17 +00:00
ape/macros.internal.h \
ape/relocations.h \
libc/intrin/bits.h \
Clean up the TLS code
2022-09-10 18:49:13 +00:00
libc/thread/tls.h \
Improve open source compatibility This change tunes the default stack size for the outside world to 8mb while at the same time, reducing Cosmopolitan's default stack size to 64kb. You can override the stack size using STATIC_STACK_SIZE(). Your build scripts should point to o//ape/public/ape.lds This change also fixes the definition of SOMAXCONN and removes AF_RDS since it's not polyfilled and Python 3.11 complained.
2022-09-08 09:48:17 +00:00
libc/calls/struct/timespec.h \
Clean up threading code some more
2022-09-13 21:57:38 +00:00
libc/thread/thread.h \
Improve open source compatibility This change tunes the default stack size for the outside world to 8mb while at the same time, reducing Cosmopolitan's default stack size to 64kb. You can override the stack size using STATIC_STACK_SIZE(). Your build scripts should point to o//ape/public/ape.lds This change also fixes the definition of SOMAXCONN and removes AF_RDS since it's not polyfilled and Python 3.11 complained.
2022-09-08 09:48:17 +00:00
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.h
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
o/ape/idata.inc: \
ape/idata.internal.h \
Add scouts honor escape hatch for source embedding
2020-06-16 02:01:28 +00:00
ape/relocations.h
Initial import
2020-06-15 14:18:57 +00:00
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
o/$(MODE)/ape/ape-no-modify-self.o: \
ape/ape.S \
ape/macros.internal.h \
ape/notice.inc \
ape/relocations.h \
Get aarch64 hello world working $ m=aarch64-tiny $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64 $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com hello world $ ls -hal o/$m/tool/hello/hello.com -rwxr-xr-x 1 jart jart 4.0K May 9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-09 09:35:05 +00:00
ape/ape.internal.h \
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
libc/dce.h \
libc/elf/def.h \
libc/intrin/asancodes.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
Fix missing Makefile dependency
2022-11-06 08:01:34 +00:00
libc/calls/metalfile.internal.h \
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
libc/nexgen32e/vidya.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/runtime/symbolic.h \
libc/sysv/consts/prot.h \
Support aarch64, silicon, arm, i486, riscv64, mipsel, ppc and s390x Your APE binaries will now run on eight hardware architectures. The tradeoff is this change adds about 800kb of bloat to the executable which to be fair is still more than half as small as Go hello world Your APE shell script extracts an embedded build of Blink, based on `$(uname -m)`. The Blink binaries are statically-linked (manylinux) which means, they'll run on every distro's stock install, including Apple M1's that haven't enabled Rosetta. Blink's emulation has been designed to be nearly as fast as Rosetta, using a JIT technique for faster startup time. Blink has also been tested more thoroughly, w/ emulating Cosmopolitan software, since we're the ones who wrote it.
2023-01-05 22:29:21 +00:00
ape/blink-aarch64.gz \
ape/blink-darwin-arm64.gz \
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
o/$(MODE)/ape/ape.elf
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF \
Make build hermetic without shell scripts - Fix some minor issues in ar.com - Have execve() look for `ape` command - Rewrite NT paths using /c/ rather /??/c:/ - Replace broken GCC symlinks with .sym files - Rewrite $PATH environment variables on startup - Make $(APE_NO_MODIFY_SELF) the default bootloader - Add all build command dependencies to build/bootstrap - Get the repository mostly building from source on non-Linux
2022-05-25 18:31:08 +00:00
-DAPE_LOADER='"o/$(MODE)/ape/ape.elf"' $<
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
o/$(MODE)/ape/ape-copy-self.o: \
ape/ape.S \
ape/macros.internal.h \
ape/notice.inc \
ape/relocations.h \
Get aarch64 hello world working $ m=aarch64-tiny $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64 $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com hello world $ ls -hal o/$m/tool/hello/hello.com -rwxr-xr-x 1 jart jart 4.0K May 9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-09 09:35:05 +00:00
ape/ape.internal.h \
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
libc/dce.h \
libc/elf/def.h \
libc/intrin/asancodes.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
Fix missing Makefile dependency
2022-11-06 08:01:34 +00:00
libc/calls/metalfile.internal.h \
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
libc/nexgen32e/vidya.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/runtime/symbolic.h \
Support aarch64, silicon, arm, i486, riscv64, mipsel, ppc and s390x Your APE binaries will now run on eight hardware architectures. The tradeoff is this change adds about 800kb of bloat to the executable which to be fair is still more than half as small as Go hello world Your APE shell script extracts an embedded build of Blink, based on `$(uname -m)`. The Blink binaries are statically-linked (manylinux) which means, they'll run on every distro's stock install, including Apple M1's that haven't enabled Rosetta. Blink's emulation has been designed to be nearly as fast as Rosetta, using a JIT technique for faster startup time. Blink has also been tested more thoroughly, w/ emulating Cosmopolitan software, since we're the ones who wrote it.
2023-01-05 22:29:21 +00:00
libc/sysv/consts/prot.h \
ape/blink-aarch64.gz \
Bump embedded blink binary to 0.9.0 This is an official tagged version. The 1.1 version used earlier was informal. Formal versions will always have major.minor.patch going forward. See https://github.com/jart/blink/tags 1.0.0 coming soon.
2023-02-23 17:57:08 +00:00
ape/blink-darwin-arm64.gz
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
-DAPE_NO_MODIFY_SELF $<
Initial import
2020-06-15 14:18:57 +00:00
Prove that Makefile is fully defined The whole repository is now buildable with GNU Make Landlock sandboxing. This proves that no Makefile targets exist which touch files other than their declared prerequisites. In order to do this, we had to: 1. Stop code morphing GCC output in package.com and instead run a newly introduced FIXUPOBJ.COM command after GCC invocations. 2. Disable all the crumby Python unit tests that do things like create files in the current directory, or rename() files between folders. This ended up being a lot of tests, but most of them are still ok. 3. Introduce an .UNSANDBOXED variable to GNU Make to disable Landlock. We currently only do this for things like `make tags`. 4. This change deletes some GNU Make code that was preventing the execve() optimization from working. This means it should no longer be necessary in most cases for command invocations to be indirected through the cocmd interpreter. 5. Missing dependencies had to be declared in certain places, in cases where they couldn't be automatically determined by MKDEPS.COM 6. The libcxx header situation has finally been tamed. One of the things that makes this difficult is MKDEPS.COM only wants to consider the first 64kb of a file, in order to go fast. But libcxx likes to have #include lines buried after huge documentation. 7. An .UNVEIL variable has been introduced to GNU Make just in case we ever wish to explicitly specify additional things that need to be whitelisted which aren't strictly prerequisites. This works in a manner similar to the recently introduced .EXTRA_PREREQS feature. There's now a new build/bootstrap/make.com prebuilt binary available. It should no longer be possible to write invalid Makefile code.
2022-08-06 10:51:50 +00:00
o/$(MODE)/ape/loader.o: ape/loader.c ape/loader.h
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b01111001 -g $(APE_LOADER_FLAGS)
Productionize new APE loader and more The APE_NO_MODIFY_SELF loader payload has been moved out of the examples folder and improved so that it works on BSD systems, and permits general elf program headers. This brings its quality up enough that it should be acceptable to use by default for many programs, e.g. Python, Lua, SQLite and Python. It's the responsibility of the user to define an appropriate TMPDIR if /tmp is considered an adversarial environment. Mac OS shall be supported by APE_NO_MODIFY_SELF soon. Fixes and improvements have been made to program_executable_name as it's now the one true way to get the absolute path of the executing image. This change fixes a memory leak in linenoise history loading, introduced by performance optimizations in 51904e2687c04d7ae20410cd94c2148972d6bae6 This change fixes a longstanding regression with Mach system calls, that 23ae9dfceb6f657862f00e674a8e4dc357a9d24d back in February which impacted our sched_yield() implementation, which is why no one noticed until now. The Blinkenlights PC emulator has been improved. We now fix rendering on XNU and BSD by not making the assumption that the kernel terminal driver understands UTF8 since that seems to break its internal modeling of \r\n which is now being addressed by using \e[𝑦H instead. The paneling is now more compact in real mode so you won't need to make your font as tiny if you're only emulating an 8086 program. The CLMUL ISA is now emulated too This change also makes improvement to time. CLOCK_MONOTONIC now does the right thing on Windows NT. The nanosecond time module functions added in Python 3.7 have been backported. This change doubles the performance of Argon2 password stretching simply by not using its copy_block and xor_block helper functions, as they were trivial to inline thus resulting in us needing to iterate over each 1024 byte block four fewer times. This change makes code size improvements. _PyUnicode_ToNumeric() was 64k in size and now it's 10k. The CJK codec lookup tables now use lazy delta zigzag deflate (δzd) encoding which reduces their size from 600k to 200k plus the code bloat caused by macro abuse in _decimal.c is now addressed so our fully-loaded statically-linked hermetically-sealed Python virtual interpreter container is now 9.4 megs in the default build mode and 5.5m in MODE=tiny which leaves plenty of room for chibicc. The pydoc web server now accommodates the use case of people who work by SSH'ing into a different machine w/ python.com -m pydoc -p8080 -h0.0.0.0 Finally Python Capsulae delenda est and won't be supported in the future
2021-10-02 15:17:04 +00:00
o/$(MODE)/ape/loader-gcc.asm: ape/loader.c
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b01111001 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=0b01111001 -S -g0 $(APE_LOADER_FLAGS)
Prove that Makefile is fully defined The whole repository is now buildable with GNU Make Landlock sandboxing. This proves that no Makefile targets exist which touch files other than their declared prerequisites. In order to do this, we had to: 1. Stop code morphing GCC output in package.com and instead run a newly introduced FIXUPOBJ.COM command after GCC invocations. 2. Disable all the crumby Python unit tests that do things like create files in the current directory, or rename() files between folders. This ended up being a lot of tests, but most of them are still ok. 3. Introduce an .UNSANDBOXED variable to GNU Make to disable Landlock. We currently only do this for things like `make tags`. 4. This change deletes some GNU Make code that was preventing the execve() optimization from working. This means it should no longer be necessary in most cases for command invocations to be indirected through the cocmd interpreter. 5. Missing dependencies had to be declared in certain places, in cases where they couldn't be automatically determined by MKDEPS.COM 6. The libcxx header situation has finally been tamed. One of the things that makes this difficult is MKDEPS.COM only wants to consider the first 64kb of a file, in order to go fast. But libcxx likes to have #include lines buried after huge documentation. 7. An .UNVEIL variable has been introduced to GNU Make just in case we ever wish to explicitly specify additional things that need to be whitelisted which aren't strictly prerequisites. This works in a manner similar to the recently introduced .EXTRA_PREREQS feature. There's now a new build/bootstrap/make.com prebuilt binary available. It should no longer be possible to write invalid Makefile code.
2022-08-06 10:51:50 +00:00
o/$(MODE)/ape/loader-xnu.o: ape/loader.c ape/loader.h
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b00001000 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b00001000 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=0b00001000 -S -g0 $(APE_LOADER_FLAGS)
Provide option to have APE not modify itself This change introduces ape-no-modify-self.o to the amalgamated release binaries, which may be used as an alternative to ape.o to make it easier to use APE in cases where the self-modifying behavior isn't acceptable. Please note that this alternative copying behavior isn't necessarily better. It introduces a whole bunch of questions of its own, which are documented in the ape.S source comment and should be considered by both the program author as well as the end-user of programs linked this way. For example, build environments that use read-only file systems and would prefer to not have a launcher wrapper (like we use in our build) can use ape-no-modify-self.o instead of ape.o and then set the $TMPDIR environment variable to point to a sane read-write-exec location. Fixes #146 See #82
2021-04-08 04:01:57 +00:00
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
o/$(MODE)/ape/ape.macho: o/$(MODE)/ape/ape.macho.dbg
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/loader-elf.o \
Productionize new APE loader and more The APE_NO_MODIFY_SELF loader payload has been moved out of the examples folder and improved so that it works on BSD systems, and permits general elf program headers. This brings its quality up enough that it should be acceptable to use by default for many programs, e.g. Python, Lua, SQLite and Python. It's the responsibility of the user to define an appropriate TMPDIR if /tmp is considered an adversarial environment. Mac OS shall be supported by APE_NO_MODIFY_SELF soon. Fixes and improvements have been made to program_executable_name as it's now the one true way to get the absolute path of the executing image. This change fixes a memory leak in linenoise history loading, introduced by performance optimizations in 51904e2687c04d7ae20410cd94c2148972d6bae6 This change fixes a longstanding regression with Mach system calls, that 23ae9dfceb6f657862f00e674a8e4dc357a9d24d back in February which impacted our sched_yield() implementation, which is why no one noticed until now. The Blinkenlights PC emulator has been improved. We now fix rendering on XNU and BSD by not making the assumption that the kernel terminal driver understands UTF8 since that seems to break its internal modeling of \r\n which is now being addressed by using \e[𝑦H instead. The paneling is now more compact in real mode so you won't need to make your font as tiny if you're only emulating an 8086 program. The CLMUL ISA is now emulated too This change also makes improvement to time. CLOCK_MONOTONIC now does the right thing on Windows NT. The nanosecond time module functions added in Python 3.7 have been backported. This change doubles the performance of Argon2 password stretching simply by not using its copy_block and xor_block helper functions, as they were trivial to inline thus resulting in us needing to iterate over each 1024 byte block four fewer times. This change makes code size improvements. _PyUnicode_ToNumeric() was 64k in size and now it's 10k. The CJK codec lookup tables now use lazy delta zigzag deflate (δzd) encoding which reduces their size from 600k to 200k plus the code bloat caused by macro abuse in _decimal.c is now addressed so our fully-loaded statically-linked hermetically-sealed Python virtual interpreter container is now 9.4 megs in the default build mode and 5.5m in MODE=tiny which leaves plenty of room for chibicc. The pydoc web server now accommodates the use case of people who work by SSH'ing into a different machine w/ python.com -m pydoc -p8080 -h0.0.0.0 Finally Python Capsulae delenda est and won't be supported in the future
2021-10-02 15:17:04 +00:00
ape/loader.lds
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
@$(ELFLINK) -z max-page-size=0x10
Undiamond Python headers This change gets the Python codebase into a state where it conforms to the conventions of this codebase. It's now possible to include headers from Python, without worrying about ordering. Python has traditionally solved that problem by "diamonding" everything in Python.h, but that's problematic since it means any change to any Python header invalidates all the build artifacts. Lastly it makes tooling not work. Since it is hard to explain to Emacs when I press C-c C-h to add an import line it shouldn't add the header that actually defines the symbol, and instead do follow the nonstandard Python convention. Progress has been made on letting Python load source code from the zip executable structure via the standard C library APIs. System calss now recognizes zip!FILENAME alternative URIs as equivalent to zip:FILENAME since Python uses colon as its delimiter. Some progress has been made on embedding the notice license terms into the Python object code. This is easier said than done since Python has an extremely complicated ownership story. - Some termios APIs have been added - Implement rewinddir() dirstream API - GetCpuCount() API added to Cosmopolitan Libc - More bugs in Cosmopolitan Libc have been fixed - zipobj.com now has flags for mangling the path - Fixed bug a priori with sendfile() on certain BSDs - Polyfill F_DUPFD and F_DUPFD_CLOEXEC across platforms - FIOCLEX / FIONCLEX now polyfilled for fast O_CLOEXEC changes - APE now supports a hybrid solution to no-self-modify for builds - Many BSD-only magnums added, e.g. O_SEARCH, O_SHLOCK, SF_NODISKIO
2021-08-12 07:42:14 +00:00
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
o/$(MODE)/ape/ape.macho.dbg: \
o/$(MODE)/ape/loader-xnu.o \
o/$(MODE)/ape/loader-macho.o \
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
ape/loader-macho.lds
Write tests for new APE loader and fix bugs - Add FreeBSD-specific mmap() flags - Reduce size of the APE loader from 8kb to 4kb - Work towards fixing the Makefile build on WSL - Automate testing of APE no-modify-self behaviors - Make the ape.S shell script code cleaner and tinier - Improve the APE sanity check to test behavior better - Fixed issue with ShowCrashReports() sigaltstack() on BSDs - Delete symbols for S_MODE magnums which wasted compile time If you checked out yesterday's APE commit, please run: rm -f /usr/bin/ape o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape" Because this change fixes certain aspects of the new ABI. We don't have automated migrations for APE loader versions yet. Thanks! You can also download prebuilt binaries here: - https://justine.lol/ape.elf (Linux/FreeBSD/NetBSD/OpenBSD) - https://justine.lol/ape.macho (Apple) Install the appropriate one as `/usr/bin/ape`.
2022-05-22 11:51:02 +00:00
@$(ELFLINK) -z max-page-size=0x10
Support Linux binfmt_misc and APE loading on Apple The "no modify self" variant of Actually Portable Executable is now supported on all platforms. If you use `$(APE_NO_MODIFY_SELF)` then ld.bfd will embed a 4096 byte ELF binary and a 4096 byte Macho file which are installed on the fly to ${TMPDIR:-/tmp}, which enables us launch the executable, without needing to copy the whole executable To prevent it from copying a tiny executable to your temp directory you need to install the `ape` command (renamed from ape-loader), to a system path. For example: # FreeBSD / NetBSD / OpenBSD make -j8 o//ape/ape cp o//ape/ape /usr/bin/ape # Mac OS # make -j8 o//ape/ape.macho curl https://justine.lol/ape.macho >/usr/bin/ape chmod +x /usr/bin/ape On Linux you can get even more performance with the new binfmt_misc support which makes launching non-modifying APE binaries as fast as launching ELF executables. Running the following command: # Linux ape/apeinstall.sh Will copy APE loader to /usr/bin/ape and register with binfmt_misc Lastly, this change also fixes a really interesting race condition with OpenBSD thread joining.
2022-05-21 14:52:58 +00:00
Initial import
2020-06-15 14:18:57 +00:00
.PHONY: o/$(MODE)/ape
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
o/$(MODE)/ape: $(APE_CHECKS) \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds \
Improve open source compatibility This change tunes the default stack size for the outside world to 8mb while at the same time, reducing Cosmopolitan's default stack size to 64kb. You can override the stack size using STATIC_STACK_SIZE(). Your build scripts should point to o//ape/public/ape.lds This change also fixes the definition of SOMAXCONN and removes AF_RDS since it's not polyfilled and Python 3.11 complained.
2022-09-08 09:48:17 +00:00
o/$(MODE)/ape/public/ape.lds \
Remove plenty of makefile misconfigurations
2022-07-21 16:16:38 +00:00
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.macho \
o/$(MODE)/ape/ape-copy-self.o \
Provide option to have APE not modify itself This change introduces ape-no-modify-self.o to the amalgamated release binaries, which may be used as an alternative to ape.o to make it easier to use APE in cases where the self-modifying behavior isn't acceptable. Please note that this alternative copying behavior isn't necessarily better. It introduces a whole bunch of questions of its own, which are documented in the ape.S source comment and should be considered by both the program author as well as the end-user of programs linked this way. For example, build environments that use read-only file systems and would prefer to not have a launcher wrapper (like we use in our build) can use ape-no-modify-self.o instead of ape.o and then set the $TMPDIR environment variable to point to a sane read-write-exec location. Fixes #146 See #82
2021-04-08 04:01:57 +00:00
o/$(MODE)/ape/ape-no-modify-self.o
Do some basic build tuning
2023-05-01 21:36:25 +00:00
endif
Get LIBC_RUNTIME and LIBC_CALLS building on aarch64
2023-05-09 08:56:56 +00:00
# these assembly files are safe to build on aarch64
o/$(MODE)/ape/ape.o: ape/ape.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/ape.o: \
ape/blink-aarch64.gz \
ape/blink-darwin-arm64.gz
o/$(MODE)/ape/ape.lds: \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
Get aarch64 hello world working $ m=aarch64-tiny $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64 $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com hello world $ ls -hal o/$m/tool/hello/hello.com -rwxr-xr-x 1 jart jart 4.0K May 9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-09 09:35:05 +00:00
ape/ape.internal.h \
Get LIBC_RUNTIME and LIBC_CALLS building on aarch64
2023-05-09 08:56:56 +00:00
libc/intrin/bits.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.h
Reference in a new issue Copy permalink