Commit graph

1020 commits

Author SHA1 Message Date
Justine Tunney
8ab3a545c6
Increase build memory quota
If you install qemu-user from apt then glibc links a lot of address
space bloat that causes pthread_create() to ENOMEM (a.k.a. EAGAIN).
Boosting the virtual memory quota from 512m to 2048m will hopefully
future proof the build for the future, as Linux distros get fatter.
Please note this only applies to MODE=aarch64 on x86_64 builds when
you're using QEMU from Debian/Ubuntu rather than installing the one
cosmo provides in third_party/qemu/qemu-aarch64.gz. This change may
also be useful to people who are using the host compiler toolchain.
2024-01-22 10:02:30 -08:00
BONNAURE Olivier
51cd83f674
Add Curve25519 method in help.txt + add new specs file (#1096) 2024-01-21 06:58:48 -08:00
Justine Tunney
d50064a779
Add Curve25519() API to Redbean 2024-01-20 01:06:19 -08:00
Brian
08793aa143
Make ... optional in .args (#1086) 2024-01-15 16:20:50 -08:00
Trung Nguyen
a2753de7fd
tool/cosmocc: Do not run fixupobj for text output (#1084)
Some compiler flags (such as -E or -MM) instruct GCC to only run the
preprocessor and produce certain text files.

In this case, we do not want to run `fixupobj` and make the tool fail
because the input is not an ELF64 binary.
2024-01-15 07:16:13 -08:00
Justine Tunney
5d80e8dbf6
Add more Intel microarchitectures 2024-01-09 01:54:26 -08:00
Justine Tunney
cb19e172da
Release Cosmopolitan v3.2.4 2024-01-08 19:37:59 -08:00
Justine Tunney
a4b455185b
Bring back gc() function
Renaming gc() to _gc() was a mistake since the better thing to do is put
it behind the _COSMO_SOURCE macro. We need this change because I haven't
wanted to use my amazing garbage collector ever since we renamed it. You
now need to define _COSMO_SOURCE yourself when using amalgamation header
and cosmocc users need to pass the -mcosmo flag to get the gc() function

Some other issues relating to cancelation have been fixed along the way.
We're also now putting cosmocc in a folder named `.cosmocc` so it can be
more safely excluded by grep --exclude-dir=.cosmocc --exclude-dir=o etc.
2024-01-08 10:26:28 -08:00
Jōshin
6cb0354e19
mkofs improvements
- Always use $PWD/o. if it doesn't exist, say how to create it.

- return 2 on unsupported platforms.
2024-01-08 17:23:09 +00:00
Trung Nguyen
b580080af1
tool/cosmocc: Allow exceptions and rtti (#1074)
With `libunwind` and `libcxxabi` included in `libcosmo`, we can now
allow users to build C++ applications with exceptions and RTTI enabled.

The default is still disabling these two to avoid bloating the binary.

Closes #1065
2024-01-08 08:47:53 -08:00
Jōshin
cafea9a0a5
Flesh out tool/zsh/mkofs
It now prevents double-mounting.

Also adds vim filetype modelines.
2024-01-08 05:18:18 +00:00
Jōshin
16099801cc
zsh mkofs tool 2024-01-08 03:43:09 +00:00
Jōshin
2e5f662dfe
mmake: different cosmocc regime
Now that cosmocc is unpacked into a version-specific directory under
cosmocc/, it makes more sense to put the versions in /opt/cosmocc and
maintain a symlink to the currently active one.
2024-01-06 19:14:13 -05:00
Justine Tunney
ce17ed60df
Support -MF and -MT in cosmocc 2024-01-06 12:10:06 -08:00
Justine Tunney
0de6a08988
Upgrade mono repo to cosmocc 3.2
The toolchain will now be downloaded going forward from multiple pinned
URLs which have shasums. Either wget or curl must be installed.

This change unblocks #1053
2024-01-05 08:02:04 -08:00
Paul Kulchenko
a334f9cc33
Update redbean ProgramDirectory to return a list of previously set directories (#1021) 2024-01-05 02:17:55 -08:00
Justine Tunney
0e49bed660
Support 40 cosmo_dlopen() function parameters
Our dynamic linking implementation is now able to support functions with
dozens of parameters. In addition to having extra integral arguments you
can now pass vector registers using intrinsic types. Lastly, you can now
return multiple values, which is useful for functions returning structs.
2024-01-04 13:41:26 -08:00
Justine Tunney
a3deef70c2
Release Cosmopolitan v3.2 2024-01-04 09:39:48 -08:00
Justine Tunney
873069fcd7
Fix -Xaarch64 in cosmocc
It's now possible to pass flags like -Xaarch64-march=armv8.2-a+dotprod
so that cosmocc will use newer ARM ISAs. For AMD64 there's another one
worth mentioning, which looks like this: -Xx86_64-mssse3
2024-01-04 07:22:05 -08:00
Justine Tunney
796148790f
Remove hard coded paths from APE bootloader
This increases risk of fork bomb but is needed to support the NixOS.
Upstream dependencies of APE (uname, mkdir, dd, chmod, gzip, and mv)
will be removed from releases, and deleted from the cosmo.zip server

See #12
2024-01-03 17:55:19 -08:00
Jōshin
2902b76168
mmake gets smarter about locating make
- Check `$COSMOCC`, defaulting to `/opt/cosmocc`.

- Try to get the full path of the repo `make.com`. I'm not aware of
  a way of getting the path that defines a zsh function, so the best
  fallback available is `$PWD`.
2024-01-01 23:34:35 -05:00
Justine Tunney
307823ae00
Upgrade to superconfigure z0.0.29 2024-01-01 19:31:58 -08:00
Justine Tunney
81949f038e
Mint APE Loader v1.10 2023-12-31 11:43:13 -08:00
Justine Tunney
83107f78ed
Introduce FreeBSD ARM64 support
It's 100% passing test fleet. Solid as a rock.
2023-12-29 20:14:02 -08:00
Justine Tunney
43fe5956ad
Use DNS implementation from Musl Libc
Now that our socket system call polyfills are good enough to support
Musl's DNS library we should be using that rather than the barebones
domain name system implementation we rolled on our own. There's many
benefits to making this change. So many, that I myself wouldn't feel
qualified to enumerate them all. The Musl DNS code had to be changed
in order to support Windows of course, which looks very solid so far
2023-12-28 23:04:35 -08:00
Paul Kulchenko
6be9477b9e
Fix redbean to only check additional content types when Lua is present (#1033)
Closes #971
2023-12-28 01:08:19 -08:00
Jōshin
fd772b9b2a
mmake uses $MAKE if available 2023-12-26 10:27:04 -05:00
Jōshin
bb2602a524
Minor mmake cleanup 2023-12-23 11:09:50 -05:00
mataha
1bc48bc8e4
Update stb (#885)
This commit and, by extension, PR attempts to update `stb` in the most
straightforward way possible as well as include fixes from main repo's
unmerged PRs for cases rearing their ugly heads during everyday usage:

 - stb#1299: stb_rect_pack: Make rect_height_compare a stable sort
 - stb#1402: stb_image: Fix "unused invalid_chunk" with STBI_FAILURE_USERMSG
 - stb#1404: stb_image: Fix gif two_back memory address
 - stb#1420: stb_image: Improve error reporting if file operations fail
   within *_from_file functions
 - stb#1445: stb_vorbis: Few static analyzers fixes
 - stb#1487: stb_vorbis: Fix residue classdata bounding for
   f->temp_memory_required
 - stb#1490: stb_vorbis: Fix broken clamp in codebook_decode_deinterleave_repeat
 - stb#1496: stb_image: Fix pnm only build
 - stb#1497: stb_image: Fix memory leaks if stbi__convert failed
 - stb#1498: stb_vorbis: Fix memory leaks in stb_vorbis
 - stb#1499: stb_vorbis: Minor change to prevent the undefined behavior -
   left shift of a negative value
 - stb#1500: stb_vorbis: Fix signed integer overflow

Includes additional small fixes that I felt didn't warrant a separate PR.
2023-12-22 21:39:27 -08:00
Jōshin
85e4c020a3
mmake: formatting + aarch64 tab-completion helper
o/aarch64/* is turned into o/$mode/* in a way analogous to o//, so
aarch64 systems can use tab complete for alternate modes just like
x86_64 can.
2023-12-18 21:29:32 -05:00
Jōshin
f73576ab8a
mmake knows about some .PHONY targets 2023-12-17 17:04:57 -05:00
Jōshin
dd57c685aa
mmake gets smarter
`o/$mode/*` is passed through as-is. `o/*` other than `$mode` has
`$mode` inserted. `*` has `o/$mode/` prepended.

Really leveraging zsh default tab completion here; if you have built
things with `MODE=` you can leverage that for perfect tab completion
in other modes.
2023-12-17 15:42:27 -05:00
Jōshin
c11c105270
mmake doesn't even run nproc unless it needs to 2023-12-17 15:09:41 -05:00
Jōshin
3654650c52
mmake only autoloads nproc if needed 2023-12-17 15:07:10 -05:00
Jōshin
b08b92248b
mmake: -j$(nproc), passthrough o//
Now `mmake o/ape/ape.elf` works.
2023-12-17 01:55:52 -05:00
Jōshin
3a8e01a77a
more modeline errata (#1019)
Somehow or another, I previously had missed `BUILD.mk` files.

In the process I found a few straggler cases where the modeline was
different from the file, including one very involved manual fix where a
file had been treated like it was ts=2 and ts=8 on separate occasions.

The commit history in the PR shows the gory details; the BUILD.mk was
automated, everything else was mostly manual.
2023-12-16 23:07:10 -05:00
Jōshin
60813003a3
tool/zsh/mmake cleanup
- collects all flags into their own array to support `mmake $target -j8`

- prints out the raw make invocation to be run
2023-12-16 19:22:17 -05:00
Jōshin
c0bbc45268
Add mmake zsh helper function (#1017) 2023-12-16 08:47:38 -08:00
Justine Tunney
897fa6ac00
Add some Emacs keywords for CUDA 2023-12-14 19:55:56 -08:00
Jōshin
2fc507c98f
Fix more vi modelines (#1006)
* modelines: tw -> sw

shiftwidth, not textwidth.

* space-surround modelines

* fix irregular modelines

* Fix modeline in titlegen.c
2023-12-13 02:28:11 -05:00
Justine Tunney
bd10cf9d38
Whip up find command for PE binaries
Using this shell script:

    #!/bin/sh
    mkdir -p exe
    for f in $(findpe); do
      if [ -e exe/${f##*/}.exe ]; then
        cp $f exe/${f##*/}-$(rand64).exe
      else
        cp $f exe/${f##*/}.exe
      fi
    done
    rm -f /mnt/videos/microsoft.zip
    zip -rj6 /mnt/videos/microsoft.zip exe
    echo /mnt/videos/microsoft.zip

Helps file reports with Microsoft about incorrect AV detections.

See #1003
2023-12-12 10:29:03 -08:00
Justine Tunney
1bd69867d1
Explain how to build FOSS in cosmocc README 2023-12-09 06:58:24 -08:00
Justine Tunney
1a96de6eda
Add libresolv from Musl Libc
Locally modified to get nameservers from Windows Registry when
`\etc\resolv.conf` isn't defined.
2023-12-08 20:04:10 -08:00
Jōshin
e16a7d8f3b
flip et / noet in modelines
`et` means `expandtab`.

```sh
rg 'vi: .* :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\) et\(.*\)  :vi/vi: \1 xoet\2:vi/'
rg 'vi: .*  :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\)noet\(.*\):vi/vi: \1et\2  :vi/'
rg 'vi: .*  :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\)xoet\(.*\):vi/vi: \1noet\2:vi/'
```
2023-12-07 22:17:11 -05:00
Jōshin
394d998315
Fix vi modelines (#989)
At least in neovim, `│vi:` is not recognized as a modeline because it
has no preceding whitespace. After fixing this, opening a file yields
an error because `net` is not an option. (`noet`, however, is.)
2023-12-05 14:37:54 -08:00
Justine Tunney
69ba516466
Revert "Re-enable apelink size optimization"
This reverts commit 7b3d7ee07b.

It broke llava-v1.5-7b-q4-server.llamafile on Windows.
2023-12-01 10:26:17 -08:00
Jōshin
7845495a93
Get tool/cosmocc/package.sh working on M1 (#970)
On aarch64 hosts, MODE= is changed to MODE=aarch64, so o// targets don't
work. So On aarch64, get apelink.com out of o/aarch64/. Also prepend ape
when calling it. And finally, fetch with curl when wget isn't installed.
2023-12-01 00:08:55 -08:00
Justine Tunney
26c70e08bf
Make it possible to package.sh cosmocc w/ modes 2023-11-30 07:12:15 -08:00
Justine Tunney
20938a45e6
Add appropriate chmod to README instructions
See Mozilla-Ocho/llamafile#13
2023-11-30 03:35:17 -08:00
Justine Tunney
7b3d7ee07b
Re-enable apelink size optimization
I'm no longer able to reproduce the PE import table corruption that was
previously observed. Since this optimization shaves up to 64kb off each
fat APE binary we build, it's worth turning this back on again, to wait
and see if something breaks, and if so, fix it. At least until the next
release is shipped.

See #965
2023-11-29 05:41:23 -08:00
Justine Tunney
41c06c6325
Make cosmocc 100x faster on MacOS
This change upgrades to superconfigure z0.0.23 which fixes an issue
where the compiler had harmless /home/... paths baked-in, which are
normally only present in the build environment, and usually skipped
over. Sadly on MacOS calling fstatat() on these paths would lead to
cloud file system ops that caused system calls to take a long time.
That's problematic, since cosmocc needs to be a 100% local command.
2023-11-29 01:48:17 -08:00
Justine Tunney
03de761890
Update cosmocc to use superconfigure z0.0.23 2023-11-28 20:04:15 -08:00
Justine Tunney
fa20edc44d
Reduce header complexity
- Remove most __ASSEMBLER__ __LINKER__ ifdefs
- Rename libc/intrin/bits.h to libc/serialize.h
- Block pthread cancelation in fchmodat() polyfill
- Remove `clang-format off` statements in third_party
2023-11-28 14:39:42 -08:00
Justine Tunney
96f979dfc5
Rename makefiles BUILD.mk
This way they appear at the top of directory listings.
2023-11-28 11:21:08 -08:00
Paul Kulchenko
0bffd09433
Allow redbean to accept --assimilate option without complaining (#916)
This allows using --assimilate on Windows or already assimilated binary.
Ref #356.
2023-11-19 19:02:30 -08:00
Justine Tunney
69faf1b403
Release Cosmpolitan v3.1 2023-11-18 20:40:35 -08:00
Justine Tunney
0c89516ac5
Fix unzip warning with apelink generated symtabs 2023-11-18 17:15:22 -08:00
Justine Tunney
3e6d536822
Correct misunderstanding with zip64 extra records 2023-11-18 14:35:57 -08:00
Justine Tunney
dbd8176ea8
Add CPU_COUNT_S() 2023-11-18 12:38:30 -08:00
Justine Tunney
72ac5f18d9
Fix dash compatible syntax issue 2023-11-17 16:47:47 -08:00
Justine Tunney
68c7c9c1e0
Clean up some code
- Use good ELF technique in cosmo_dlopen()
- Make strerror() conform more to other libc impls
- Introduce __clear_cache() and use it in cosmo_dlopen()
- Remove libc/fmt/fmt.h header (trying to kill off LIBC_FMT)
2023-11-16 17:31:07 -08:00
tkchia
7010a8081e
Fix regression when building deathstar.com for aarch64 (#957)
It turns out my earlier commit ddc08dc974 caused a build with
MODE=aarch64 to fail.  The commit changed deathstar.c to link
in code to support a VGA console, but this is not implemented
yet for AArch64.  Thanks to @ahgamut for spotting this issue.
2023-11-16 12:13:17 -08:00
Justine Tunney
8318d67503
Fix regression in cosmocc toolchain 2023-11-15 21:41:53 -08:00
Justine Tunney
1351d3cede
Remove bool from public headers 2023-11-15 20:58:46 -08:00
tkchia
ddc08dc974
[metal] Make deathstar.com draw onto both VGA and serial consoles (#955)
In addition, deathstar.com can now be booted up via UEFI.
2023-11-15 19:40:54 -08:00
Justine Tunney
e4584ace81
Get cosmo_dlopen() working better on System Five
Imported functions are now aspected with a trampoline that blocks
signals and changes the thread-local storage register. This means
bigger more complicated libraries can now be imported even though
the whole technique remains fundamentally unsafe.
2023-11-15 10:56:30 -08:00
Justine Tunney
9f9aec013a
Do better job documenting platform issues 2023-11-14 18:15:58 -08:00
Justine Tunney
d1e764b3cf
Modernize README.md 2023-11-12 22:04:07 -08:00
Paul Kulchenko
5c1fdc9819
Fix session changeset procesing (reverting some of 0d748ad5 changes) (#949)
This removes strdup from Lua SQLite wrapper, as it's applied to binary changeset
strings, thus cutting them short (because of strlen).
2023-11-12 14:25:50 -08:00
Justine Tunney
f25beb341c
Add architecture flag to zipobj 2023-11-12 09:35:59 -08:00
Justine Tunney
bed77186c3
Use simple locks in dlmalloc 2023-11-12 09:00:49 -08:00
Justine Tunney
0863427b3a
Make development more pleasant on MacOS Arm64 2023-11-12 05:42:17 -08:00
Justine Tunney
c6d3802d3a
Add more fixes for new cosmocc toolchain
We now have an `#include <cxxabi.h>` header which defines all the APIs
Cosmopolitan's implemented so far. The `cosmocc` README.md file is now
greatly expanded with documentation.
2023-11-11 23:28:19 -08:00
Gautham
95124cacbe
Fixes for building superconfigure (#948) 2023-11-11 22:11:11 -08:00
Justine Tunney
291103ad8d
Redesign cosmocc toolchain
The `cosmocc` compiler is now being distributed as a self-contained
toolchain that's path-agnostic and it no longer requires you clone the
Cosmop repo to use it. The bin/ folder has been deleted from the mono
repo. The `fatcosmocc` command has been renamed to `cosmocc`. MacOS
support now works very well.
2023-11-11 14:18:08 -08:00
Justine Tunney
d0ad2694ed
Iterate more on recent changes 2023-11-11 00:28:22 -08:00
Paul Kulchenko
7138399f96
Close Lua state before exiting redbean worker to allow GC to collect (#941) 2023-11-10 18:37:00 -08:00
Justine Tunney
cdbd10c9d0
Introduce X86_CHECK() macro 2023-11-10 15:41:55 -08:00
Justine Tunney
3d3cdcf276
Fix APE uname direction
Fixes #942
2023-11-10 12:34:25 -08:00
Justine Tunney
d2f49ca175
Improve mkdeps
Our makefile generator now accepts badly formatted include lines. It's
now more hermetic with better error checking in the cosmo repo, and it
can be configured to not be hermetic at all.
2023-11-10 04:14:27 -08:00
Justine Tunney
ea28f93a26
Support <isystem> includes in monorepo 2023-11-08 19:14:56 -08:00
Justine Tunney
5330442d85
Avoid APE fork bomb
This change addresses a $PATH resolution issue where APE depends on
uname and uname is an APE program. So sorry to anyone this impacted
we'll get a release out soon.
2023-11-08 11:37:42 -08:00
mataha
cc3e3de044
Make args replace ... only at the end of file (#909)
So that the string '...' can be used verbatim elsewhere in the file.
2023-11-08 10:00:52 -08:00
Justine Tunney
e961385e55
Put more thought into i/o polyfills
wait4() is now solid enough to run `make -j100` on Windows. You can now
use MSG_DONTWAIT on Windows. There was a handle leak in accept() that's
been fixed. Our WIN32 overlapped i/o code has been simplified. Priority
class now inherits into subprocesses, so the verynice command will work
and the signal mask will now be inherited by execve() and posix_spawn()
2023-11-07 18:32:35 -08:00
Paul Kulchenko
b893479417
Add database and statement readonly checks to sqlite in redbean (#914) 2023-11-05 13:29:09 -08:00
Justine Tunney
585c86e2a4
Support \n in /zip/.args files 2023-11-04 07:25:20 -07:00
Justine Tunney
024be3b009
Introduce getifaddrs()
This function was invented by the BSDs (it's not in POSIX.1). It
provides a high-level interface into ioctl(SIOCGIFCONF) which is
comparatively clumsy to use. We already made the ioctls portable
across our entire support vector back in 2021, so this interface
is portable too. See o//tool/viz/getifaddrs.com for our demo app
2023-11-02 08:33:03 -07:00
Justine Tunney
7b284f6bda
Fix bugs and regressions in the pledge command
This change gets the pledge (formerly pledge.com) command back in tip
top shape for a 3.0.1 cosmos release. It now runs on all platforms, even
though it's mostly a no-op on ones that lack the kernel security stuff.
The binary footprint is now smaller, since it no longer needs to link
malloc. It's also now able to be built as a fat binary.
2023-11-01 06:08:58 -07:00
Justine Tunney
c9fecf3a55
Make improvements
- You can now run `make -j8 toolchain` on Windows
- You can now run `make -j` on MacOS ARM64 and BSD OSes
- You can now use our Emacs dev environment on MacOS/Windows
- Fix bug where the x16 register was being corrupted by --ftrace
- The programs under build/bootstrap/ are updated as fat binaries
- The Makefile now explains how to download cosmocc-0.0.12 toolchain
- The build scripts under bin/ now support "cosmo" branded toolchains
- stat() now goes faster on Windows (shaves 100ms off `make` latency)
- Code cleanup and added review on the Windows signal checking code
- posix_spawnattr_setrlimit() now works around MacOS ARM64 bugs
- Landlock Make now favors posix_spawn() on non-Linux/OpenBSD
- posix_spawn() now has better --strace logging on Windows
- fstatat() can now avoid EACCES in more cases on Windows
- fchmod() can now change the readonly bit on Windows
2023-10-15 16:45:00 -07:00
Justine Tunney
cdf556e7d2
Implement signal handler tail recursion
GNU Make on Windows now appears to be working reliably. This change also
fixes a bug where, after fork the Windows thread handle wasn't reset and
that caused undefined behavior using SetThreadContext() with our signals
2023-10-14 10:38:15 -07:00
Justine Tunney
a657f3e878
Delete old wcwidth() implementation
This shaves away 144kb of bss memory from every binary linking printf at
the expense of slightly increased binary footprint. Kudos for Byron Lai.
2023-10-14 03:15:05 -07:00
Justine Tunney
2db2f40a98
Rewrite special file handling on Windows
This change gets GNU grep working. What caused it to not work, is it
wouldn't write to an output file descriptor when its dev/ino equaled
/dev/null's. So now we invent special dev/ino values for these files
2023-10-14 02:53:34 -07:00
Justine Tunney
d1a283a588
Fix redbean LaunchBrowser() 2023-10-13 11:39:00 -07:00
Justine Tunney
bdc453b22d
Fix redbean "-1 workers remain" blemish 2023-10-13 08:33:09 -07:00
Justine Tunney
0e0398cd5d
Fix MODE=tinylinux build 2023-10-13 08:30:18 -07:00
Justine Tunney
4bcb107cb0
Fix ctrl-c in redbean on Windows 2023-10-13 08:10:03 -07:00
Justine Tunney
d458642790
Write more tests and improve kill() on Windows 2023-10-13 04:38:45 -07:00
Justine Tunney
f7343319cc
Cull the examples folder 2023-10-11 21:45:32 -07:00
Paul Kulchenko
f92ad74e6b
Update pidfile processing to truncate when writing pid (#910) 2023-10-11 21:45:27 -07:00
Paul Kulchenko
6e4b9b6515
Add EncodeBase32() to Redbean (#856) 2023-10-11 20:06:20 -07:00