Commit graph

485 commits

Author SHA1 Message Date
Justine Tunney
39bf41f4eb Make numerous improvements
- Python static hello world now 1.8mb
- Python static fully loaded now 10mb
- Python HTTPS client now uses MbedTLS
- Python REPL now completes import stmts
- Increase stack size for Python for now
- Begin synthesizing posixpath and ntpath
- Restore Python \N{UNICODE NAME} support
- Restore Python NFKD symbol normalization
- Add optimized code path for Intel SHA-NI
- Get more Python unit tests passing faster
- Get Python help() pagination working on NT
- Python hashlib now supports MbedTLS PBKDF2
- Make memcpy/memmove/memcmp/bcmp/etc. faster
- Add Mersenne Twister and Vigna to LIBC_RAND
- Provide privileged __printf() for error code
- Fix zipos opendir() so that it reports ENOTDIR
- Add basic chmod() implementation for Windows NT
- Add Cosmo's best functions to Python cosmo module
- Pin function trace indent depth to that of caller
- Show memory diagram on invalid access in MODE=dbg
- Differentiate stack overflow on crash in MODE=dbg
- Add stb_truetype and tools for analyzing font files
- Upgrade to UNICODE 13 and reduce its binary footprint
- COMPILE.COM now logs resource usage of build commands
- Start implementing basic poll() support on bare metal
- Set getauxval(AT_EXECFN) to GetModuleFileName() on NT
- Add descriptions to strerror() in non-TINY build modes
- Add COUNTBRANCH() macro to help with micro-optimizations
- Make error / backtrace / asan / memory code more unbreakable
- Add fast perfect C implementation of μ-Law and a-Law audio codecs
- Make strtol() functions consistent with other libc implementations
- Improve Linenoise implementation (see also github.com/jart/bestline)
- COMPILE.COM now suppresses stdout/stderr of successful build commands
2021-09-28 01:52:34 -07:00
Paul Kulchenko
fa7b4f5bd1 Updated lua-Argon2 to use Argon2_id variant
As recommended in RFC9106:
https://www.rfc-editor.org/rfc/rfc9106.html#name-recommendations
2021-09-27 21:42:40 -07:00
Paul Kulchenko
b883473a6c Remove optimized blamka rounds from Argon2
Optimized blamka rounds occasionally SIGSERV for me,
so the reference implementation is used instead.
2021-09-27 21:42:40 -07:00
Paul Kulchenko
af1153e057 Reorganize handling of added SSE intrinsics to simplify. 2021-09-27 21:42:40 -07:00
Paul Kulchenko
2e4b40042a Add missing Intel SSE2 intrinsics needed for Argon2 2021-09-27 21:42:40 -07:00
Paul Kulchenko
42aafc847e Update lua-argon2 for cosmopolitan/redbean 2021-09-27 21:42:40 -07:00
Paul Kulchenko
b51db9f923 Add lua-argon2 module for redbean/lua 2021-09-27 21:42:40 -07:00
Paul Kulchenko
6fba9601ff Update Argon2 for cosmopolitan libc 2021-09-27 21:42:40 -07:00
Paul Kulchenko
9e7773a076 Add Argon2 implementation 2021-09-27 21:42:40 -07:00
jacereda
48a2668239
Fix bad dependency (#276) 2021-09-27 20:39:38 -07:00
Paul Kulchenko
5ffeaa8e33
Fix SSE scalar ops (#271) 2021-09-17 16:49:53 -07:00
Florian Lemaitre
18ccfeb919
Embed APE loader inside APE (#267)
The `ape-no-modify-self.o` bootloader now has an mmap-based
payload that helps read-only APE binaries be load faster since it
doesn't need to copy any files.
2021-09-17 16:49:10 -07:00
Justine Tunney
b5f743cdc3 Begin incorporating Python unit tests into build
We now build a separate APE binary for each test so they can run in
parallel. We've got 148 tests running fast and stable so far.
2021-09-12 21:04:44 -07:00
Justine Tunney
51904e2687 Improve Python and Linenoise
This change reinvents all the GNU Readline features I discovered that I
couldn't live without, e.g. UTF-8, CTRL-R search and CTRL-Y yanking. It
now feels just as good in terms of user interface from the subconscious
workflow perspective. It's real nice to finally have an embeddable line
reader that's actually good with a 30 kb footprint and a bsd-2 license.

This change adds a directory to the examples folder, explaining how the
new Python compiler may be used.  Some of the bugs with Python binaries
have been addressed but overall it's still a work in progress.
2021-09-11 22:30:37 -07:00
Justine Tunney
ad52387b74 Fix race condition in quickjs build config (#251)
qjsc.com now has a -n do nothing flag so the makefile can create a
localized binary. See also #267 where we have an exciting new change
aiming to address this particular APE gotcha.
2021-09-11 22:16:17 -07:00
agreppin
6ad0602392
quickjs: add repl + qjscalc (#251)
- Compile repl.js & qjscalc.js and include them in qjs.com zip
- Change quickjs-libc.c / js_os_poll to handle Windows console

This last change is needed because poll and select on Windows forbid
non-socket handles and Cosmopolitan Libc hasn't polyfilled that yet.
2021-09-11 16:52:49 -07:00
Paul Kulchenko
1b93066883
Add GetCookie to redbean Lua (#269) 2021-09-11 16:46:21 -07:00
Justine Tunney
559b024e1d Decentralize Python native module linkage
We can now link even smaller Python binaries. For example, the hello.com
program in the Python build directory is a compiled linked executable of
hello.py which just prints hello world. Using decentralized sections, we
can make that binary 1.9mb in size (noting that python.com is 6.3 megs!)

This works for nontrivial programs too. For example, say we want an APE
binary that's equivalent to python.com -m http.server. Our makefile now
builds such a binary using the new launcher and it's only 3.2mb in size
since Python sources get turned into ELF objects, which tell our linker
that we need things like native hashing algorithm code.
2021-09-07 11:40:11 -07:00
Justine Tunney
dfa0359b50 Exclude .py files in MODE=rel / tiny 2021-09-06 19:34:57 -07:00
Justine Tunney
4f41f2184d Improve Python tree-shaking 2021-09-06 19:24:10 -07:00
Justine Tunney
5bb2275788 Get bzip2 to build 2021-09-06 17:36:47 -07:00
Justine Tunney
a8fb68af4b Add bzip2 1.0.8 2021-09-06 15:51:57 -07:00
Paul Kulchenko
1eed7d47bd
Add error reporting to redbean Slurp (#268)
This should allow `content = assert(Slurp(filename))` to work as
expected and report an error if file doesn't exist or can't be read.
2021-09-06 08:21:37 -07:00
Justine Tunney
44c87b83ff Implement tree-shaking for Python sources 2021-09-05 01:20:03 -07:00
Justine Tunney
81287b7ec0 Introduce Python objectifier (#259) 2021-09-04 15:44:00 -07:00
Justine Tunney
34b68f1945 Make mappings unlimited on NT
This change might also fix fork() in certain cases on NT.
2021-09-04 13:20:47 -07:00
Justine Tunney
ab64c746cc Fix issues with InfoZip build
Artifacts weren't being invalidated because THIRD_PARTY_ZIP_SRCS_C
needed to be renamed THIRD_PARTY_ZIP_SRCS. This caused a build break.
2021-09-04 06:50:57 -07:00
Justine Tunney
a81192e0b9 Fix some build breaks 2021-09-04 02:29:57 -07:00
Gautham
27f7ffd4fd
Add speedups from pyston (#264)
This should make Python go 30% faster. It does that by trading
away some debuggability, like _tracemalloc. It can be re-enabled
using `make MODE=dbg`.
2021-09-04 02:21:37 -07:00
Paul Kulchenko
31dd714081
Add SetCookie method to redbean Lua (#265) 2021-09-04 02:12:12 -07:00
Justine Tunney
969174e155 Add APE interpreter example (#263) 2021-09-04 00:58:21 -07:00
Justine Tunney
5b60e5a37d Fix termios struct on Linux
The termios::c_cc field turned out to be incorrectly defined on Linux
due to some confusion between the glibc and kernel definitions. We'll
be using the kernel definition, since it has the strongest consensus.

Fields have been have been added to struct stat for BSD compatibility
such as st_birthtim, plus the GLIBC compatibility of isystem/sys/stat
has been improved.
2021-09-03 22:19:41 -07:00
Paul Kulchenko
0584684a82
Improve logger API (#262)
This breaking change improves naming consistency.

- Rename LOGF to INFOF
- Rename recently introduced ANYF to LOGF
- Remove V* log calls, as they are not being used
2021-09-03 21:14:26 -07:00
Justine Tunney
50937be752 Fix select() on Windows for timeout (#141) 2021-08-26 15:59:55 -07:00
Justine Tunney
3085ac7837 Improve system call support 2021-08-25 21:36:17 -07:00
Gautham
63b867bd2f
Added _multiprocessing to Python (#259)
Also changed some PYTHON_YOINKs so that http.server would work in
MODE=tiny.
2021-08-25 19:45:59 -07:00
Paul Kulchenko
0b7737b20a
Fix redbean StoreAsset w/ Deflate (#258) 2021-08-25 05:43:26 -07:00
Paul Kulchenko
818926dbae
Fix redbean latency and resource usage logging (#257) 2021-08-22 20:16:10 -07:00
Justine Tunney
7d25fb0090 Import some Lua documentation
I personally find it easier to read the documentation in Emacs
using JavaDoc style comments.
2021-08-22 15:03:04 -07:00
Paul Kulchenko
41b9eb6873
Redbean log improvements (#256)
* Update log level for redbean messages for better log content
* Add categories to redbean log messages
* Reorganize Lua error logging for consistency
* Replace perror logging with FATALF
* Update server failure reporting to better identify response code
* Replace `flogf` with DIEF logging for consistency
2021-08-22 15:01:52 -07:00
Justine Tunney
00611e9b06 Improve ZIP filesystem and change its prefix
The ZIP filesystem has a breaking change. You now need to use /zip/ to
open() / opendir() / etc. assets within the ZIP structure of your APE
binary, instead of the previous convention of using zip: or zip! URIs.
This is needed because Python likes to use absolute paths, and having
ZIP paths encoded like URIs simply broke too many things.

Many more system calls have been updated to be able to operate on ZIP
files and file descriptors. In particular fcntl() and ioctl() since
Python would do things like ask if a ZIP file is a terminal and get
confused when the old implementation mistakenly said yes, because the
fastest way to guarantee native file descriptors is to dup(2). This
change also improves the async signal safety of zipos and ensures it
doesn't maintain any open file descriptors beyond that which the user
has opened.

This change makes a lot of progress towards adding magic numbers that
are specific to platforms other than Linux. The philosophy here is that,
if you use an operating system like FreeBSD, then you should be able to
take advantage of FreeBSD exclusive features, even if we don't polyfill
them on other platforms. For example, you can now open() a file with the
O_VERIFY flag. If your program runs on other platforms, then Cosmo will
automatically set O_VERIFY to zero. This lets you safely use it without
the need for #ifdef or ifstatements which detract from readability.

One of the blindspots of the ASAN memory hardening we use to offer Rust
like assurances has always been that memory passed to the kernel via
system calls (e.g. writev) can't be checked automatically since the
kernel wasn't built with MODE=asan. This change makes more progress
ensuring that each system call will verify the soundness of memory
before it's passed to the kernel. The code for doing these checks is
fast, particularly for buffers, where it can verify 64 bytes a cycle.

- Correct O_LOOP definition on NT
- Introduce program_executable_name
- Add ASAN guards to more system calls
- Improve termios compatibility with BSDs
- Fix bug in Windows auxiliary value encoding
- Add BSD and XNU specific errnos and open flags
- Add check to ensure build doesn't talk to internet
2021-08-22 01:11:53 -07:00
Paul Kulchenko
2730c66f4a
Add GetTime to redbean Lua (#255) 2021-08-22 00:59:47 -07:00
Paul Kulchenko
c506584174
Don't hyphenate delta log lines (#253) 2021-08-19 09:43:01 -07:00
Justine Tunney
ed485391da Reduce memory needed by ASAN morgue 2021-08-19 09:34:24 -07:00
Justine Tunney
d5a37de435 Fix upstream Lua test breakage due to Linenoise 2021-08-19 09:34:24 -07:00
Justine Tunney
65f32fad52 Fix regression in _gclongjmp() 2021-08-19 09:34:24 -07:00
Justine Tunney
da45c7c80b Clean up gdtoa code 2021-08-19 09:34:24 -07:00
Paul Kulchenko
7341336b1a
Improve redbean docs (#249)
- Add Location to ProgramHeader exclusions
- Add ProgramHeader documentation
- Update ProgramRedirect to check location validity
2021-08-19 09:34:50 -07:00
Paul Kulchenko
a63b147a93
Add file/line reporting to redbean LuaLog (#250) 2021-08-19 09:32:33 -07:00
Paul Kulchenko
0ba3617e7f
Add ERRORF macro for kLogError log level (#252) 2021-08-19 04:14:08 -07:00