Commit graph

332 commits

Author SHA1 Message Date
Joshua Wierenga
fea68b142e Fully support old windows 8ib user space virtual memory
Addresses are converted in blocks(asan, automap, stack, ...), with the start and end addresses modified using (address & 0xffffff) | ((address >> 28) << 24).

Following addresses are only specified to frame accuracy with the last 4 hex digits chopped off.
null, loader, image and arena blocks have not been modified.
asan     [0x00007fff, 0x10007fff] ⇒ [0x00007fff, 0x01007fff] Not currently supported.
automap  [0x10008004, 0x1fe7fffb] ⇒ [0x01008004, 0x01e7fffb]
_mmi     [0x1fe7fffc, 0x1ffffffb] ⇒ [0x01e7fffc, 0x01fffffb] Listed as memtrack outside of the memtrack files.
fixedmap [0x30000004, 0x40000003] ⇒ [0x03000004, 0x04000003]
nsync    [0x6fc00004, 0x6fcffffb] ⇒ [0x06c00004, 0x06cffffb]
zipos    [0x6fd00004, 0x6fdffffb] ⇒ [0x06d00004, 0x06dffffb] I don't think this is used currently on the vista branch.
g_fds    [0x6fe00004, 0x6feffffb] ⇒ [0x06e00004, 0x06effffb]
winargs  [0x6ffffffe, 0x6fffffff] ⇒ [0x06fffffe, 0x06ffffff] Not entirely confident on if this is accurate.
stack    [0x70000000, 0x70000003] ⇒ [0x07000000, 0x07000003] Not entirely confident on if this is accurate.

This map is documented in memtrack64-8tib.txt with the original 128tib version renamed to memtrack64-128tib.txt.

Modified windows 8.1 check macro name to match existing windows 10 one.
Modified IsShadowFrame to correctly identify the asan block on old windows. Required as the asan block on 128tib virtual memory systems is larger than 2^43 - 1 and so all blocks were reporting as asan.
2023-03-20 21:07:01 -04:00
Joshua Wierenga
8f001857de Fix build breaks
Revert the changes that 827f25f made to libc/calls/readv-metal.c regarding internal functions having underlines due to 6f7d0cb being missing at the time.
Use the versions of libc/intrin/lockfileex.c and libc/intrin/unlockfileex.c from 3f49889 to account for strace changes.
Revert d86916e and use the versions of libc/calls/getgroups.c, libc/calls/setgroups.c, libc/runtime/getlogin.c and libc/runtime/getlogin_r.c from 4c40c50 to again account for strace changes.
Fix include statements and internal underlined function calls in libc/stdio/cocmd.c, libc/testlib/extract.c and test/libc/stdio/system_test.c.
Fix missing closing paren in nsync memory size macro in memtrack.internal.h.
2023-03-20 21:07:01 -04:00
Justine Tunney
7b26b42769 Pay off more technical debt
This makes breaking changes to add underscores to many non-standard
function names provided by the c library. MODE=tiny is now tinier and
we now use smaller locks that are better for tiny apps in this mode.
Some headers have been renamed to be in the same folder as the build
package, so it'll be easier to know which build dependency is needed.
Certain old misguided interfaces have been removed. Intel intrinsics
headers are now listed in libc/isystem (but not in the amalgamation)
to help further improve open source compatibility. Header complexity
has also been reduced. Lastly, more shell scripts are now available.

Compared to 6f7d0cb1c3, some tiny corrections were made in libc/intrin/g_fds.c and libc/zipos/open.c including double semi colons and incorrect indentation for existing vista changes that were manually pulled from this commit previously.
2023-03-20 21:07:01 -04:00
Justine Tunney
555260d2e5 Optimize memory layout
Compared to b69f3d2488, old windows specific fd, zipos and nsync memory ranges in libc/runtime/memtrack.internal.h were kept.
2023-03-20 21:07:01 -04:00
Gavin Hayes
0740e68ea0
Add imaxdiv, wcscoll, getdtablesize (#639) (#721)
Co-authored-by: Hugues Morisset <morisset.hugues@gmail.com>
2023-01-29 19:33:00 -08:00
Gavin Hayes
9c5a7795ad
vista: backport execve escaping and using cocmd as shell for system, etc. (#660)
* Introduce testlib_extract() helper

* Have execve() escape double quotes in cmd.exe's preferred style

This makes it possible for us to use system() and popen() with paths
that redirect to filenames that contain spaces, e.g.

    system("echo.com hello >\"hello there.txt\"")

It's difficult to solve this problem, because WIN32 only allows passing
one single argument when launching programs and each program is allowed
to tokenize that however it wants. Most software follows the convention
of cmd.exe which is poorly documented and positively byzantine.

In the future we're going to solve this by not using cmd.exe at all and
instead embedding the cocmd.com interpreter into the system() function.
In the meantime, our documentation has been updated to help recalibrate
any expectation the user might hold regarding the security of using the
Windows command interpreter.

Fixes #644

* Introduce double quote support in cocmd.com shell

* Add some tests for execve()

* Embed cocmd.com interpreter for system() / open()

This change lets you use system() in an easier and portable way. The
problem with the call in the past has always been that bourne and
cmd.com on Windows have less than nothing in common, so pretty much the
only command system() could be used for across platforms was maybe echo.
cmd.exe is also a security liability due to its escaping rules.

Since cocmd.com implements 85% of what we need from bourne, in a really
tiny way, it makes perfect sense to be embedded in these functionss. We
get a huge performance boost too.

Fixes #644

* Support whitespace after cocmd output redirection

Co-authored-by: Justine Tunney <jtunney@gmail.com>
2022-10-11 21:17:50 -07:00
Gavin Hayes
f4ff1729d1
Backport main branch improvements (#658)
* Add sys_ prefix to unwrapped system calls
* This change also implements getlogin() and getlogin_r().
* Add getgroups and setgroups (#619)
* Fix getgroups / setgroups tests across platforms. See #619
* Change accept type to struct sockaddr * (#630)
* vista: use old strace path
2022-10-10 20:44:29 -07:00
Joshua Wierenga
4381b3d925
Get cosmopolitation working on Vista again (#635)
* Fixes virtual memory support for Windows Vista/7/8

Fixes file desciptor and zipos virtual memory sizes given that old versions of windows limit their user virtual memory space to 8tb.
Includes some changes from b69f3d2488 and 6f7d0cb1c3 in preparation for potentially using them on old windows and because it avoids duplicating addresses everywhere.

* Optimise windows version checking

Added GetNtMinorVersion() and IsAtleastWindows8p1() macros which get nt version info from the peb.

* Planned nsync memory region for old windows

Added nsync regon start and size macros that can switch to smaller values on old windows, just like previous fds and zipos change.
2022-10-04 01:20:39 -07:00
Justine Tunney
c64e7eaf7c
Fix F_DUPFD_CLOEXEC on some systems 2022-09-15 12:48:26 -07:00
Justine Tunney
a5fa90a21f
Fix POSIX advisory locks on Windows 2022-09-15 12:44:00 -07:00
Justine Tunney
050062bcbb
Fix preadv() and pwritev() 2022-09-15 12:30:36 -07:00
Justine Tunney
827f25f054
Fix bug with sendfile() on XNU 2022-09-15 12:27:30 -07:00
Justine Tunney
790c661317
Fix the build 2022-09-15 12:02:44 -07:00
tkchia
c4eb701d56
Bare metal VGA: implement "status report" escape codes (#613)
* Bare metal VGA: implement "status report" escape codes
* Minor fix to pseudoteletypewriter code

Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-09-15 12:02:35 -07:00
Justine Tunney
b5cb71ab84
Use *NSYNC for POSIX threads locking APIs
Condition variables, barriers, and r/w locks now work very well.
2022-09-11 11:04:50 -07:00
Justine Tunney
3de35e196c
Fix the build 2022-09-10 19:57:50 -07:00
Justine Tunney
5dff63a31c
Import nsync 2022-09-10 16:12:31 -07:00
Justine Tunney
cdb2284f0d
Remove stdio lock macros from amalgamation 2022-09-10 12:03:36 -07:00
Justine Tunney
155b378a39
Tidy up the threading implementation
The organization of the source files is now much more rational.
Old experiments that didn't work out are now deleted. Naming of
things like files is now more intuitive.
2022-09-10 02:56:25 -07:00
Justine Tunney
3265324e00
Don't relocate file descriptor memory
This change fixes #496 where ASAN spotted a race condition that could
happen in multithreaded programs, with more than OPEN_MAX descriptors
when using ZipOS or Windows NT, which require tracking open file info
and this change fixes that table so it never relocates, thus allowing
us to continue to enjoy the benefits of avoiding locks while reading.
2022-09-09 16:54:28 -07:00
Justine Tunney
c3208eb9d5
Add more POSIX threads APIs 2022-09-09 11:30:33 -07:00
Gautham
094ccbb4bd
Add more foo64 stubs for Rust (#607) 2022-09-09 06:50:45 -07:00
Justine Tunney
2d17ab016c
Perform more low-level code cleanup 2022-09-09 04:07:08 -07:00
Justine Tunney
9f963dc597
Clean up some of the threading code 2022-09-08 12:31:56 -07:00
Justine Tunney
0547eabcd6
Polyfill IPv6 on non-Linux 2022-09-08 06:06:22 -07:00
Justine Tunney
114176c304
Add more foo64() functions for Rust 2022-09-07 05:48:57 -07:00
Gavin Hayes
a849a63771
Implement sigpending for sysv and nt (#597) 2022-09-07 05:38:12 -07:00
Justine Tunney
4339d9f15e
Add pthread attributes and other libc functions 2022-09-07 05:28:32 -07:00
tkchia
3fdb1c14f1
Add VGA support on bare metal (#588)
If your main module has this declaration:

    STATIC_YOINK("vga_console");

Then a VGA driver will be linked into your executable which
displays your stdio characters on the PC display, whereas
before we could only use the serial port. Your display is an
ANSI terminal and it's still a work in progress.
2022-09-06 18:41:08 -07:00
Gavin Hayes
598640864a
Fix inability to add some signals to mask on NT (#596) 2022-09-06 18:35:26 -07:00
Justine Tunney
d861d2787b
Import OpenBSD sorting algorithms
OpenBSD's qsort() function is more secure than the ones used by
FreeBSD, NetBSD and MacOS. The best part is it goes faster too!
This change also imports the OpenBSD mergesort() and heapsort()
2022-09-06 17:52:26 -07:00
Justine Tunney
55c6297e13
Make more compatibility improvements 2022-09-06 07:04:13 -07:00
Justine Tunney
12d9e1e128
Improve quality of our ANSI C clock() function
It now works most excellently across all supported operating
sytsems (earlier it didn't work on NT and XNU). Demo code is
available in examples/clock.c and this change also adds some
of the newer ANSI C time functions like timespec_get(), plus
timespec_getres() which hasn't even come out yet as it's C23
2022-09-05 23:03:49 -07:00
Justine Tunney
7ff0ea8c13
Make pthread mutexes more scalable
pthread_mutex_lock() now uses a better algorithm which goes much faster
in multithreaded environments that have lock contention. This comes at
the cost of adding some fixed-cost overhead to mutex invocations. That
doesn't matter for Cosmopolitan because our core libraries all encode
locking operations as NOP instructions when in single-threaded mode.
Overhead only applies starting the moment you first call clone().
2022-09-05 15:57:51 -07:00
Gavin Hayes
7de2f229a7
Discard ignored signals on New Technology (#592) 2022-09-05 09:17:56 -07:00
Justine Tunney
8cd4248f7f
Make SIG_DFL / SIG_IGN C++ compatible 2022-09-05 08:49:26 -07:00
Justine Tunney
6a04bc3318
Release redbean 2.0.18 2022-09-04 06:57:59 -07:00
Justine Tunney
0c70e8963d Add notpossible keyword
This is the same as `unreachable` except it always traps violations,
even if we're not running in MODE=dbg. This is useful for impossible
conditions relating to system calls. It avoids terrifying bugs where
control falls through to an unrelated function.
2022-09-03 20:35:31 -07:00
Justine Tunney
b66bd064d8 Improve quality of uname/gethostname/getdomainname 2022-09-03 20:20:40 -07:00
Justine Tunney
c5c4dfcd21 Improve quality of raise(), abort(), and tkill()
This change fixes a nasty bug where SIG_IGN and SIG_DFL weren't working
as advertised on BSDs. This change also fixes the tkill() definition on
MacOS so it maps to __pthread_kill().
2022-09-03 20:17:54 -07:00
Gavin Hayes
263711965f
Change sigaction_f to match sysv signature (#585) 2022-09-02 05:08:35 -07:00
Justine Tunney
5a632cf72d Add lchown, lchmod, statvfs, fstatvfs 2022-08-22 20:50:16 -07:00
Justine Tunney
57b39d8793 Revert recent fixupobj.com change 2022-08-21 17:33:54 -07:00
Justine Tunney
bf757c4646 Fix _Thread_local with fork() on Windows 2022-08-21 15:51:44 -07:00
Justine Tunney
73845be1f0 Restore zip.com and .symtab files
This change restores the .symtab symbol table files in our flagship
programs (e.g. redbean.com, python.com) needed to show backtraces. This
also rolls back earlier changes to zip.com w.r.t. temp directories since
the right way to do it turned out to be the -b DIR flag.

This change also improves the performance of zip.com. It turned out
mmap() wasn't being used, because zip.com was assuming a 4096-byte
granularity, but cosmo requires 65536. There was also a chance to speed
up stdio scanning using the unlocked functions.
2022-08-21 00:17:20 -07:00
Justine Tunney
83d41e4588 Clean up some code 2022-08-20 12:32:51 -07:00
Daniil Kulchenko
2827df688a
Fix M1 Rosetta signal handling (#455) (#553)
Rosetta does something strange to the signal handling registers but
setting SA_SIGINFO prevents the issue from happening. Set the flag
on XNU to work around the issue.
2022-08-19 13:00:50 -07:00
Justine Tunney
75832f7379 Polyfill sysinfo() a ilttle bit on BSDs 2022-08-18 15:53:11 -07:00
Justine Tunney
30e1c5bca9 Add statfs() and fstatfs() to redbean unix module 2022-08-17 23:27:17 -07:00
Justine Tunney
c2211c9e63 Polyfill statfs() and fstatfs() on Windows 2022-08-17 19:01:51 -07:00