Addresses are converted in blocks(asan, automap, stack, ...), with the start and end addresses modified using (address & 0xffffff) | ((address >> 28) << 24).
Following addresses are only specified to frame accuracy with the last 4 hex digits chopped off.
null, loader, image and arena blocks have not been modified.
asan [0x00007fff, 0x10007fff] ⇒ [0x00007fff, 0x01007fff] Not currently supported.
automap [0x10008004, 0x1fe7fffb] ⇒ [0x01008004, 0x01e7fffb]
_mmi [0x1fe7fffc, 0x1ffffffb] ⇒ [0x01e7fffc, 0x01fffffb] Listed as memtrack outside of the memtrack files.
fixedmap [0x30000004, 0x40000003] ⇒ [0x03000004, 0x04000003]
nsync [0x6fc00004, 0x6fcffffb] ⇒ [0x06c00004, 0x06cffffb]
zipos [0x6fd00004, 0x6fdffffb] ⇒ [0x06d00004, 0x06dffffb] I don't think this is used currently on the vista branch.
g_fds [0x6fe00004, 0x6feffffb] ⇒ [0x06e00004, 0x06effffb]
winargs [0x6ffffffe, 0x6fffffff] ⇒ [0x06fffffe, 0x06ffffff] Not entirely confident on if this is accurate.
stack [0x70000000, 0x70000003] ⇒ [0x07000000, 0x07000003] Not entirely confident on if this is accurate.
This map is documented in memtrack64-8tib.txt with the original 128tib version renamed to memtrack64-128tib.txt.
Modified windows 8.1 check macro name to match existing windows 10 one.
Modified IsShadowFrame to correctly identify the asan block on old windows. Required as the asan block on 128tib virtual memory systems is larger than 2^43 - 1 and so all blocks were reporting as asan.
Revert the changes that 827f25f made to libc/calls/readv-metal.c regarding internal functions having underlines due to 6f7d0cb being missing at the time.
Use the versions of libc/intrin/lockfileex.c and libc/intrin/unlockfileex.c from 3f49889 to account for strace changes.
Revert d86916e and use the versions of libc/calls/getgroups.c, libc/calls/setgroups.c, libc/runtime/getlogin.c and libc/runtime/getlogin_r.c from 4c40c50 to again account for strace changes.
Fix include statements and internal underlined function calls in libc/stdio/cocmd.c, libc/testlib/extract.c and test/libc/stdio/system_test.c.
Fix missing closing paren in nsync memory size macro in memtrack.internal.h.
This makes breaking changes to add underscores to many non-standard
function names provided by the c library. MODE=tiny is now tinier and
we now use smaller locks that are better for tiny apps in this mode.
Some headers have been renamed to be in the same folder as the build
package, so it'll be easier to know which build dependency is needed.
Certain old misguided interfaces have been removed. Intel intrinsics
headers are now listed in libc/isystem (but not in the amalgamation)
to help further improve open source compatibility. Header complexity
has also been reduced. Lastly, more shell scripts are now available.
Compared to 6f7d0cb1c3, some tiny corrections were made in libc/intrin/g_fds.c and libc/zipos/open.c including double semi colons and incorrect indentation for existing vista changes that were manually pulled from this commit previously.
* Introduce testlib_extract() helper
* Have execve() escape double quotes in cmd.exe's preferred style
This makes it possible for us to use system() and popen() with paths
that redirect to filenames that contain spaces, e.g.
system("echo.com hello >\"hello there.txt\"")
It's difficult to solve this problem, because WIN32 only allows passing
one single argument when launching programs and each program is allowed
to tokenize that however it wants. Most software follows the convention
of cmd.exe which is poorly documented and positively byzantine.
In the future we're going to solve this by not using cmd.exe at all and
instead embedding the cocmd.com interpreter into the system() function.
In the meantime, our documentation has been updated to help recalibrate
any expectation the user might hold regarding the security of using the
Windows command interpreter.
Fixes#644
* Introduce double quote support in cocmd.com shell
* Add some tests for execve()
* Embed cocmd.com interpreter for system() / open()
This change lets you use system() in an easier and portable way. The
problem with the call in the past has always been that bourne and
cmd.com on Windows have less than nothing in common, so pretty much the
only command system() could be used for across platforms was maybe echo.
cmd.exe is also a security liability due to its escaping rules.
Since cocmd.com implements 85% of what we need from bourne, in a really
tiny way, it makes perfect sense to be embedded in these functionss. We
get a huge performance boost too.
Fixes#644
* Support whitespace after cocmd output redirection
Co-authored-by: Justine Tunney <jtunney@gmail.com>
* Add sys_ prefix to unwrapped system calls
* This change also implements getlogin() and getlogin_r().
* Add getgroups and setgroups (#619)
* Fix getgroups / setgroups tests across platforms. See #619
* Change accept type to struct sockaddr * (#630)
* vista: use old strace path
* Fixes virtual memory support for Windows Vista/7/8
Fixes file desciptor and zipos virtual memory sizes given that old versions of windows limit their user virtual memory space to 8tb.
Includes some changes from b69f3d2488 and 6f7d0cb1c3 in preparation for potentially using them on old windows and because it avoids duplicating addresses everywhere.
* Optimise windows version checking
Added GetNtMinorVersion() and IsAtleastWindows8p1() macros which get nt version info from the peb.
* Planned nsync memory region for old windows
Added nsync regon start and size macros that can switch to smaller values on old windows, just like previous fds and zipos change.
You can now run bare metal on bare metal!
* Fix handling of int 0x15 eax = 0xe820 memory map
* Fix some issues in initial page table creation
* hello4.com now works outside emulators
* Ensure area for identity page tables are zeroed first
* Simplify logic for creating page table entries, this partly
reverts 577c0f6226
* Add degenerate MBR partition entry, to ease testing
Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
The organization of the source files is now much more rational.
Old experiments that didn't work out are now deleted. Naming of
things like files is now more intuitive.
This change fixes#496 where ASAN spotted a race condition that could
happen in multithreaded programs, with more than OPEN_MAX descriptors
when using ZipOS or Windows NT, which require tracking open file info
and this change fixes that table so it never relocates, thus allowing
us to continue to enjoy the benefits of avoiding locks while reading.
- You can now use _gc(malloc()) in multithreaded programs
- This change fixes a bug where fork() on NT disabled TLS
- Fixed TLS code morphing on XNU/NT, for R8-R15 registers
This is the same as `unreachable` except it always traps violations,
even if we're not running in MODE=dbg. This is useful for impossible
conditions relating to system calls. It avoids terrifying bugs where
control falls through to an unrelated function.
This change fixes a nasty bug where SIG_IGN and SIG_DFL weren't working
as advertised on BSDs. This change also fixes the tkill() definition on
MacOS so it maps to __pthread_kill().
- Polyfill pselect() on Windows
- Add -O NOFILE flag to pledge.com
- Polyfill ppoll() on NetBSD, XNU, and Windows
- Support negative numbers and errno in sizetol()
- Add .RSS, .NOFILE, and .MAXCORE to Landlock Make
- Fix issue with .PLEDGE preventing touching of output files
- Add __watch() function (like ftrace) for logging memory changes
It turned out that specifying all SRCS and INCS as dependencies on the
pattern rules for all headers, caused `make` memory usage to skyrocket
from 40mb ot 160mb. This change also reduces the build graph another 4%.
* Fix deterministic startup stack setup, especially for bare metal
* Implement __enable_tls() on bare metal
* Get __get_tls_privileged() working on bare metal
- 10.5% reduction of o//depend dependency graph
- 8.8% reduction in latency of make command
- Fix issue with temporary file cleanup
There's a new -w option in compile.com that turns off the recent
Landlock output path workaround for "good commands" which do not
unlink() the output file like GNU tooling does.
Our new GNU Make unveil sandboxing appears to have zero overhead
in the grand scheme of things. Full builds are pretty fast since
the only thing that's actually slowed us down is probably libcxx
make -j16 MODE=rel
RL: took 85,732,063µs wall time
RL: ballooned to 323,612kb in size
RL: needed 828,560,521µs cpu (11% kernel)
RL: caused 39,080,670 page faults (99% memcpy)
RL: 350,073 context switches (72% consensual)
RL: performed 0 reads and 11,494,960 write i/o operations
pledge() and unveil() no longer consider ENOSYS to be an error.
These functions have also been added to Python's cosmo module.
This change also removes some WIN32 APIs and System Five magnums
which we're not using and it's doubtful anyone else would be too
- We now kill the program on violations like OpenBSD
- We now print a message explaining which promise is needed
- This change also fixes a linkage bug with thread local storage
- Your sigaction() handlers should now be more thread safe
A new `__pledge_mode` global has been introduced to make pledge() more
customizable on Linux. For example:
__attribute__((__constructor__)) static void init(void) {
__pledge_mode = SECCOMP_RET_ERRNO | EPERM;
}
Can be used to restore our old permissive pledge() behavior.
The whole repository is now buildable with GNU Make Landlock sandboxing.
This proves that no Makefile targets exist which touch files other than
their declared prerequisites. In order to do this, we had to:
1. Stop code morphing GCC output in package.com and instead run a
newly introduced FIXUPOBJ.COM command after GCC invocations.
2. Disable all the crumby Python unit tests that do things like create
files in the current directory, or rename() files between folders.
This ended up being a lot of tests, but most of them are still ok.
3. Introduce an .UNSANDBOXED variable to GNU Make to disable Landlock.
We currently only do this for things like `make tags`.
4. This change deletes some GNU Make code that was preventing the
execve() optimization from working. This means it should no longer
be necessary in most cases for command invocations to be indirected
through the cocmd interpreter.
5. Missing dependencies had to be declared in certain places, in cases
where they couldn't be automatically determined by MKDEPS.COM
6. The libcxx header situation has finally been tamed. One of the
things that makes this difficult is MKDEPS.COM only wants to
consider the first 64kb of a file, in order to go fast. But libcxx
likes to have #include lines buried after huge documentation.
7. An .UNVEIL variable has been introduced to GNU Make just in case
we ever wish to explicitly specify additional things that need to
be whitelisted which aren't strictly prerequisites. This works in
a manner similar to the recently introduced .EXTRA_PREREQS feature.
There's now a new build/bootstrap/make.com prebuilt binary available. It
should no longer be possible to write invalid Makefile code.
This change also fixes a bug with gettid() being incorrect after fork().
We now implement the ENOENT behavior for getauxval(). The getuid() etc.
system calls are now faster too. Plus issetugid() will work on BSDs.
- Fix getpriority()
- Add AT_MINSIGSTKSZ
- Fix bugs in BPF code
- Show more stuff in printargs.com
- Write manual test for pledge.com
- pledge() now generates tinier BPF code
- Have pledge("exec") only enable execve()
- Fix pledge.com chroot setuid functionality
- Improve pledge.com unveiling of ape loader
