mirrors/cosmopolitan
1
0
Fork 0
mirror of https://github.com/jart/cosmopolitan.git synced 2025-02-07 06:53:33 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
cosmopolitan/third_party/dlmalloc/locks.inc

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

144 lines
3.6 KiB
PHP
Raw Normal View History

Eliminate cyclic locks in runtime This change introduces a new deadlock detector for Cosmo's POSIX threads implementation. Error check mutexes will now track a DAG of nested locks and report EDEADLK when a deadlock is theoretically possible. These will occur rarely, but it's important for production hardening your code. You don't even need to change your mutexes to use the POSIX error check mode because `cosmocc -mdbg` will enable error checking on mutexes by default globally. When cycles are found, an error message showing your demangled symbols describing the strongly connected component are printed and then the SIGTRAP is raised, which means you'll also get a backtrace if you're using ShowCrashReports() too. This new error checker is so low-level and so pure that it's able to verify the relationships of every libc runtime lock, including those locks upon which the mutex implementation depends.
2024-12-17 04:51:27 +00:00
#include "libc/cosmo.h"
#include "libc/intrin/kprintf.h"
#include "libc/intrin/maps.h"
#include "libc/thread/thread.h"
Make emacs not croak when editing dlmalloc
2022-06-10 00:45:30 +00:00
/* --------------------------- Lock preliminaries ------------------------ */
/*
When locks are defined, there is one global lock, plus
one per-mspace lock.
The global lock_ensures that mparams.magic and other unique
mparams values are initialized only once. It also protects
sequences of calls to MORECORE. In many cases sys_alloc requires
two calls, that should not be interleaved with calls by other
threads. This does not protect against direct calls to MORECORE
by other threads not using this lock, so there is still code to
cope the best we can on interference.
Per-mspace locks surround calls to malloc, free, etc.
By default, locks are simple non-reentrant mutexes.
Because lock-protected regions generally have bounded times, it is
OK to use the supplied simple spinlocks. Spinlocks are likely to
improve performance for lightly contended applications, but worsen
performance under heavy contention.
If USE_LOCKS is > 1, the definitions of lock routines here are
bypassed, in which case you will need to define the type MLOCK_T,
and at least INITIAL_LOCK, DESTROY_LOCK, ACQUIRE_LOCK, RELEASE_LOCK
and TRY_LOCK. You must also declare a
static MLOCK_T malloc_global_mutex = { initialization values };.
*/
Use good locks in dlmalloc Using mere spin locks causes runitd.com to go painstakingly slow on NetBSD for reasons that aren't clear yet.
2023-12-28 12:55:50 +00:00
#if USE_SPIN_LOCKS
Fix nsync_mu_unlock_slow_() on Apple Silicon We torture test dlmalloc() in test/libc/stdio/memory_test.c. That test was crashing on occasion on Apple M1 microprocessors when dlmalloc was using *NSYNC locks. It was relatively easy to spot the cause, which is this one particular compare and swap operation, which needed to change to use sequentially-consistent ordering rather than an acquire barrier
2023-11-13 18:57:02 +00:00
Improve threading and i/o routines - On Windows connect() can now be interrupted by a signal; connect() w/ O_NONBLOCK will now raise EINPROGRESS; and connect() with SO_SNDTIMEO will raise ETIMEDOUT after the interval has elapsed. - We now get the AcceptEx(), ConnectEx(), and TransmitFile() functions from the WIN32 API the officially blessed way, using WSAIoctl(). - Do nothing on Windows when fsync() is called on a directory handle. This was raising EACCES earlier becaues GENERIC_WRITE is required on the handle. It's possible to FlushFileBuffers() a directory handle if it's opened with write access but MSDN doesn't document what it does. If you have any idea, please let us know! - Prefer manual reset event objects for read() and write() on Windows. - Do some code cleanup on our dlmalloc customizations. - Fix errno type error in Windows blocking routines. - Make the futex polyfill simpler and faster.
2023-10-13 01:53:17 +00:00
#define MLOCK_T atomic_uint
Eliminate cyclic locks in runtime This change introduces a new deadlock detector for Cosmo's POSIX threads implementation. Error check mutexes will now track a DAG of nested locks and report EDEADLK when a deadlock is theoretically possible. These will occur rarely, but it's important for production hardening your code. You don't even need to change your mutexes to use the POSIX error check mode because `cosmocc -mdbg` will enable error checking on mutexes by default globally. When cycles are found, an error message showing your demangled symbols describing the strongly connected component are printed and then the SIGTRAP is raised, which means you'll also get a backtrace if you're using ShowCrashReports() too. This new error checker is so low-level and so pure that it's able to verify the relationships of every libc runtime lock, including those locks upon which the mutex implementation depends.
2024-12-17 04:51:27 +00:00
static int malloc_inlk(MLOCK_T *lk) {
atomic_store_explicit(lk, 0, memory_order_relaxed);
return 0;
}
Improve threading and i/o routines - On Windows connect() can now be interrupted by a signal; connect() w/ O_NONBLOCK will now raise EINPROGRESS; and connect() with SO_SNDTIMEO will raise ETIMEDOUT after the interval has elapsed. - We now get the AcceptEx(), ConnectEx(), and TransmitFile() functions from the WIN32 API the officially blessed way, using WSAIoctl(). - Do nothing on Windows when fsync() is called on a directory handle. This was raising EACCES earlier becaues GENERIC_WRITE is required on the handle. It's possible to FlushFileBuffers() a directory handle if it's opened with write access but MSDN doesn't document what it does. If you have any idea, please let us know! - Prefer manual reset event objects for read() and write() on Windows. - Do some code cleanup on our dlmalloc customizations. - Fix errno type error in Windows blocking routines. - Make the futex polyfill simpler and faster.
2023-10-13 01:53:17 +00:00
static int malloc_wipe(MLOCK_T *lk) {
Add string similarity function
2024-08-17 23:45:07 +00:00
atomic_store_explicit(lk, 0, memory_order_relaxed);
Improve threading and i/o routines - On Windows connect() can now be interrupted by a signal; connect() w/ O_NONBLOCK will now raise EINPROGRESS; and connect() with SO_SNDTIMEO will raise ETIMEDOUT after the interval has elapsed. - We now get the AcceptEx(), ConnectEx(), and TransmitFile() functions from the WIN32 API the officially blessed way, using WSAIoctl(). - Do nothing on Windows when fsync() is called on a directory handle. This was raising EACCES earlier becaues GENERIC_WRITE is required on the handle. It's possible to FlushFileBuffers() a directory handle if it's opened with write access but MSDN doesn't document what it does. If you have any idea, please let us know! - Prefer manual reset event objects for read() and write() on Windows. - Do some code cleanup on our dlmalloc customizations. - Fix errno type error in Windows blocking routines. - Make the futex polyfill simpler and faster.
2023-10-13 01:53:17 +00:00
return 0;
}
Eliminate cyclic locks in runtime This change introduces a new deadlock detector for Cosmo's POSIX threads implementation. Error check mutexes will now track a DAG of nested locks and report EDEADLK when a deadlock is theoretically possible. These will occur rarely, but it's important for production hardening your code. You don't even need to change your mutexes to use the POSIX error check mode because `cosmocc -mdbg` will enable error checking on mutexes by default globally. When cycles are found, an error message showing your demangled symbols describing the strongly connected component are printed and then the SIGTRAP is raised, which means you'll also get a backtrace if you're using ShowCrashReports() too. This new error checker is so low-level and so pure that it's able to verify the relationships of every libc runtime lock, including those locks upon which the mutex implementation depends.
2024-12-17 04:51:27 +00:00
static int malloc_kilk(MLOCK_T *lk) {
return 0;
}
Improve threading and i/o routines - On Windows connect() can now be interrupted by a signal; connect() w/ O_NONBLOCK will now raise EINPROGRESS; and connect() with SO_SNDTIMEO will raise ETIMEDOUT after the interval has elapsed. - We now get the AcceptEx(), ConnectEx(), and TransmitFile() functions from the WIN32 API the officially blessed way, using WSAIoctl(). - Do nothing on Windows when fsync() is called on a directory handle. This was raising EACCES earlier becaues GENERIC_WRITE is required on the handle. It's possible to FlushFileBuffers() a directory handle if it's opened with write access but MSDN doesn't document what it does. If you have any idea, please let us know! - Prefer manual reset event objects for read() and write() on Windows. - Do some code cleanup on our dlmalloc customizations. - Fix errno type error in Windows blocking routines. - Make the futex polyfill simpler and faster.
2023-10-13 01:53:17 +00:00
static int malloc_lock(MLOCK_T *lk) {
Fix package.sh build error
2024-07-26 13:57:24 +00:00
for (;;) {
if (!atomic_exchange_explicit(lk, 1, memory_order_acquire))
break;
for (;;)
if (!atomic_load_explicit(lk, memory_order_relaxed))
break;
Pay off more technical debt This makes breaking changes to add underscores to many non-standard function names provided by the c library. MODE=tiny is now tinier and we now use smaller locks that are better for tiny apps in this mode. Some headers have been renamed to be in the same folder as the build package, so it'll be easier to know which build dependency is needed. Certain old misguided interfaces have been removed. Intel intrinsics headers are now listed in libc/isystem (but not in the amalgamation) to help further improve open source compatibility. Header complexity has also been reduced. Lastly, more shell scripts are now available.
2022-09-13 06:10:38 +00:00
}
return 0;
}
Eliminate cyclic locks in runtime This change introduces a new deadlock detector for Cosmo's POSIX threads implementation. Error check mutexes will now track a DAG of nested locks and report EDEADLK when a deadlock is theoretically possible. These will occur rarely, but it's important for production hardening your code. You don't even need to change your mutexes to use the POSIX error check mode because `cosmocc -mdbg` will enable error checking on mutexes by default globally. When cycles are found, an error message showing your demangled symbols describing the strongly connected component are printed and then the SIGTRAP is raised, which means you'll also get a backtrace if you're using ShowCrashReports() too. This new error checker is so low-level and so pure that it's able to verify the relationships of every libc runtime lock, including those locks upon which the mutex implementation depends.
2024-12-17 04:51:27 +00:00
static int malloc_unlk(MLOCK_T *lk) {
Pay off more technical debt This makes breaking changes to add underscores to many non-standard function names provided by the c library. MODE=tiny is now tinier and we now use smaller locks that are better for tiny apps in this mode. Some headers have been renamed to be in the same folder as the build package, so it'll be easier to know which build dependency is needed. Certain old misguided interfaces have been removed. Intel intrinsics headers are now listed in libc/isystem (but not in the amalgamation) to help further improve open source compatibility. Header complexity has also been reduced. Lastly, more shell scripts are now available.
2022-09-13 06:10:38 +00:00
atomic_store_explicit(lk, 0, memory_order_release);
return 0;
}
Fix nsync_mu_unlock_slow_() on Apple Silicon We torture test dlmalloc() in test/libc/stdio/memory_test.c. That test was crashing on occasion on Apple M1 microprocessors when dlmalloc was using *NSYNC locks. It was relatively easy to spot the cause, which is this one particular compare and swap operation, which needed to change to use sequentially-consistent ordering rather than an acquire barrier
2023-11-13 18:57:02 +00:00
#else
Eliminate cyclic locks in runtime This change introduces a new deadlock detector for Cosmo's POSIX threads implementation. Error check mutexes will now track a DAG of nested locks and report EDEADLK when a deadlock is theoretically possible. These will occur rarely, but it's important for production hardening your code. You don't even need to change your mutexes to use the POSIX error check mode because `cosmocc -mdbg` will enable error checking on mutexes by default globally. When cycles are found, an error message showing your demangled symbols describing the strongly connected component are printed and then the SIGTRAP is raised, which means you'll also get a backtrace if you're using ShowCrashReports() too. This new error checker is so low-level and so pure that it's able to verify the relationships of every libc runtime lock, including those locks upon which the mutex implementation depends.
2024-12-17 04:51:27 +00:00
#define MLOCK_T struct MallocLock
Fix nsync_mu_unlock_slow_() on Apple Silicon We torture test dlmalloc() in test/libc/stdio/memory_test.c. That test was crashing on occasion on Apple M1 microprocessors when dlmalloc was using *NSYNC locks. It was relatively easy to spot the cause, which is this one particular compare and swap operation, which needed to change to use sequentially-consistent ordering rather than an acquire barrier
2023-11-13 18:57:02 +00:00
Eliminate cyclic locks in runtime This change introduces a new deadlock detector for Cosmo's POSIX threads implementation. Error check mutexes will now track a DAG of nested locks and report EDEADLK when a deadlock is theoretically possible. These will occur rarely, but it's important for production hardening your code. You don't even need to change your mutexes to use the POSIX error check mode because `cosmocc -mdbg` will enable error checking on mutexes by default globally. When cycles are found, an error message showing your demangled symbols describing the strongly connected component are printed and then the SIGTRAP is raised, which means you'll also get a backtrace if you're using ShowCrashReports() too. This new error checker is so low-level and so pure that it's able to verify the relationships of every libc runtime lock, including those locks upon which the mutex implementation depends.
2024-12-17 04:51:27 +00:00
struct MallocLock {
#if DEBUG
void *edges;
#endif
nsync_mu mu;
};
static int malloc_inlk(MLOCK_T *lk) {
Fix nsync_mu_unlock_slow_() on Apple Silicon We torture test dlmalloc() in test/libc/stdio/memory_test.c. That test was crashing on occasion on Apple M1 microprocessors when dlmalloc was using *NSYNC locks. It was relatively easy to spot the cause, which is this one particular compare and swap operation, which needed to change to use sequentially-consistent ordering rather than an acquire barrier
2023-11-13 18:57:02 +00:00
bzero(lk, sizeof(*lk));
return 0;
}
Eliminate cyclic locks in runtime This change introduces a new deadlock detector for Cosmo's POSIX threads implementation. Error check mutexes will now track a DAG of nested locks and report EDEADLK when a deadlock is theoretically possible. These will occur rarely, but it's important for production hardening your code. You don't even need to change your mutexes to use the POSIX error check mode because `cosmocc -mdbg` will enable error checking on mutexes by default globally. When cycles are found, an error message showing your demangled symbols describing the strongly connected component are printed and then the SIGTRAP is raised, which means you'll also get a backtrace if you're using ShowCrashReports() too. This new error checker is so low-level and so pure that it's able to verify the relationships of every libc runtime lock, including those locks upon which the mutex implementation depends.
2024-12-17 04:51:27 +00:00
static int malloc_wipe(MLOCK_T *lk) {
bzero(&lk->mu, sizeof(lk->mu));
return 0;
}
static int malloc_kilk(MLOCK_T *lk) {
return 0;
}
Fix nsync_mu_unlock_slow_() on Apple Silicon We torture test dlmalloc() in test/libc/stdio/memory_test.c. That test was crashing on occasion on Apple M1 microprocessors when dlmalloc was using *NSYNC locks. It was relatively easy to spot the cause, which is this one particular compare and swap operation, which needed to change to use sequentially-consistent ordering rather than an acquire barrier
2023-11-13 18:57:02 +00:00
static int malloc_lock(MLOCK_T *lk) {
Eliminate cyclic locks in runtime This change introduces a new deadlock detector for Cosmo's POSIX threads implementation. Error check mutexes will now track a DAG of nested locks and report EDEADLK when a deadlock is theoretically possible. These will occur rarely, but it's important for production hardening your code. You don't even need to change your mutexes to use the POSIX error check mode because `cosmocc -mdbg` will enable error checking on mutexes by default globally. When cycles are found, an error message showing your demangled symbols describing the strongly connected component are printed and then the SIGTRAP is raised, which means you'll also get a backtrace if you're using ShowCrashReports() too. This new error checker is so low-level and so pure that it's able to verify the relationships of every libc runtime lock, including those locks upon which the mutex implementation depends.
2024-12-17 04:51:27 +00:00
#if DEBUG
__deadlock_check(lk, 0);
#endif
nsync_mu_lock(&lk->mu);
#if DEBUG
__deadlock_record(lk, 0);
__deadlock_track(lk, 0);
#endif
Fix nsync_mu_unlock_slow_() on Apple Silicon We torture test dlmalloc() in test/libc/stdio/memory_test.c. That test was crashing on occasion on Apple M1 microprocessors when dlmalloc was using *NSYNC locks. It was relatively easy to spot the cause, which is this one particular compare and swap operation, which needed to change to use sequentially-consistent ordering rather than an acquire barrier
2023-11-13 18:57:02 +00:00
return 0;
}
Eliminate cyclic locks in runtime This change introduces a new deadlock detector for Cosmo's POSIX threads implementation. Error check mutexes will now track a DAG of nested locks and report EDEADLK when a deadlock is theoretically possible. These will occur rarely, but it's important for production hardening your code. You don't even need to change your mutexes to use the POSIX error check mode because `cosmocc -mdbg` will enable error checking on mutexes by default globally. When cycles are found, an error message showing your demangled symbols describing the strongly connected component are printed and then the SIGTRAP is raised, which means you'll also get a backtrace if you're using ShowCrashReports() too. This new error checker is so low-level and so pure that it's able to verify the relationships of every libc runtime lock, including those locks upon which the mutex implementation depends.
2024-12-17 04:51:27 +00:00
static int malloc_unlk(MLOCK_T *lk) {
#if DEBUG
if (__deadlock_tracked(lk) == 0) {
kprintf("error: unlock malloc mutex not owned by caller: %t\n", lk);
DebugBreak();
}
#endif
nsync_mu_unlock(&lk->mu);
#if DEBUG
__deadlock_untrack(lk);
#endif
Fix nsync_mu_unlock_slow_() on Apple Silicon We torture test dlmalloc() in test/libc/stdio/memory_test.c. That test was crashing on occasion on Apple M1 microprocessors when dlmalloc was using *NSYNC locks. It was relatively easy to spot the cause, which is this one particular compare and swap operation, which needed to change to use sequentially-consistent ordering rather than an acquire barrier
2023-11-13 18:57:02 +00:00
return 0;
}
#endif
Improve threading and i/o routines - On Windows connect() can now be interrupted by a signal; connect() w/ O_NONBLOCK will now raise EINPROGRESS; and connect() with SO_SNDTIMEO will raise ETIMEDOUT after the interval has elapsed. - We now get the AcceptEx(), ConnectEx(), and TransmitFile() functions from the WIN32 API the officially blessed way, using WSAIoctl(). - Do nothing on Windows when fsync() is called on a directory handle. This was raising EACCES earlier becaues GENERIC_WRITE is required on the handle. It's possible to FlushFileBuffers() a directory handle if it's opened with write access but MSDN doesn't document what it does. If you have any idea, please let us know! - Prefer manual reset event objects for read() and write() on Windows. - Do some code cleanup on our dlmalloc customizations. - Fix errno type error in Windows blocking routines. - Make the futex polyfill simpler and faster.
2023-10-13 01:53:17 +00:00
#define ACQUIRE_LOCK(lk) malloc_lock(lk)
Eliminate cyclic locks in runtime This change introduces a new deadlock detector for Cosmo's POSIX threads implementation. Error check mutexes will now track a DAG of nested locks and report EDEADLK when a deadlock is theoretically possible. These will occur rarely, but it's important for production hardening your code. You don't even need to change your mutexes to use the POSIX error check mode because `cosmocc -mdbg` will enable error checking on mutexes by default globally. When cycles are found, an error message showing your demangled symbols describing the strongly connected component are printed and then the SIGTRAP is raised, which means you'll also get a backtrace if you're using ShowCrashReports() too. This new error checker is so low-level and so pure that it's able to verify the relationships of every libc runtime lock, including those locks upon which the mutex implementation depends.
2024-12-17 04:51:27 +00:00
#define RELEASE_LOCK(lk) malloc_unlk(lk)
#define INITIAL_LOCK(lk) malloc_inlk(lk)
#define REFRESH_LOCK(lk) malloc_wipe(lk)
#define DESTROY_LOCK(lk) malloc_kilk(lk)
Make emacs not croak when editing dlmalloc
2022-06-10 00:45:30 +00:00
Use *NSYNC for POSIX threads locking APIs Condition variables, barriers, and r/w locks now work very well.
2022-09-11 18:02:07 +00:00
#define USE_LOCK_BIT (2U)
Make emacs not croak when editing dlmalloc
2022-06-10 00:45:30 +00:00
struct malloc_chunk {
size_t prev_foot; /* Size of previous chunk (if free). */
size_t head; /* Size and inuse bits. */
struct malloc_chunk* fd; /* double links -- used only if free. */
struct malloc_chunk* bk;
};
typedef struct malloc_chunk mchunk;
typedef struct malloc_chunk* mchunkptr;
typedef struct malloc_chunk* sbinptr; /* The type of bins of chunks */
typedef unsigned int bindex_t; /* Described below */
typedef unsigned int binmap_t; /* Described below */
typedef unsigned int flag_t; /* The type of various bit flag sets */
Reference in a new issue Copy permalink