linux-stable/lib
David Jeffery 106397376c sbitmap: fix batching wakeup
Current code supposes that it is enough to provide forward progress by
just waking up one wait queue after one completion batch is done.

Unfortunately this way isn't enough, cause waiter can be added to wait
queue just after it is woken up.

Follows one example(64 depth, wake_batch is 8)

1) all 64 tags are active

2) in each wait queue, there is only one single waiter

3) each time one completion batch(8 completions) wakes up just one
   waiter in each wait queue, then immediately one new sleeper is added
   to this wait queue

4) after 64 completions, 8 waiters are wakeup, and there are still 8
   waiters in each wait queue

5) after another 8 active tags are completed, only one waiter can be
   wakeup, and the other 7 can't be waken up anymore.

Turns out it isn't easy to fix this problem, so simply wakeup enough
waiters for single batch.

Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Chengming Zhou <zhouchengming@bytedance.com>
Cc: Jan Kara <jack@suse.cz>
Signed-off-by: David Jeffery <djeffery@redhat.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Reviewed-by: Gabriel Krisman Bertazi <krisman@suse.de>
Reviewed-by: Keith Busch <kbusch@kernel.org>
Link: https://lore.kernel.org/r/20230721095715.232728-1-ming.lei@redhat.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2023-07-21 11:40:20 -06:00
..
842
crypto This update includes the following changes: 2023-06-30 21:27:13 -07:00
dim linux/dim: Do nothing if no time delta between samples 2023-05-09 11:06:45 +02:00
fonts lib/fonts: fix undefined behavior in bit shift for get_default_font 2022-11-18 13:55:09 -08:00
kunit - Arnd Bergmann has fixed a bunch of -Wmissing-prototypes in 2023-06-28 10:59:38 -07:00
livepatch
lz4
lzo
math math64: favor kernel-doc from header files 2022-11-21 14:30:53 -07:00
mpi lib/mpi: Fix buffer overrun when SG is too long 2023-01-06 17:15:46 +08:00
pldmfw lib: remove MODULE_LICENSE in non-modules 2023-04-13 13:13:53 -07:00
raid6 raid6: neon: add missing prototypes 2023-06-13 15:13:20 -07:00
reed_solomon treewide: use get_random_u32_below() instead of deprecated function 2022-11-18 02:15:15 +01:00
test_fortify
vdso vdso: Improve cmd_vdso_check to check all dynamic relocations 2023-03-21 21:15:34 +01:00
xz
zlib_deflate lib/zlib: DFLTCC deflate does not write all available bits for Z_NO_FLUSH 2023-02-27 17:00:14 -08:00
zlib_dfltcc lib/zlib: remove redundation assignement of avail_in dfltcc_gdht() 2023-02-02 22:50:10 -08:00
zlib_inflate lib/zlib: Split deflate and inflate states for DFLTCC 2023-02-02 22:50:09 -08:00
zstd add intptr_t 2023-06-09 17:44:13 -07:00
.gitignore
Kconfig Kconfig: introduce HAS_IOPORT option and select it as necessary 2023-04-05 22:15:19 +02:00
Kconfig.debug - Arnd Bergmann has fixed a bunch of -Wmissing-prototypes in 2023-06-28 10:59:38 -07:00
Kconfig.kasan kasan: treat meminstrinsic as builtins in uninstrumented files 2023-03-02 21:54:22 -08:00
Kconfig.kcsan Kernel concurrency sanitizer (KCSAN) updates for v6.3 2023-02-25 13:02:20 -08:00
Kconfig.kfence
Kconfig.kgdb parisc: Convert PDC console to an early console 2022-10-11 12:01:24 +02:00
Kconfig.kmsan kmsan: make sure PREEMPT_RT is off 2022-11-08 15:57:24 -08:00
Kconfig.ubsan ubsan: remove cc-option test for UBSAN_TRAP 2023-05-17 12:01:54 -07:00
Makefile - Yosry Ahmed brought back some cgroup v1 stats in OOM logs. 2023-06-28 10:28:11 -07:00
argv_split.c
ashldi3.c
ashrdi3.c
asn1_decoder.c
asn1_encoder.c
assoc_array.c
atomic64.c
atomic64_test.c
audit.c
base64.c
bcd.c
bch.c
bitfield_kunit.c
bitmap.c lib/bitmap: remove bitmap_ord_to_pos 2022-09-26 12:19:12 -07:00
bitrev.c
bootconfig-data.S
bootconfig.c
bsearch.c
btree.c btree: remove MODULE_LICENSE in non-modules 2023-04-13 13:13:54 -07:00
bucket_locks.c
bug.c cpuidle: lib/bug: Disable rcu_is_watching() during WARN/BUG 2023-01-31 15:01:45 +01:00
build_OID_registry
buildid.c ELF: fix all "Elf" typos 2023-04-08 13:45:37 -07:00
bust_spinlocks.c kernel/panic: Drop unblank_screen call 2022-09-01 16:55:35 +02:00
check_signature.c
checksum.c
checksum_kunit.c x86/csum: Improve performance of `csum_partial` 2023-05-25 10:55:18 -07:00
clz_ctz.c
clz_tab.c
cmdline.c lib/cmdline: avoid page fault in next_arg 2022-09-11 21:55:06 -07:00
cmdline_kunit.c treewide: use get_random_{u8,u16}() when possible, part 1 2022-10-11 17:42:58 -06:00
cmpdi2.c
compat_audit.c
cpu_rmap.c lib: cpu_rmap: Fix potential use-after-free in irq_cpu_rmap_release() 2023-06-07 21:25:00 -07:00
cpumask.c lib/cpumask: update comment for cpumask_local_spread() 2023-02-07 18:20:00 -08:00
cpumask_kunit.c cpumask: re-introduce constant-sized cpumask optimizations 2023-03-05 14:30:34 -08:00
crc-ccitt.c
crc-itu-t.c
crc-t10dif.c
crc4.c
crc7.c
crc8.c
crc16.c
crc32.c
crc32defs.h
crc32test.c
crc64-rocksoft.c
crc64.c
ctype.c
debug_info.c
debug_locks.c
debugobjects.c debugobjects: Recheck debug_objects_enabled before reporting 2023-06-07 14:16:12 +02:00
dec_and_lock.c perf: Fix perf_event_pmu_context serialization 2023-01-31 20:37:18 +01:00
decompress.c
decompress_bunzip2.c
decompress_inflate.c decompressor: provide missing prototypes 2023-06-09 17:44:17 -07:00
decompress_unlz4.c
decompress_unlzma.c
decompress_unlzo.c
decompress_unxz.c decompressor: provide missing prototypes 2023-06-09 17:44:17 -07:00
decompress_unzstd.c decompressor: provide missing prototypes 2023-06-09 17:44:17 -07:00
devmem_is_allowed.c lib: devmem_is_allowed: include linux/io.h 2023-06-09 17:44:15 -07:00
devres.c devres: show which resource was invalid in __devm_ioremap_resource() 2023-06-23 17:04:04 -07:00
dhry.h lib: add Dhrystone benchmark test 2023-02-02 22:50:01 -08:00
dhry_1.c lib: add Dhrystone benchmark test 2023-02-02 22:50:01 -08:00
dhry_2.c lib: add Dhrystone benchmark test 2023-02-02 22:50:01 -08:00
dhry_run.c lib: dhry: fix unstable smp_processor_id(_) usage 2023-03-23 17:18:35 -07:00
digsig.c
dump_stack.c
dynamic_debug.c dyndbg: use the module notifier callbacks 2023-03-09 12:58:36 -08:00
dynamic_queue_limits.c
earlycpio.c lib: move from strlcpy with unused retval to strscpy 2022-09-11 21:55:10 -07:00
errname.c printf: fix errname.c list 2023-02-15 15:44:43 +01:00
error-inject.c error-injection: remove EI_ETYPE_NONE 2023-02-02 22:50:00 -08:00
errseq.c
extable.c
fault-inject-usercopy.c
fault-inject.c fault-inject: allow configuration via configfs 2023-04-13 07:38:54 -06:00
fdt.c
fdt_addresses.c
fdt_empty_tree.c
fdt_ro.c
fdt_rw.c
fdt_strerror.c
fdt_sw.c
fdt_wip.c
find_bit.c cpumask: introduce for_each_cpu_or 2023-03-19 10:02:04 -07:00
find_bit_benchmark.c treewide: use get_random_u32_below() instead of deprecated function 2022-11-18 02:15:15 +01:00
flex_proportions.c flex_proportions: Disable preemption entering the write section. 2022-09-19 14:35:08 +02:00
fortify_kunit.c fortify: Allow KUnit test to build without FORTIFY 2023-05-16 14:07:49 -07:00
gen_crc32table.c
gen_crc64table.c
genalloc.c lib/genalloc: use try_cmpxchg in {set,clear}_bits_ll 2023-02-02 22:50:05 -08:00
generic-radix-tree.c
glob.c
globtest.c
group_cpus.c lib/group_cpus: Export group_cpus_evenly() 2023-04-21 03:02:31 -04:00
hashtable_test.c lib/hashtable_test.c: add test for the hashtable structure 2023-02-08 14:28:17 -07:00
hexdump.c
hweight.c
idr.c
inflate.c
interval_tree.c interval-tree: Add a utility to iterate over spans in an interval tree 2022-11-29 16:34:15 -04:00
interval_tree_test.c
iomap.c kmsan: add iomap support 2022-10-03 14:03:21 -07:00
iomap_copy.c
iommu-helper.c
iov_iter.c iov_iter: remove iov_iter_get_pages and iov_iter_get_pages_alloc 2023-06-16 10:08:09 -06:00
irq_poll.c
irq_regs.c
is_signed_type_kunit.c lib: assume char is unsigned 2022-11-19 00:56:15 +01:00
is_single_threaded.c
kasprintf.c
kfifo.c
klist.c
kobject.c kobject: Use return value of strreplace() 2023-06-05 15:31:12 -07:00
kobject_uevent.c
kstrtox.c
kstrtox.h
libcrc32c.c libcrc32c: remove crc32c_impl 2023-04-17 18:01:23 +02:00
linear_ranges.c
list-test.c list: test: Test the klist structure 2023-03-31 09:21:35 -06:00
list_debug.c
list_sort.c
llist.c llist: avoid extra memory read in llist_add_batch 2022-11-18 13:55:06 -08:00
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-rtmutex.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c
lockref.c lockref: stop doing cpu_relax in the cmpxchg loop 2023-01-13 14:35:38 -06:00
logic_iomem.c
logic_pio.c
lru_cache.c lru_cache: remove unused lc_private, lc_set, lc_index_of 2022-11-22 19:38:39 -07:00
lshrdi3.c
maple_tree.c maple_tree: simplify and clean up mas_wr_node_store() 2023-06-09 16:25:45 -07:00
memcat_p.c
memcpy_kunit.c kunit: memcpy: Split slow memcpy tests into MEMCPY_SLOW_KUNIT_TEST 2023-01-25 12:24:40 -08:00
memory-notifier-error-inject.c
memregion.c
memweight.c
muldi3.c
net_utils.c mac_pton: Clean up the header inclusions 2023-06-06 13:18:32 +02:00
netdev-notifier-error-inject.c
nlattr.c netlink: prevent potential spectre v1 gadgets 2023-01-20 17:52:32 -08:00
nmi_backtrace.c x86/nmi: Print reasons why backtrace NMIs are ignored 2023-01-19 15:55:12 -08:00
notifier-error-inject.c lib/notifier-error-inject: fix error when writing -errno to debugfs file 2022-11-30 16:13:16 -08:00
notifier-error-inject.h
objagg.c
of-reconfig-notifier-error-inject.c
oid_registry.c lib/oid_registry.c: remove redundant assignment to variable num 2022-11-18 13:55:06 -08:00
once.c once: rename _SLOW to _SLEEPABLE 2022-10-03 17:34:32 -07:00
overflow_kunit.c overflow: Add struct_size_t() helper 2023-05-26 13:52:19 -07:00
packing.c lib: packing: remove MODULE_LICENSE in non-modules 2023-03-09 23:08:04 -08:00
parman.c
parser.c lib: parser: update documentation for match_NUMBER functions 2023-03-02 21:54:22 -08:00
pci_iomap.c
percpu-refcount.c percpu-refcount: Use call_rcu_hurry() for atomic switch 2022-11-30 13:16:40 -08:00
percpu_counter.c pcpcntr: remove percpu_counter_sum_all() 2023-03-19 10:02:04 -07:00
percpu_test.c
plist.c
pm-notifier-error-inject.c
polynomial.c
radix-tree.c radix-tree: move declarations to header 2023-06-12 11:31:50 -07:00
radix-tree.h radix-tree: move declarations to header 2023-06-12 11:31:50 -07:00
random32.c treewide: use get_random_bytes() when possible 2022-10-11 17:42:58 -06:00
ratelimit.c ratelimit: Fix data-races in ___ratelimit(). 2022-08-24 13:46:57 +01:00
rbtree.c lib/rbtree: use '+' instead of '|' for setting color. 2023-04-18 16:39:33 -07:00
rbtree_test.c
rcuref.c atomics: Provide rcuref - scalable reference counting 2023-03-28 10:39:29 +02:00
ref_tracker.c lib/ref_tracker: remove warnings in case of allocation failure 2023-06-05 15:28:42 -07:00
refcount.c
rhashtable.c rhashtable: Allow rhashtable to be used from irq-safe contexts 2022-12-09 10:42:56 +00:00
sbitmap.c sbitmap: fix batching wakeup 2023-07-21 11:40:20 -06:00
scatterlist.c Move netfs_extract_iter_to_sg() to lib/scatterlist.c 2023-06-08 13:42:33 +02:00
seq_buf.c seq_buf: Add seq_buf_do_printk() helper 2023-04-25 21:03:14 -04:00
sg_pool.c lib/sg_pool: change module_init(sg_pool_init) to subsys_initcall 2022-09-23 16:46:19 +02:00
sg_split.c
siphash.c
siphash_kunit.c siphash: Convert selftest to KUnit 2022-11-01 10:04:52 -07:00
slub_kunit.c linux-kselftest-kunit-next-6.2-rc1 2022-12-12 16:42:57 -08:00
smp_processor_id.c
sort.c
stackdepot.c lib/stackdepot: kmsan: mark API outputs as initialized 2023-03-28 16:20:13 -07:00
stackinit_kunit.c kernel/range: Uplevel the cxl subsystem's range_contains() helper 2023-02-10 17:32:37 -08:00
stmp_device.c
strcat_kunit.c string: Add Kunit tests for strcat() family 2023-05-16 14:08:02 -07:00
string.c string: use __builtin_memcpy() in strlcpy/strlcat 2023-06-01 11:24:50 -07:00
string_helpers.c lib/string_helpers: Change returned value of the strreplace() 2023-06-05 15:31:12 -07:00
strncpy_from_user.c
strnlen_user.c
strscpy_kunit.c fortify: Short-circuit known-safe calls to strscpy() 2022-11-01 10:04:52 -07:00
syscall.c
test-kstrtox.c
test-string_helpers.c lib/test-string_helpers: replace UNESCAPE_ANY by UNESCAPE_ALL_MASK 2023-04-08 13:45:39 -07:00
test_bitmap.c lib/bitmap: add tests for for_each() loops 2022-10-01 10:22:58 -07:00
test_bitops.c
test_bits.c
test_blackhole_dev.c
test_bpf.c bpf/tests: Use struct_size() 2023-05-31 12:58:38 +02:00
test_debug_virtual.c
test_dynamic_debug.c dyndbg: test DECLARE_DYNDBG_CLASSMAP, sysfs nodes 2022-09-07 17:04:49 +02:00
test_firmware.c test_firmware: return ENOMEM instead of ENOSPC on failed memory allocation 2023-06-15 13:42:18 +02:00
test_fprobe.c fprobe: Pass return address to the handlers 2023-06-06 21:39:55 +09:00
test_fpu.c
test_free_pages.c
test_hash.c
test_hexdump.c treewide: use get_random_u32_inclusive() when possible 2022-11-18 02:18:02 +01:00
test_hmm.c hmm-tests: add test for migrate_device_range() 2022-10-12 18:51:50 -07:00
test_hmm_uapi.h hmm-tests: add test for migrate_device_range() 2022-10-12 18:51:50 -07:00
test_ida.c
test_kmod.c test_kmod: stop kernel-doc warnings 2023-01-25 14:07:21 -08:00
test_kprobes.c test_kprobes: Add recursed kprobe test case 2023-02-21 08:52:42 +09:00
test_linear_ranges.c lib/test_linear_ranges: Use LINEAR_RANGE() 2022-11-16 13:32:32 +00:00
test_list_sort.c treewide: use get_random_u32_below() instead of deprecated function 2022-11-18 02:15:15 +01:00
test_lockup.c
test_maple_tree.c maple_tree: update testing code for mas_{next,prev,walk} 2023-06-09 16:25:35 -07:00
test_memcat_p.c
test_meminit.c lib/test_meminit: add checks for the allocation functions 2022-10-12 18:51:49 -07:00
test_min_heap.c treewide: use get_random_u32() when possible 2022-10-11 17:42:58 -06:00
test_module.c
test_objagg.c treewide: use get_random_bytes() when possible 2022-10-11 17:42:58 -06:00
test_parman.c
test_printf.c mm, printk: introduce new format %pGt for page_type 2023-03-28 16:20:09 -07:00
test_ref_tracker.c lib/ref_tracker: improve printing stats 2023-06-05 15:28:42 -07:00
test_rhashtable.c Networking changes for 6.2. 2022-12-13 15:47:48 -08:00
test_scanf.c
test_sort.c
test_static_key_base.c
test_static_keys.c
test_string.c
test_sysctl.c test_sysclt: Test for registering a mount point 2023-06-18 02:32:54 -07:00
test_ubsan.c
test_user_copy.c
test_uuid.c
test_vmalloc.c lib/test_vmalloc.c: avoid garbage in page array 2023-06-12 11:31:51 -07:00
test_xarray.c
textsearch.c
timerqueue.c
trace_readwrite.c asm-generic/io: Add _RET_IP_ to MMIO trace for more accurate debug info 2022-11-21 22:02:10 +01:00
ts_bm.c lib/ts_bm: reset initial match offset for every block of text 2023-06-26 13:26:39 +02:00
ts_fsm.c
ts_kmp.c
ubsan.c ubsan: add prototypes for internal functions 2023-05-30 16:42:01 -07:00
ubsan.h ubsan: add prototypes for internal functions 2023-05-30 16:42:01 -07:00
ucmpdi2.c
ucs2_string.c
usercopy.c uaccess: Add speculation barrier to copy_from_user() 2023-02-21 14:45:22 -08:00
uuid.c treewide: use get_random_bytes() when possible 2022-10-11 17:42:58 -06:00
vsprintf.c - Nick Piggin's "shoot lazy tlbs" series, to improve the peformance of 2023-04-27 19:42:02 -07:00
win_minmax.c lib/win_minmax: use /* notation for regular comments 2023-01-11 16:14:21 -08:00
xarray.c radix-tree: move declarations to header 2023-06-12 11:31:50 -07:00
xxhash.c