Roman Gushchin 9b78faee48 mm: memcontrol: switch to rcu protection in drain_all_stock() ... commit e1a366be5c upstream. Commit 72f0184c8a ("mm, memcg: remove hotplug locking from try_charge") introduced css_tryget()/css_put() calls in drain_all_stock(), which are supposed to protect the target memory cgroup from being released during the mem_cgroup_is_descendant() call. However, it's not completely safe. In theory, memcg can go away between reading stock->cached pointer and calling css_tryget(). This can happen if drain_all_stock() races with drain_local_stock() performed on the remote cpu as a result of a work, scheduled by the previous invocation of drain_all_stock(). The race is a bit theoretical and there are few chances to trigger it, but the current code looks a bit confusing, so it makes sense to fix it anyway. The code looks like as if css_tryget() and css_put() are used to protect stocks drainage. It's not necessary because stocked pages are holding references to the cached cgroup. And it obviously won't work for works, scheduled on other cpus. So, let's read the stock->cached pointer and evaluate the memory cgroup inside a rcu read section, and get rid of css_tryget()/css_put() calls. Link: http://lkml.kernel.org/r/20190802192241.3253165-1-guro@fb.com Signed-off-by: Roman Gushchin <guro@fb.com> Acked-by: Michal Hocko <mhocko@suse.com> Cc: Hillf Danton <hdanton@sina.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Vladimir Davydov <vdavydov.dev@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Fixes: cdec2e4265 ("memcg: coalesce charging via percpu storage") Signed-off-by: GONG, Ruiqi <gongruiqi1@huawei.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>		2024-03-01 13:06:10 +01:00
..
kasan	panic: Consolidate open-coded panic_on_warn checks	2023-02-06 07:49:46 +01:00
backing-dev.c	mm: bdi: initialize bdi_min_ratio when bdi is unregistered	2021-12-14 10:18:05 +01:00
balloon_compaction.c
bootmem.c
cleancache.c
cma.c	mm/cma.c: fail if fixed declaration can't be honored	2019-08-06 19:06:51 +02:00
cma.h
cma_debug.c	mm/cma_debug.c: fix the break condition in cma_maxchunk_get()	2019-06-15 11:54:01 +02:00
compaction.c	mm/compaction.c: clear total_{migrate,free}_scanned before scanning a new zone	2019-10-05 13:10:13 +02:00
debug.c
debug_page_ref.c
dmapool.c
early_ioremap.c
fadvise.c
failslab.c
filemap.c	mm: fs: initialize fsdata passed to write_begin/write_end interface	2022-11-25 17:40:29 +01:00
frame_vector.c	v4l2: don't fall back to follow_pfn() if pin_user_pages_fast() fails	2022-12-08 11:18:32 +01:00
frontswap.c	treewide: Remove uninitialized_var() usage	2023-08-11 11:45:01 +02:00
gup.c	gup: document and work around "COW can break either way" issue	2021-04-28 13:16:51 +02:00
gup_benchmark.c	mm/gup_benchmark.c: prevent integer overflow in ioctl	2019-12-01 09:17:07 +01:00
highmem.c
hmm.c	mm/memory_hotplug: shrink zones when offlining memory	2020-01-29 16:43:27 +01:00
huge_memory.c	mm/huge_memory.c: don't discard hugepage if other processes are mapping it	2021-07-20 16:15:58 +02:00
hugetlb.c	mm,hugetlb: take hugetlb_lock before decrementing h->resv_huge_pages	2022-11-03 23:52:29 +09:00
hugetlb_cgroup.c	mm: hugetlb: switch to css_tryget() in hugetlb_cgroup_charge_cgroup()	2019-11-20 18:45:20 +01:00
hwpoison-inject.c
init-mm.c
internal.h	mm/thp: fix vma_address() if virtual address below file offset	2021-07-11 12:49:28 +02:00
interval_tree.c
Kconfig	mm/hmm: select mmu notifier when selecting HMM	2019-06-15 11:54:00 +02:00
Kconfig.debug
khugepaged.c	mm/khugepaged: invoke MMU notifiers in shmem/file collapse paths	2023-01-18 11:29:59 +01:00
kmemleak-test.c
kmemleak.c	Revert "mm: kmemleak: take a full lowmem check in kmemleak_*_phys()"	2022-09-15 12:17:02 +02:00
ksm.c	treewide: Remove uninitialized_var() usage	2023-08-11 11:45:01 +02:00
list_lru.c	mm: list_lru: set shrinker map bit when child nr_items is not zero	2020-12-11 13:25:02 +01:00
maccess.c	uaccess: Add non-pagefault user-space write function	2020-09-09 19:04:29 +02:00
madvise.c
Makefile
memblock.c	memblock: use kfree() to release kmalloced memblock regions	2022-03-02 11:38:14 +01:00
memcontrol.c	mm: memcontrol: switch to rcu protection in drain_all_stock()	2024-03-01 13:06:10 +01:00
memfd.c	memfd: fix F_SEAL_WRITE after shmem huge page allocated	2022-03-08 19:04:10 +01:00
memory-failure.c	mm/memory-failure: check the mapcount of the precise page	2024-01-15 18:23:42 +01:00
memory.c	mm: fix unmap_mapping_range high bits shift bug	2024-01-15 18:23:43 +01:00
memory_hotplug.c	mm/memory_hotplug: use "unsigned long" for PFN in zone_for_pfn_range()	2021-09-22 11:48:12 +02:00
mempolicy.c	treewide: Remove uninitialized_var() usage	2023-08-11 11:45:01 +02:00
mempool.c
memtest.c
migrate.c	mm/migrate_device.c: flush TLB while holding PTL	2022-10-05 10:36:44 +02:00
mincore.c	mm/mincore.c: make mincore() more conservative	2019-05-22 07:37:40 +02:00
mlock.c	mm/mlock.c: change count_mm_mlocked_page_nr return type	2019-07-10 09:53:40 +02:00
mm_init.c
mmap.c	mm: Fix TLB flush for not-first PFNMAP mappings in unmap_region()	2022-09-20 12:26:48 +02:00
mmu_context.c
mmu_notifier.c	mm/mmu_notifier: use hlist_add_head_rcu()	2019-07-31 07:27:08 +02:00
mmzone.c
mprotect.c	mm, numa: fix bad pmd by atomically check for pmd_trans_huge when marking page tables prot_numa	2020-03-11 14:15:00 +01:00
mremap.c	mmmremap.c: avoid pointless invalidate_range_start/end on mremap(old_size=0)	2022-04-15 14:15:05 +02:00
msync.c
nobootmem.c
nommu.c	x86/mm: split vmalloc_sync_all()	2020-03-25 08:06:13 +01:00
oom_kill.c	mm, oom: do not trigger out_of_memory from the #PF	2021-11-26 11:36:17 +01:00
page-writeback.c	mm/writeback: fix possible divide-by-zero in wb_dirty_limits(), again	2024-02-23 08:12:56 +01:00
page_alloc.c	mm/page_alloc: fix potential deadlock on zonelist_update_seq seqlock	2023-05-17 11:13:28 +02:00
page_counter.c	mm/page_counter.c: fix protection usage propagation	2020-08-21 11:05:33 +02:00
page_ext.c	mm/page_ext.c: fix an imbalance with kmemleak	2019-04-05 22:32:58 +02:00
page_idle.c	mm/page_idle.c: fix oops because end_pfn is larger than max_pfn	2019-07-03 13:14:45 +02:00
page_io.c	mm: fix unexpected zeroed page mapping with zram swap	2022-05-12 12:20:25 +02:00
page_isolation.c
page_owner.c	mm/page_owner: don't access uninitialized memmaps when reading /proc/pagetypeinfo	2019-10-29 09:19:58 +01:00
page_poison.c	page_poison: play nicely with KASAN	2019-04-05 22:32:59 +02:00
page_vma_mapped.c	mm/thp: another PVMW_SYNC fix in page_vma_mapped_walk()	2021-07-11 12:49:30 +02:00
pagewalk.c	mm: pagewalk: fix termination condition in walk_pte_range()	2020-10-01 13:14:32 +02:00
percpu-internal.h
percpu-km.c	percpu: convert spin_lock_irq to spin_lock_irqsave.	2019-02-12 19:47:12 +01:00
percpu-stats.c
percpu-vm.c
percpu.c	treewide: Remove uninitialized_var() usage	2023-08-11 11:45:01 +02:00
pgtable-generic.c	mm/thp: fix __split_huge_pmd_locked() on shmem migration entry	2021-07-11 12:49:27 +02:00
process_vm_access.c
quicklist.c
readahead.c	vfs: fix readahead(2) on block devices	2023-11-20 10:29:16 +01:00
rmap.c	mm/rmap: Fix anon_vma->degree ambiguity leading to double-reuse	2022-09-05 10:26:34 +02:00
rodata_test.c
shmem.c	shmem: fix a race between shmem_unused_huge_shrink and shmem_evict_inode	2022-01-27 09:04:16 +01:00
slab.c	mm/slab.c: fix an infinite loop in leaks_show()	2019-06-15 11:54:01 +02:00
slab.h	mm: kmemleak: slob: respect SLAB_NOLEAKTRACE flag	2021-11-26 11:36:23 +01:00
slab_common.c	mm: slab: fix kmem_cache_create failed when sysfs node not destroyed	2021-07-28 11:13:44 +02:00
slob.c
slub.c	treewide: Remove uninitialized_var() usage	2023-08-11 11:45:01 +02:00
sparse-vmemmap.c
sparse.c	mm/sparse: add the missing sparse_buffer_fini() in error branch	2021-05-22 10:59:38 +02:00
swap.c	treewide: Remove uninitialized_var() usage	2023-08-11 11:45:01 +02:00
swap_cgroup.c
swap_slots.c
swap_state.c	mm/swap_state: fix a data race in swapin_nr_pages	2020-10-01 13:14:47 +02:00
swapfile.c	mm/swap: fix swap_info_struct race between swapoff and get_swap_pages()	2023-04-20 12:04:39 +02:00
truncate.c	mm/thp: unmap_mapping_page() to fix THP truncate_cleanup_page()	2021-07-11 12:49:28 +02:00
usercopy.c	mm/usercopy: return 1 from hardened_usercopy __setup() handler	2022-04-15 14:14:59 +02:00
userfaultfd.c	userfaultfd: fix mmap_changing checking in mfill_atomic_hugetlb	2024-03-01 13:06:08 +01:00
util.c	random: move randomize_page() into mm where it belongs	2022-06-25 11:49:12 +02:00
vmacache.c
vmalloc.c	mm/vmalloc.c: don't dereference possible NULL pointer in __vunmap()	2020-06-03 08:19:49 +02:00
vmpressure.c	mm/vmpressure.c: fix a signedness bug in vmpressure_register_event()	2019-10-17 13:45:19 -07:00
vmscan.c	mm/vmscan.c: fix data races using kswapd_classzone_idx	2020-10-01 13:14:41 +02:00
vmstat.c	mm, vmstat: drop zone->lock in /proc/pagetypeinfo	2021-06-03 08:38:02 +02:00
workingset.c
z3fold.c
zbud.c
zpool.c
zsmalloc.c	zsmalloc: fix races between asynchronous zspage free and page migration	2022-06-06 08:24:21 +02:00
zswap.c