License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 14:07:57 +00:00
|
|
|
/* SPDX-License-Identifier: GPL-2.0 */
|
2015-05-13 19:38:40 +00:00
|
|
|
/*
|
|
|
|
* linux/cgroup-defs.h - basic definitions for cgroup
|
|
|
|
*
|
|
|
|
* This file provides basic type and interface. Include this file directly
|
|
|
|
* only if necessary to avoid cyclic dependencies.
|
|
|
|
*/
|
|
|
|
#ifndef _LINUX_CGROUP_DEFS_H
|
|
|
|
#define _LINUX_CGROUP_DEFS_H
|
|
|
|
|
|
|
|
#include <linux/limits.h>
|
|
|
|
#include <linux/list.h>
|
|
|
|
#include <linux/idr.h>
|
|
|
|
#include <linux/wait.h>
|
|
|
|
#include <linux/mutex.h>
|
|
|
|
#include <linux/rcupdate.h>
|
2017-03-08 08:00:40 +00:00
|
|
|
#include <linux/refcount.h>
|
2015-05-13 19:38:40 +00:00
|
|
|
#include <linux/percpu-refcount.h>
|
2015-05-13 20:35:16 +00:00
|
|
|
#include <linux/percpu-rwsem.h>
|
2017-09-25 15:12:05 +00:00
|
|
|
#include <linux/u64_stats_sync.h>
|
2015-05-13 19:38:40 +00:00
|
|
|
#include <linux/workqueue.h>
|
2021-12-16 02:55:38 +00:00
|
|
|
#include <linux/bpf-cgroup-defs.h>
|
2018-10-26 22:06:31 +00:00
|
|
|
#include <linux/psi_types.h>
|
2015-05-13 19:38:40 +00:00
|
|
|
|
|
|
|
#ifdef CONFIG_CGROUPS
|
|
|
|
|
|
|
|
struct cgroup;
|
|
|
|
struct cgroup_root;
|
|
|
|
struct cgroup_subsys;
|
|
|
|
struct cgroup_taskset;
|
|
|
|
struct kernfs_node;
|
|
|
|
struct kernfs_ops;
|
|
|
|
struct kernfs_open_file;
|
2015-05-29 08:52:59 +00:00
|
|
|
struct seq_file;
|
2019-03-05 23:45:48 +00:00
|
|
|
struct poll_table_struct;
|
2015-05-13 19:38:40 +00:00
|
|
|
|
|
|
|
#define MAX_CGROUP_TYPE_NAMELEN 32
|
|
|
|
#define MAX_CGROUP_ROOT_NAMELEN 64
|
|
|
|
#define MAX_CFTYPE_NAME 64
|
|
|
|
|
|
|
|
/* define the enumeration of all cgroup subsystems */
|
|
|
|
#define SUBSYS(_x) _x ## _cgrp_id,
|
|
|
|
enum cgroup_subsys_id {
|
|
|
|
#include <linux/cgroup_subsys.h>
|
|
|
|
CGROUP_SUBSYS_COUNT,
|
|
|
|
};
|
|
|
|
#undef SUBSYS
|
|
|
|
|
|
|
|
/* bits in struct cgroup_subsys_state flags field */
|
|
|
|
enum {
|
|
|
|
CSS_NO_REF = (1 << 0), /* no reference counting for this css */
|
|
|
|
CSS_ONLINE = (1 << 1), /* between ->css_online() and ->css_offline() */
|
|
|
|
CSS_RELEASED = (1 << 2), /* refcnt reached zero, released */
|
2016-03-03 14:57:58 +00:00
|
|
|
CSS_VISIBLE = (1 << 3), /* css is visible to userland */
|
2017-05-15 13:34:06 +00:00
|
|
|
CSS_DYING = (1 << 4), /* css is dying */
|
2015-05-13 19:38:40 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
/* bits in struct cgroup flags field */
|
|
|
|
enum {
|
|
|
|
/* Control Group requires release notifications to userspace */
|
|
|
|
CGRP_NOTIFY_ON_RELEASE,
|
|
|
|
/*
|
|
|
|
* Clone the parent's configuration when creating a new child
|
|
|
|
* cpuset cgroup. For historical reasons, this option can be
|
|
|
|
* specified at mount time and thus is implemented here.
|
|
|
|
*/
|
|
|
|
CGRP_CPUSET_CLONE_CHILDREN,
|
cgroup: cgroup v2 freezer
Cgroup v1 implements the freezer controller, which provides an ability
to stop the workload in a cgroup and temporarily free up some
resources (cpu, io, network bandwidth and, potentially, memory)
for some other tasks. Cgroup v2 lacks this functionality.
This patch implements freezer for cgroup v2.
Cgroup v2 freezer tries to put tasks into a state similar to jobctl
stop. This means that tasks can be killed, ptraced (using
PTRACE_SEIZE*), and interrupted. It is possible to attach to
a frozen task, get some information (e.g. read registers) and detach.
It's also possible to migrate a frozen tasks to another cgroup.
This differs cgroup v2 freezer from cgroup v1 freezer, which mostly
tried to imitate the system-wide freezer. However uninterruptible
sleep is fine when all tasks are going to be frozen (hibernation case),
it's not the acceptable state for some subset of the system.
Cgroup v2 freezer is not supporting freezing kthreads.
If a non-root cgroup contains kthread, the cgroup still can be frozen,
but the kthread will remain running, the cgroup will be shown
as non-frozen, and the notification will not be delivered.
* PTRACE_ATTACH is not working because non-fatal signal delivery
is blocked in frozen state.
There are some interface differences between cgroup v1 and cgroup v2
freezer too, which are required to conform the cgroup v2 interface
design principles:
1) There is no separate controller, which has to be turned on:
the functionality is always available and is represented by
cgroup.freeze and cgroup.events cgroup control files.
2) The desired state is defined by the cgroup.freeze control file.
Any hierarchical configuration is allowed.
3) The interface is asynchronous. The actual state is available
using cgroup.events control file ("frozen" field). There are no
dedicated transitional states.
4) It's allowed to make any changes with the cgroup hierarchy
(create new cgroups, remove old cgroups, move tasks between cgroups)
no matter if some cgroups are frozen.
Signed-off-by: Roman Gushchin <guro@fb.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
No-objection-from-me-by: Oleg Nesterov <oleg@redhat.com>
Cc: kernel-team@fb.com
2019-04-19 17:03:04 +00:00
|
|
|
|
|
|
|
/* Control group has to be frozen. */
|
|
|
|
CGRP_FREEZE,
|
|
|
|
|
|
|
|
/* Cgroup is frozen. */
|
|
|
|
CGRP_FROZEN,
|
2021-05-08 12:15:38 +00:00
|
|
|
|
|
|
|
/* Control group has to be killed. */
|
|
|
|
CGRP_KILL,
|
2015-05-13 19:38:40 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
/* cgroup_root->flags */
|
|
|
|
enum {
|
|
|
|
CGRP_ROOT_NOPREFIX = (1 << 1), /* mounted subsystems have no named prefix */
|
|
|
|
CGRP_ROOT_XATTR = (1 << 2), /* supports extended attributes */
|
2017-06-27 18:30:28 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Consider namespaces as delegation boundaries. If this flag is
|
|
|
|
* set, controller specific interface files in a namespace root
|
|
|
|
* aren't writeable from inside the namespace.
|
|
|
|
*/
|
|
|
|
CGRP_ROOT_NS_DELEGATE = (1 << 3),
|
2017-08-17 19:33:09 +00:00
|
|
|
|
2022-07-23 14:28:28 +00:00
|
|
|
/*
|
|
|
|
* Reduce latencies on dynamic cgroup modifications such as task
|
|
|
|
* migrations and controller on/offs by disabling percpu operation on
|
|
|
|
* cgroup_threadgroup_rwsem. This makes hot path operations such as
|
|
|
|
* forks and exits into the slow path and more expensive.
|
|
|
|
*
|
|
|
|
* The static usage pattern of creating a cgroup, enabling controllers,
|
|
|
|
* and then seeding it with CLONE_INTO_CGROUP doesn't require write
|
|
|
|
* locking cgroup_threadgroup_rwsem and thus doesn't benefit from
|
|
|
|
* favordynmod.
|
|
|
|
*/
|
|
|
|
CGRP_ROOT_FAVOR_DYNMODS = (1 << 4),
|
|
|
|
|
2017-08-17 19:33:09 +00:00
|
|
|
/*
|
|
|
|
* Enable cpuset controller in v1 cgroup to use v2 behavior.
|
|
|
|
*/
|
2022-07-23 14:28:28 +00:00
|
|
|
CGRP_ROOT_CPUSET_V2_MODE = (1 << 16),
|
mm, memcg: consider subtrees in memory.events
memory.stat and other files already consider subtrees in their output, and
we should too in order to not present an inconsistent interface.
The current situation is fairly confusing, because people interacting with
cgroups expect hierarchical behaviour in the vein of memory.stat,
cgroup.events, and other files. For example, this causes confusion when
debugging reclaim events under low, as currently these always read "0" at
non-leaf memcg nodes, which frequently causes people to misdiagnose breach
behaviour. The same confusion applies to other counters in this file when
debugging issues.
Aggregation is done at write time instead of at read-time since these
counters aren't hot (unlike memory.stat which is per-page, so it does it
at read time), and it makes sense to bundle this with the file
notifications.
After this patch, events are propagated up the hierarchy:
[root@ktst ~]# cat /sys/fs/cgroup/system.slice/memory.events
low 0
high 0
max 0
oom 0
oom_kill 0
[root@ktst ~]# systemd-run -p MemoryMax=1 true
Running as unit: run-r251162a189fb4562b9dabfdc9b0422f5.service
[root@ktst ~]# cat /sys/fs/cgroup/system.slice/memory.events
low 0
high 0
max 7
oom 1
oom_kill 1
As this is a change in behaviour, this can be reverted to the old
behaviour by mounting with the `memory_localevents' flag set. However, we
use the new behaviour by default as there's a lack of evidence that there
are any current users of memory.events that would find this change
undesirable.
akpm: this is a behaviour change, so Cc:stable. THis is so that
forthcoming distros which use cgroup v2 are more likely to pick up the
revised behaviour.
Link: http://lkml.kernel.org/r/20190208224419.GA24772@chrisdown.name
Signed-off-by: Chris Down <chris@chrisdown.name>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Shakeel Butt <shakeelb@google.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: Roman Gushchin <guro@fb.com>
Cc: Dennis Zhou <dennis@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-06-01 05:30:22 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Enable legacy local memory.events.
|
|
|
|
*/
|
2022-07-23 14:28:28 +00:00
|
|
|
CGRP_ROOT_MEMORY_LOCAL_EVENTS = (1 << 17),
|
mm: memcontrol: recursive memory.low protection
Right now, the effective protection of any given cgroup is capped by its
own explicit memory.low setting, regardless of what the parent says. The
reasons for this are mostly historical and ease of implementation: to make
delegation of memory.low safe, effective protection is the min() of all
memory.low up the tree.
Unfortunately, this limitation makes it impossible to protect an entire
subtree from another without forcing the user to make explicit protection
allocations all the way to the leaf cgroups - something that is highly
undesirable in real life scenarios.
Consider memory in a data center host. At the cgroup top level, we have a
distinction between system management software and the actual workload the
system is executing. Both branches are further subdivided into individual
services, job components etc.
We want to protect the workload as a whole from the system management
software, but that doesn't mean we want to protect and prioritize
individual workload wrt each other. Their memory demand can vary over
time, and we'd want the VM to simply cache the hottest data within the
workload subtree. Yet, the current memory.low limitations force us to
allocate a fixed amount of protection to each workload component in order
to get protection from system management software in general. This
results in very inefficient resource distribution.
Another concern with mandating downward allocation is that, as the
complexity of the cgroup tree grows, it gets harder for the lower levels
to be informed about decisions made at the host-level. Consider a
container inside a namespace that in turn creates its own nested tree of
cgroups to run multiple workloads. It'd be extremely difficult to
configure memory.low parameters in those leaf cgroups that on one hand
balance pressure among siblings as the container desires, while also
reflecting the host-level protection from e.g. rpm upgrades, that lie
beyond one or more delegation and namespacing points in the tree.
It's highly unusual from a cgroup interface POV that nested levels have to
be aware of and reflect decisions made at higher levels for them to be
effective.
To enable such use cases and scale configurability for complex trees, this
patch implements a resource inheritance model for memory that is similar
to how the CPU and the IO controller implement work-conserving resource
allocations: a share of a resource allocated to a subree always applies to
the entire subtree recursively, while allowing, but not mandating,
children to further specify distribution rules.
That means that if protection is explicitly allocated among siblings,
those configured shares are being followed during page reclaim just like
they are now. However, if the memory.low set at a higher level is not
fully claimed by the children in that subtree, the "floating" remainder is
applied to each cgroup in the tree in proportion to its size. Since
reclaim pressure is applied in proportion to size as well, each child in
that tree gets the same boost, and the effect is neutral among siblings -
with respect to each other, they behave as if no memory control was
enabled at all, and the VM simply balances the memory demands optimally
within the subtree. But collectively those cgroups enjoy a boost over the
cgroups in neighboring trees.
E.g. a leaf cgroup with a memory.low setting of 0 no longer means that
it's not getting a share of the hierarchically assigned resource, just
that it doesn't claim a fixed amount of it to protect from its siblings.
This allows us to recursively protect one subtree (workload) from another
(system management), while letting subgroups compete freely among each
other - without having to assign fixed shares to each leaf, and without
nested groups having to echo higher-level settings.
The floating protection composes naturally with fixed protection.
Consider the following example tree:
A A: low = 2G
/ \ A1: low = 1G
A1 A2 A2: low = 0G
As outside pressure is applied to this tree, A1 will enjoy a fixed
protection from A2 of 1G, but the remaining, unclaimed 1G from A is split
evenly among A1 and A2, coming out to 1.5G and 0.5G.
There is a slight risk of regressing theoretical setups where the
top-level cgroups don't know about the true budgeting and set bogusly high
"bypass" values that are meaningfully allocated down the tree. Such
setups would rely on unclaimed protection to be discarded, and
distributing it would change the intended behavior. Be safe and hide the
new behavior behind a mount option, 'memory_recursiveprot'.
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Tejun Heo <tj@kernel.org>
Acked-by: Roman Gushchin <guro@fb.com>
Acked-by: Chris Down <chris@chrisdown.name>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Michal Koutný <mkoutny@suse.com>
Link: http://lkml.kernel.org/r/20200227195606.46212-4-hannes@cmpxchg.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-04-02 04:07:07 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Enable recursive subtree protection
|
|
|
|
*/
|
2022-07-23 14:28:28 +00:00
|
|
|
CGRP_ROOT_MEMORY_RECURSIVE_PROT = (1 << 18),
|
2015-05-13 19:38:40 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
/* cftype->flags */
|
|
|
|
enum {
|
|
|
|
CFTYPE_ONLY_ON_ROOT = (1 << 0), /* only create on root cgrp */
|
|
|
|
CFTYPE_NOT_ON_ROOT = (1 << 1), /* don't create on root cgrp */
|
2017-06-27 18:30:28 +00:00
|
|
|
CFTYPE_NS_DELEGATABLE = (1 << 2), /* writeable beyond delegation boundaries */
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
CFTYPE_NO_PREFIX = (1 << 3), /* (DON'T USE FOR NEW FILES) no subsys prefix */
|
2015-09-18 21:54:23 +00:00
|
|
|
CFTYPE_WORLD_WRITABLE = (1 << 4), /* (DON'T USE FOR NEW FILES) S_IWUGO */
|
2018-11-08 15:08:46 +00:00
|
|
|
CFTYPE_DEBUG = (1 << 5), /* create when cgroup_debug */
|
2015-05-13 19:38:40 +00:00
|
|
|
|
|
|
|
/* internal flags, do not use outside cgroup core proper */
|
|
|
|
__CFTYPE_ONLY_ON_DFL = (1 << 16), /* only on default hierarchy */
|
|
|
|
__CFTYPE_NOT_ON_DFL = (1 << 17), /* not on default hierarchy */
|
2022-09-06 19:38:42 +00:00
|
|
|
__CFTYPE_ADDED = (1 << 18),
|
2015-05-13 19:38:40 +00:00
|
|
|
};
|
|
|
|
|
2015-09-18 21:54:23 +00:00
|
|
|
/*
|
|
|
|
* cgroup_file is the handle for a file instance created in a cgroup which
|
|
|
|
* is used, for example, to generate file changed notifications. This can
|
|
|
|
* be obtained by setting cftype->file_offset.
|
|
|
|
*/
|
|
|
|
struct cgroup_file {
|
|
|
|
/* do not access any fields from outside cgroup core */
|
|
|
|
struct kernfs_node *kn;
|
2018-04-26 21:29:04 +00:00
|
|
|
unsigned long notified_at;
|
|
|
|
struct timer_list notify_timer;
|
2015-09-18 21:54:23 +00:00
|
|
|
};
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/*
|
|
|
|
* Per-subsystem/per-cgroup state maintained by the system. This is the
|
|
|
|
* fundamental structural building block that controllers deal with.
|
|
|
|
*
|
|
|
|
* Fields marked with "PI:" are public and immutable and may be accessed
|
|
|
|
* directly without synchronization.
|
|
|
|
*/
|
|
|
|
struct cgroup_subsys_state {
|
|
|
|
/* PI: the cgroup that this css is attached to */
|
|
|
|
struct cgroup *cgroup;
|
|
|
|
|
|
|
|
/* PI: the cgroup subsystem that this css is attached to */
|
|
|
|
struct cgroup_subsys *ss;
|
|
|
|
|
|
|
|
/* reference count - access via css_[try]get() and css_put() */
|
|
|
|
struct percpu_ref refcnt;
|
|
|
|
|
|
|
|
/* siblings list anchored at the parent's ->children */
|
|
|
|
struct list_head sibling;
|
|
|
|
struct list_head children;
|
|
|
|
|
2018-04-26 21:29:05 +00:00
|
|
|
/* flush target list anchored at cgrp->rstat_css_list */
|
|
|
|
struct list_head rstat_css_node;
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/*
|
|
|
|
* PI: Subsys-unique ID. 0 is unused and root is always 1. The
|
|
|
|
* matching css can be looked up using css_from_id().
|
|
|
|
*/
|
|
|
|
int id;
|
|
|
|
|
|
|
|
unsigned int flags;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Monotonically increasing unique serial number which defines a
|
|
|
|
* uniform order among all csses. It's guaranteed that all
|
|
|
|
* ->children lists are in the ascending order of ->serial_nr and
|
|
|
|
* used to allow interrupting and resuming iterations.
|
|
|
|
*/
|
|
|
|
u64 serial_nr;
|
|
|
|
|
2016-01-21 20:31:11 +00:00
|
|
|
/*
|
|
|
|
* Incremented by online self and children. Used to guarantee that
|
|
|
|
* parents are not offlined before their children.
|
|
|
|
*/
|
|
|
|
atomic_t online_cnt;
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/* percpu_ref killing and RCU release */
|
|
|
|
struct work_struct destroy_work;
|
2018-03-14 19:45:14 +00:00
|
|
|
struct rcu_work destroy_rwork;
|
2017-04-07 01:47:57 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* PI: the parent css. Placed here for cache proximity to following
|
|
|
|
* fields of the containing structure.
|
|
|
|
*/
|
|
|
|
struct cgroup_subsys_state *parent;
|
2015-05-13 19:38:40 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
/*
|
|
|
|
* A css_set is a structure holding pointers to a set of
|
|
|
|
* cgroup_subsys_state objects. This saves space in the task struct
|
|
|
|
* object and speeds up fork()/exit(), since a single inc/dec and a
|
|
|
|
* list_add()/del() can bump the reference count on the entire cgroup
|
|
|
|
* set for a task.
|
|
|
|
*/
|
|
|
|
struct css_set {
|
|
|
|
/*
|
2016-12-27 19:49:05 +00:00
|
|
|
* Set of subsystem states, one for each subsystem. This array is
|
|
|
|
* immutable after creation apart from the init_css_set during
|
|
|
|
* subsystem registration (at boot time).
|
2015-05-13 19:38:40 +00:00
|
|
|
*/
|
2016-12-27 19:49:05 +00:00
|
|
|
struct cgroup_subsys_state *subsys[CGROUP_SUBSYS_COUNT];
|
|
|
|
|
|
|
|
/* reference count */
|
2017-03-08 08:00:40 +00:00
|
|
|
refcount_t refcount;
|
2016-12-27 19:49:05 +00:00
|
|
|
|
2017-05-15 13:34:02 +00:00
|
|
|
/*
|
|
|
|
* For a domain cgroup, the following points to self. If threaded,
|
|
|
|
* to the matching cset of the nearest domain ancestor. The
|
|
|
|
* dom_cset provides access to the domain cgroup and its csses to
|
|
|
|
* which domain level resource consumptions should be charged.
|
|
|
|
*/
|
|
|
|
struct css_set *dom_cset;
|
|
|
|
|
2016-12-27 19:49:05 +00:00
|
|
|
/* the default cgroup associated with this css_set */
|
|
|
|
struct cgroup *dfl_cgrp;
|
2015-05-13 19:38:40 +00:00
|
|
|
|
2017-06-13 21:18:01 +00:00
|
|
|
/* internal task count, protected by css_set_lock */
|
|
|
|
int nr_tasks;
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/*
|
|
|
|
* Lists running through all tasks using this cgroup group.
|
|
|
|
* mg_tasks lists tasks which belong to this cset but are in the
|
|
|
|
* process of being migrated out or in. Protected by
|
|
|
|
* css_set_rwsem, but, during migration, once tasks are moved to
|
|
|
|
* mg_tasks, it can be read safely while holding cgroup_mutex.
|
|
|
|
*/
|
|
|
|
struct list_head tasks;
|
|
|
|
struct list_head mg_tasks;
|
2019-05-31 17:38:58 +00:00
|
|
|
struct list_head dying_tasks;
|
2015-05-13 19:38:40 +00:00
|
|
|
|
2016-12-27 19:49:05 +00:00
|
|
|
/* all css_task_iters currently walking this cset */
|
|
|
|
struct list_head task_iters;
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/*
|
2021-05-24 08:29:43 +00:00
|
|
|
* On the default hierarchy, ->subsys[ssid] may point to a css
|
2016-12-27 19:49:05 +00:00
|
|
|
* attached to an ancestor instead of the cgroup this css_set is
|
|
|
|
* associated with. The following node is anchored at
|
|
|
|
* ->subsys[ssid]->cgroup->e_csets[ssid] and provides a way to
|
|
|
|
* iterate through all css's attached to a given cgroup.
|
2015-05-13 19:38:40 +00:00
|
|
|
*/
|
2016-12-27 19:49:05 +00:00
|
|
|
struct list_head e_cset_node[CGROUP_SUBSYS_COUNT];
|
2015-05-13 19:38:40 +00:00
|
|
|
|
2017-05-15 13:34:02 +00:00
|
|
|
/* all threaded csets whose ->dom_cset points to this cset */
|
|
|
|
struct list_head threaded_csets;
|
|
|
|
struct list_head threaded_csets_node;
|
|
|
|
|
2016-12-27 19:49:05 +00:00
|
|
|
/*
|
|
|
|
* List running through all cgroup groups in the same hash
|
|
|
|
* slot. Protected by css_set_lock
|
|
|
|
*/
|
|
|
|
struct hlist_node hlist;
|
2015-05-13 19:38:40 +00:00
|
|
|
|
|
|
|
/*
|
2016-12-27 19:49:05 +00:00
|
|
|
* List of cgrp_cset_links pointing at cgroups referenced from this
|
|
|
|
* css_set. Protected by css_set_lock.
|
2015-05-13 19:38:40 +00:00
|
|
|
*/
|
2016-12-27 19:49:05 +00:00
|
|
|
struct list_head cgrp_links;
|
2015-05-13 19:38:40 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* List of csets participating in the on-going migration either as
|
|
|
|
* source or destination. Protected by cgroup_mutex.
|
|
|
|
*/
|
cgroup: Use separate src/dst nodes when preloading css_sets for migration
Each cset (css_set) is pinned by its tasks. When we're moving tasks around
across csets for a migration, we need to hold the source and destination
csets to ensure that they don't go away while we're moving tasks about. This
is done by linking cset->mg_preload_node on either the
mgctx->preloaded_src_csets or mgctx->preloaded_dst_csets list. Using the
same cset->mg_preload_node for both the src and dst lists was deemed okay as
a cset can't be both the source and destination at the same time.
Unfortunately, this overloading becomes problematic when multiple tasks are
involved in a migration and some of them are identity noop migrations while
others are actually moving across cgroups. For example, this can happen with
the following sequence on cgroup1:
#1> mkdir -p /sys/fs/cgroup/misc/a/b
#2> echo $$ > /sys/fs/cgroup/misc/a/cgroup.procs
#3> RUN_A_COMMAND_WHICH_CREATES_MULTIPLE_THREADS &
#4> PID=$!
#5> echo $PID > /sys/fs/cgroup/misc/a/b/tasks
#6> echo $PID > /sys/fs/cgroup/misc/a/cgroup.procs
the process including the group leader back into a. In this final migration,
non-leader threads would be doing identity migration while the group leader
is doing an actual one.
After #3, let's say the whole process was in cset A, and that after #4, the
leader moves to cset B. Then, during #6, the following happens:
1. cgroup_migrate_add_src() is called on B for the leader.
2. cgroup_migrate_add_src() is called on A for the other threads.
3. cgroup_migrate_prepare_dst() is called. It scans the src list.
4. It notices that B wants to migrate to A, so it tries to A to the dst
list but realizes that its ->mg_preload_node is already busy.
5. and then it notices A wants to migrate to A as it's an identity
migration, it culls it by list_del_init()'ing its ->mg_preload_node and
putting references accordingly.
6. The rest of migration takes place with B on the src list but nothing on
the dst list.
This means that A isn't held while migration is in progress. If all tasks
leave A before the migration finishes and the incoming task pins it, the
cset will be destroyed leading to use-after-free.
This is caused by overloading cset->mg_preload_node for both src and dst
preload lists. We wanted to exclude the cset from the src list but ended up
inadvertently excluding it from the dst list too.
This patch fixes the issue by separating out cset->mg_preload_node into
->mg_src_preload_node and ->mg_dst_preload_node, so that the src and dst
preloadings don't interfere with each other.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Mukesh Ojha <quic_mojha@quicinc.com>
Reported-by: shisiyuan <shisiyuan19870131@gmail.com>
Link: http://lkml.kernel.org/r/1654187688-27411-1-git-send-email-shisiyuan@xiaomi.com
Link: https://www.spinics.net/lists/cgroups/msg33313.html
Fixes: f817de98513d ("cgroup: prepare migration path for unified hierarchy")
Cc: stable@vger.kernel.org # v3.16+
2022-06-13 22:19:50 +00:00
|
|
|
struct list_head mg_src_preload_node;
|
|
|
|
struct list_head mg_dst_preload_node;
|
2015-05-13 19:38:40 +00:00
|
|
|
struct list_head mg_node;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If this cset is acting as the source of migration the following
|
2016-03-08 16:51:26 +00:00
|
|
|
* two fields are set. mg_src_cgrp and mg_dst_cgrp are
|
|
|
|
* respectively the source and destination cgroups of the on-going
|
|
|
|
* migration. mg_dst_cset is the destination cset the target tasks
|
|
|
|
* on this cset should be migrated to. Protected by cgroup_mutex.
|
2015-05-13 19:38:40 +00:00
|
|
|
*/
|
|
|
|
struct cgroup *mg_src_cgrp;
|
2016-03-08 16:51:26 +00:00
|
|
|
struct cgroup *mg_dst_cgrp;
|
2015-05-13 19:38:40 +00:00
|
|
|
struct css_set *mg_dst_cset;
|
|
|
|
|
cgroup: ignore css_sets associated with dead cgroups during migration
Before 2e91fa7f6d45 ("cgroup: keep zombies associated with their
original cgroups"), all dead tasks were associated with init_css_set.
If a zombie task is requested for migration, while migration prep
operations would still be performed on init_css_set, the actual
migration would ignore zombie tasks. As init_css_set is always valid,
this worked fine.
However, after 2e91fa7f6d45, zombie tasks stay with the css_set it was
associated with at the time of death. Let's say a task T associated
with cgroup A on hierarchy H-1 and cgroup B on hiearchy H-2. After T
becomes a zombie, it would still remain associated with A and B. If A
only contains zombie tasks, it can be removed. On removal, A gets
marked offline but stays pinned until all zombies are drained. At
this point, if migration is initiated on T to a cgroup C on hierarchy
H-2, migration path would try to prepare T's css_set for migration and
trigger the following.
WARNING: CPU: 0 PID: 1576 at kernel/cgroup.c:474 cgroup_get+0x121/0x160()
CPU: 0 PID: 1576 Comm: bash Not tainted 4.4.0-work+ #289
...
Call Trace:
[<ffffffff8127e63c>] dump_stack+0x4e/0x82
[<ffffffff810445e8>] warn_slowpath_common+0x78/0xb0
[<ffffffff810446d5>] warn_slowpath_null+0x15/0x20
[<ffffffff810c33e1>] cgroup_get+0x121/0x160
[<ffffffff810c349b>] link_css_set+0x7b/0x90
[<ffffffff810c4fbc>] find_css_set+0x3bc/0x5e0
[<ffffffff810c5269>] cgroup_migrate_prepare_dst+0x89/0x1f0
[<ffffffff810c7547>] cgroup_attach_task+0x157/0x230
[<ffffffff810c7a17>] __cgroup_procs_write+0x2b7/0x470
[<ffffffff810c7bdc>] cgroup_tasks_write+0xc/0x10
[<ffffffff810c4790>] cgroup_file_write+0x30/0x1b0
[<ffffffff811c68fc>] kernfs_fop_write+0x13c/0x180
[<ffffffff81151673>] __vfs_write+0x23/0xe0
[<ffffffff81152494>] vfs_write+0xa4/0x1a0
[<ffffffff811532d4>] SyS_write+0x44/0xa0
[<ffffffff814af2d7>] entry_SYSCALL_64_fastpath+0x12/0x6f
It doesn't make sense to prepare migration for css_sets pointing to
dead cgroups as they are guaranteed to contain only zombies which are
ignored later during migration. This patch makes cgroup destruction
path mark all affected css_sets as dead and updates the migration path
to ignore them during preparation.
Signed-off-by: Tejun Heo <tj@kernel.org>
Fixes: 2e91fa7f6d45 ("cgroup: keep zombies associated with their original cgroups")
Cc: stable@vger.kernel.org # v4.4+
2016-03-16 00:43:04 +00:00
|
|
|
/* dead and being drained, ignore for migration */
|
|
|
|
bool dead;
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/* For RCU-protected deletion */
|
|
|
|
struct rcu_head rcu_head;
|
|
|
|
};
|
|
|
|
|
2018-04-26 21:29:04 +00:00
|
|
|
struct cgroup_base_stat {
|
|
|
|
struct task_cputime cputime;
|
2022-06-29 21:14:26 +00:00
|
|
|
|
|
|
|
#ifdef CONFIG_SCHED_CORE
|
|
|
|
u64 forceidle_sum;
|
|
|
|
#endif
|
2018-04-26 21:29:04 +00:00
|
|
|
};
|
|
|
|
|
2017-09-25 15:12:05 +00:00
|
|
|
/*
|
2018-04-26 21:29:04 +00:00
|
|
|
* rstat - cgroup scalable recursive statistics. Accounting is done
|
|
|
|
* per-cpu in cgroup_rstat_cpu which is then lazily propagated up the
|
|
|
|
* hierarchy on reads.
|
2017-09-25 15:12:05 +00:00
|
|
|
*
|
2018-04-26 21:29:04 +00:00
|
|
|
* When a stat gets updated, the cgroup_rstat_cpu and its ancestors are
|
2017-09-25 15:12:05 +00:00
|
|
|
* linked into the updated tree. On the following read, propagation only
|
|
|
|
* considers and consumes the updated tree. This makes reading O(the
|
|
|
|
* number of descendants which have been active since last read) instead of
|
|
|
|
* O(the total number of descendants).
|
|
|
|
*
|
|
|
|
* This is important because there can be a lot of (draining) cgroups which
|
|
|
|
* aren't active and stat may be read frequently. The combination can
|
|
|
|
* become very expensive. By propagating selectively, increasing reading
|
|
|
|
* frequency decreases the cost of each read.
|
2018-04-26 21:29:04 +00:00
|
|
|
*
|
|
|
|
* This struct hosts both the fields which implement the above -
|
|
|
|
* updated_children and updated_next - and the fields which track basic
|
|
|
|
* resource statistics on top of it - bsync, bstat and last_bstat.
|
2017-09-25 15:12:05 +00:00
|
|
|
*/
|
2018-04-26 21:29:04 +00:00
|
|
|
struct cgroup_rstat_cpu {
|
2017-09-25 15:12:05 +00:00
|
|
|
/*
|
2018-04-26 21:29:04 +00:00
|
|
|
* ->bsync protects ->bstat. These are the only fields which get
|
|
|
|
* updated in the hot path.
|
2017-09-25 15:12:05 +00:00
|
|
|
*/
|
2018-04-26 21:29:04 +00:00
|
|
|
struct u64_stats_sync bsync;
|
|
|
|
struct cgroup_base_stat bstat;
|
2017-09-25 15:12:05 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Snapshots at the last reading. These are used to calculate the
|
|
|
|
* deltas to propagate to the global counters.
|
|
|
|
*/
|
2018-04-26 21:29:04 +00:00
|
|
|
struct cgroup_base_stat last_bstat;
|
2017-09-25 15:12:05 +00:00
|
|
|
|
2023-08-07 03:29:30 +00:00
|
|
|
/*
|
|
|
|
* This field is used to record the cumulative per-cpu time of
|
|
|
|
* the cgroup and its descendants. Currently it can be read via
|
|
|
|
* eBPF/drgn etc, and we are still trying to determine how to
|
|
|
|
* expose it in the cgroupfs interface.
|
|
|
|
*/
|
|
|
|
struct cgroup_base_stat subtree_bstat;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Snapshots at the last reading. These are used to calculate the
|
|
|
|
* deltas to propagate to the per-cpu subtree_bstat.
|
|
|
|
*/
|
|
|
|
struct cgroup_base_stat last_subtree_bstat;
|
|
|
|
|
2017-09-25 15:12:05 +00:00
|
|
|
/*
|
|
|
|
* Child cgroups with stat updates on this cpu since the last read
|
|
|
|
* are linked on the parent's ->updated_children through
|
|
|
|
* ->updated_next.
|
|
|
|
*
|
|
|
|
* In addition to being more compact, singly-linked list pointing
|
|
|
|
* to the cgroup makes it unnecessary for each per-cpu struct to
|
|
|
|
* point back to the associated cgroup.
|
|
|
|
*
|
2018-04-26 21:29:04 +00:00
|
|
|
* Protected by per-cpu cgroup_rstat_cpu_lock.
|
2017-09-25 15:12:05 +00:00
|
|
|
*/
|
|
|
|
struct cgroup *updated_children; /* terminated by self cgroup */
|
|
|
|
struct cgroup *updated_next; /* NULL iff not on the list */
|
|
|
|
};
|
|
|
|
|
cgroup: cgroup v2 freezer
Cgroup v1 implements the freezer controller, which provides an ability
to stop the workload in a cgroup and temporarily free up some
resources (cpu, io, network bandwidth and, potentially, memory)
for some other tasks. Cgroup v2 lacks this functionality.
This patch implements freezer for cgroup v2.
Cgroup v2 freezer tries to put tasks into a state similar to jobctl
stop. This means that tasks can be killed, ptraced (using
PTRACE_SEIZE*), and interrupted. It is possible to attach to
a frozen task, get some information (e.g. read registers) and detach.
It's also possible to migrate a frozen tasks to another cgroup.
This differs cgroup v2 freezer from cgroup v1 freezer, which mostly
tried to imitate the system-wide freezer. However uninterruptible
sleep is fine when all tasks are going to be frozen (hibernation case),
it's not the acceptable state for some subset of the system.
Cgroup v2 freezer is not supporting freezing kthreads.
If a non-root cgroup contains kthread, the cgroup still can be frozen,
but the kthread will remain running, the cgroup will be shown
as non-frozen, and the notification will not be delivered.
* PTRACE_ATTACH is not working because non-fatal signal delivery
is blocked in frozen state.
There are some interface differences between cgroup v1 and cgroup v2
freezer too, which are required to conform the cgroup v2 interface
design principles:
1) There is no separate controller, which has to be turned on:
the functionality is always available and is represented by
cgroup.freeze and cgroup.events cgroup control files.
2) The desired state is defined by the cgroup.freeze control file.
Any hierarchical configuration is allowed.
3) The interface is asynchronous. The actual state is available
using cgroup.events control file ("frozen" field). There are no
dedicated transitional states.
4) It's allowed to make any changes with the cgroup hierarchy
(create new cgroups, remove old cgroups, move tasks between cgroups)
no matter if some cgroups are frozen.
Signed-off-by: Roman Gushchin <guro@fb.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
No-objection-from-me-by: Oleg Nesterov <oleg@redhat.com>
Cc: kernel-team@fb.com
2019-04-19 17:03:04 +00:00
|
|
|
struct cgroup_freezer_state {
|
|
|
|
/* Should the cgroup and its descendants be frozen. */
|
|
|
|
bool freeze;
|
|
|
|
|
|
|
|
/* Should the cgroup actually be frozen? */
|
|
|
|
int e_freeze;
|
|
|
|
|
|
|
|
/* Fields below are protected by css_set_lock */
|
|
|
|
|
|
|
|
/* Number of frozen descendant cgroups */
|
|
|
|
int nr_frozen_descendants;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Number of tasks, which are counted as frozen:
|
|
|
|
* frozen, SIGSTOPped, and PTRACEd.
|
|
|
|
*/
|
|
|
|
int nr_frozen_tasks;
|
|
|
|
};
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
struct cgroup {
|
|
|
|
/* self css with NULL ->ss, points back to this cgroup */
|
|
|
|
struct cgroup_subsys_state self;
|
|
|
|
|
|
|
|
unsigned long flags; /* "unsigned long" so bitops work */
|
|
|
|
|
2015-11-20 20:55:52 +00:00
|
|
|
/*
|
|
|
|
* The depth this cgroup is at. The root is at depth zero and each
|
|
|
|
* step down the hierarchy increments the level. This along with
|
2022-07-29 23:10:16 +00:00
|
|
|
* ancestors[] can determine whether a given cgroup is a
|
2015-11-20 20:55:52 +00:00
|
|
|
* descendant of another without traversing the hierarchy.
|
|
|
|
*/
|
|
|
|
int level;
|
|
|
|
|
2017-07-28 17:28:44 +00:00
|
|
|
/* Maximum allowed descent tree depth */
|
|
|
|
int max_depth;
|
|
|
|
|
2017-08-02 16:55:29 +00:00
|
|
|
/*
|
|
|
|
* Keep track of total numbers of visible and dying descent cgroups.
|
|
|
|
* Dying cgroups are cgroups which were deleted by a user,
|
|
|
|
* but are still existing because someone else is holding a reference.
|
2017-07-28 17:28:44 +00:00
|
|
|
* max_descendants is a maximum allowed number of descent cgroups.
|
2019-04-19 17:03:03 +00:00
|
|
|
*
|
|
|
|
* nr_descendants and nr_dying_descendants are protected
|
|
|
|
* by cgroup_mutex and css_set_lock. It's fine to read them holding
|
|
|
|
* any of cgroup_mutex and css_set_lock; for writing both locks
|
|
|
|
* should be held.
|
2017-08-02 16:55:29 +00:00
|
|
|
*/
|
|
|
|
int nr_descendants;
|
|
|
|
int nr_dying_descendants;
|
2017-07-28 17:28:44 +00:00
|
|
|
int max_descendants;
|
2017-08-02 16:55:29 +00:00
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/*
|
2015-10-15 20:41:49 +00:00
|
|
|
* Each non-empty css_set associated with this cgroup contributes
|
2017-07-17 01:43:33 +00:00
|
|
|
* one to nr_populated_csets. The counter is zero iff this cgroup
|
|
|
|
* doesn't have any tasks.
|
|
|
|
*
|
|
|
|
* All children which have non-zero nr_populated_csets and/or
|
2017-05-15 13:34:02 +00:00
|
|
|
* nr_populated_children of their own contribute one to either
|
|
|
|
* nr_populated_domain_children or nr_populated_threaded_children
|
|
|
|
* depending on their type. Each counter is zero iff all cgroups
|
|
|
|
* of the type in the subtree proper don't have any tasks.
|
2015-05-13 19:38:40 +00:00
|
|
|
*/
|
2017-07-17 01:43:33 +00:00
|
|
|
int nr_populated_csets;
|
2017-05-15 13:34:02 +00:00
|
|
|
int nr_populated_domain_children;
|
|
|
|
int nr_populated_threaded_children;
|
|
|
|
|
|
|
|
int nr_threaded_children; /* # of live threaded child cgroups */
|
2015-05-13 19:38:40 +00:00
|
|
|
|
|
|
|
struct kernfs_node *kn; /* cgroup kernfs entry */
|
2015-09-18 21:54:23 +00:00
|
|
|
struct cgroup_file procs_file; /* handle for "cgroup.procs" */
|
|
|
|
struct cgroup_file events_file; /* handle for "cgroup.events" */
|
2015-05-13 19:38:40 +00:00
|
|
|
|
sched/psi: Per-cgroup PSI accounting disable/re-enable interface
PSI accounts stalls for each cgroup separately and aggregates it
at each level of the hierarchy. This may cause non-negligible overhead
for some workloads when under deep level of the hierarchy.
commit 3958e2d0c34e ("cgroup: make per-cgroup pressure stall tracking configurable")
make PSI to skip per-cgroup stall accounting, only account system-wide
to avoid this each level overhead.
But for our use case, we also want leaf cgroup PSI stats accounted for
userspace adjustment on that cgroup, apart from only system-wide adjustment.
So this patch introduce a per-cgroup PSI accounting disable/re-enable
interface "cgroup.pressure", which is a read-write single value file that
allowed values are "0" and "1", the defaults is "1" so per-cgroup
PSI stats is enabled by default.
Implementation details:
It should be relatively straight-forward to disable and re-enable
state aggregation, time tracking, averaging on a per-cgroup level,
if we can live with losing history from while it was disabled.
I.e. the avgs will restart from 0, total= will have gaps.
But it's hard or complex to stop/restart groupc->tasks[] updates,
which is not implemented in this patch. So we always update
groupc->tasks[] and PSI_ONCPU bit in psi_group_change() even when
the cgroup PSI stats is disabled.
Suggested-by: Johannes Weiner <hannes@cmpxchg.org>
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Link: https://lkml.kernel.org/r/20220907090332.2078-1-zhouchengming@bytedance.com
2022-09-07 09:03:32 +00:00
|
|
|
/* handles for "{cpu,memory,io,irq}.pressure" */
|
|
|
|
struct cgroup_file psi_files[NR_PSI_RESOURCES];
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/*
|
|
|
|
* The bitmask of subsystems enabled on the child cgroups.
|
|
|
|
* ->subtree_control is the one configured through
|
2021-12-01 01:17:36 +00:00
|
|
|
* "cgroup.subtree_control" while ->subtree_ss_mask is the effective
|
2016-02-23 03:25:46 +00:00
|
|
|
* one which may have more subsystems enabled. Controller knobs
|
|
|
|
* are made available iff it's enabled in ->subtree_control.
|
2015-05-13 19:38:40 +00:00
|
|
|
*/
|
2016-02-23 03:25:47 +00:00
|
|
|
u16 subtree_control;
|
|
|
|
u16 subtree_ss_mask;
|
2016-03-03 14:57:59 +00:00
|
|
|
u16 old_subtree_control;
|
|
|
|
u16 old_subtree_ss_mask;
|
2015-05-13 19:38:40 +00:00
|
|
|
|
|
|
|
/* Private pointers for each registered subsystem */
|
|
|
|
struct cgroup_subsys_state __rcu *subsys[CGROUP_SUBSYS_COUNT];
|
|
|
|
|
|
|
|
struct cgroup_root *root;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* List of cgrp_cset_links pointing at css_sets with tasks in this
|
|
|
|
* cgroup. Protected by css_set_lock.
|
|
|
|
*/
|
|
|
|
struct list_head cset_links;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* On the default hierarchy, a css_set for a cgroup with some
|
|
|
|
* susbsys disabled will point to css's which are associated with
|
|
|
|
* the closest ancestor which has the subsys enabled. The
|
|
|
|
* following lists all css_sets which point to this cgroup's css
|
|
|
|
* for the given subsystem.
|
|
|
|
*/
|
|
|
|
struct list_head e_csets[CGROUP_SUBSYS_COUNT];
|
|
|
|
|
2017-05-15 13:34:02 +00:00
|
|
|
/*
|
|
|
|
* If !threaded, self. If threaded, it points to the nearest
|
|
|
|
* domain ancestor. Inside a threaded subtree, cgroups are exempt
|
|
|
|
* from process granularity and no-internal-task constraint.
|
|
|
|
* Domain level resource consumptions which aren't tied to a
|
|
|
|
* specific task are charged to the dom_cgrp.
|
|
|
|
*/
|
|
|
|
struct cgroup *dom_cgrp;
|
2018-10-04 20:28:08 +00:00
|
|
|
struct cgroup *old_dom_cgrp; /* used while enabling threaded */
|
2017-05-15 13:34:02 +00:00
|
|
|
|
2018-04-26 21:29:04 +00:00
|
|
|
/* per-cpu recursive resource statistics */
|
|
|
|
struct cgroup_rstat_cpu __percpu *rstat_cpu;
|
2018-04-26 21:29:05 +00:00
|
|
|
struct list_head rstat_css_list;
|
2018-04-26 21:29:04 +00:00
|
|
|
|
2017-09-25 15:12:05 +00:00
|
|
|
/* cgroup basic resource statistics */
|
2019-11-06 20:49:57 +00:00
|
|
|
struct cgroup_base_stat last_bstat;
|
2018-04-26 21:29:04 +00:00
|
|
|
struct cgroup_base_stat bstat;
|
|
|
|
struct prev_cputime prev_cputime; /* for printing out cputime */
|
2017-09-25 15:12:05 +00:00
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/*
|
|
|
|
* list of pidlists, up to two for each namespace (one for procs, one
|
|
|
|
* for tasks); created on demand.
|
|
|
|
*/
|
|
|
|
struct list_head pidlists;
|
|
|
|
struct mutex pidlist_mutex;
|
|
|
|
|
|
|
|
/* used to wait for offlining of csses */
|
|
|
|
wait_queue_head_t offline_waitq;
|
|
|
|
|
|
|
|
/* used to schedule release agent */
|
|
|
|
struct work_struct release_agent_work;
|
2015-11-20 20:55:52 +00:00
|
|
|
|
2018-10-26 22:06:31 +00:00
|
|
|
/* used to track pressure stalls */
|
2022-05-26 12:26:56 +00:00
|
|
|
struct psi_group *psi;
|
2018-10-26 22:06:31 +00:00
|
|
|
|
2016-11-23 15:52:26 +00:00
|
|
|
/* used to store eBPF programs */
|
|
|
|
struct cgroup_bpf bpf;
|
|
|
|
|
2018-07-03 15:14:55 +00:00
|
|
|
/* If there is block congestion on this cgroup. */
|
|
|
|
atomic_t congestion_count;
|
|
|
|
|
cgroup: cgroup v2 freezer
Cgroup v1 implements the freezer controller, which provides an ability
to stop the workload in a cgroup and temporarily free up some
resources (cpu, io, network bandwidth and, potentially, memory)
for some other tasks. Cgroup v2 lacks this functionality.
This patch implements freezer for cgroup v2.
Cgroup v2 freezer tries to put tasks into a state similar to jobctl
stop. This means that tasks can be killed, ptraced (using
PTRACE_SEIZE*), and interrupted. It is possible to attach to
a frozen task, get some information (e.g. read registers) and detach.
It's also possible to migrate a frozen tasks to another cgroup.
This differs cgroup v2 freezer from cgroup v1 freezer, which mostly
tried to imitate the system-wide freezer. However uninterruptible
sleep is fine when all tasks are going to be frozen (hibernation case),
it's not the acceptable state for some subset of the system.
Cgroup v2 freezer is not supporting freezing kthreads.
If a non-root cgroup contains kthread, the cgroup still can be frozen,
but the kthread will remain running, the cgroup will be shown
as non-frozen, and the notification will not be delivered.
* PTRACE_ATTACH is not working because non-fatal signal delivery
is blocked in frozen state.
There are some interface differences between cgroup v1 and cgroup v2
freezer too, which are required to conform the cgroup v2 interface
design principles:
1) There is no separate controller, which has to be turned on:
the functionality is always available and is represented by
cgroup.freeze and cgroup.events cgroup control files.
2) The desired state is defined by the cgroup.freeze control file.
Any hierarchical configuration is allowed.
3) The interface is asynchronous. The actual state is available
using cgroup.events control file ("frozen" field). There are no
dedicated transitional states.
4) It's allowed to make any changes with the cgroup hierarchy
(create new cgroups, remove old cgroups, move tasks between cgroups)
no matter if some cgroups are frozen.
Signed-off-by: Roman Gushchin <guro@fb.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
No-objection-from-me-by: Oleg Nesterov <oleg@redhat.com>
Cc: kernel-team@fb.com
2019-04-19 17:03:04 +00:00
|
|
|
/* Used to store internal freezer state */
|
|
|
|
struct cgroup_freezer_state freezer;
|
|
|
|
|
bpf: Implement cgroup storage available to non-cgroup-attached bpf progs
Similar to sk/inode/task storage, implement similar cgroup local storage.
There already exists a local storage implementation for cgroup-attached
bpf programs. See map type BPF_MAP_TYPE_CGROUP_STORAGE and helper
bpf_get_local_storage(). But there are use cases such that non-cgroup
attached bpf progs wants to access cgroup local storage data. For example,
tc egress prog has access to sk and cgroup. It is possible to use
sk local storage to emulate cgroup local storage by storing data in socket.
But this is a waste as it could be lots of sockets belonging to a particular
cgroup. Alternatively, a separate map can be created with cgroup id as the key.
But this will introduce additional overhead to manipulate the new map.
A cgroup local storage, similar to existing sk/inode/task storage,
should help for this use case.
The life-cycle of storage is managed with the life-cycle of the
cgroup struct. i.e. the storage is destroyed along with the owning cgroup
with a call to bpf_cgrp_storage_free() when cgroup itself
is deleted.
The userspace map operations can be done by using a cgroup fd as a key
passed to the lookup, update and delete operations.
Typically, the following code is used to get the current cgroup:
struct task_struct *task = bpf_get_current_task_btf();
... task->cgroups->dfl_cgrp ...
and in structure task_struct definition:
struct task_struct {
....
struct css_set __rcu *cgroups;
....
}
With sleepable program, accessing task->cgroups is not protected by rcu_read_lock.
So the current implementation only supports non-sleepable program and supporting
sleepable program will be the next step together with adding rcu_read_lock
protection for rcu tagged structures.
Since map name BPF_MAP_TYPE_CGROUP_STORAGE has been used for old cgroup local
storage support, the new map name BPF_MAP_TYPE_CGRP_STORAGE is used
for cgroup storage available to non-cgroup-attached bpf programs. The old
cgroup storage supports bpf_get_local_storage() helper to get the cgroup data.
The new cgroup storage helper bpf_cgrp_storage_get() can provide similar
functionality. While old cgroup storage pre-allocates storage memory, the new
mechanism can also pre-allocate with a user space bpf_map_update_elem() call
to avoid potential run-time memory allocation failure.
Therefore, the new cgroup storage can provide all functionality w.r.t.
the old one. So in uapi bpf.h, the old BPF_MAP_TYPE_CGROUP_STORAGE is alias to
BPF_MAP_TYPE_CGROUP_STORAGE_DEPRECATED to indicate the old cgroup storage can
be deprecated since the new one can provide the same functionality.
Acked-by: David Vernet <void@manifault.com>
Signed-off-by: Yonghong Song <yhs@fb.com>
Link: https://lore.kernel.org/r/20221026042850.673791-1-yhs@fb.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2022-10-26 04:28:50 +00:00
|
|
|
#ifdef CONFIG_BPF_SYSCALL
|
|
|
|
struct bpf_local_storage __rcu *bpf_cgrp_storage;
|
|
|
|
#endif
|
|
|
|
|
2022-07-29 23:10:16 +00:00
|
|
|
/* All ancestors including self */
|
|
|
|
struct cgroup *ancestors[];
|
2015-05-13 19:38:40 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
/*
|
|
|
|
* A cgroup_root represents the root of a cgroup hierarchy, and may be
|
|
|
|
* associated with a kernfs_root to form an active hierarchy. This is
|
|
|
|
* internal to cgroup core. Don't access directly from controllers.
|
|
|
|
*/
|
|
|
|
struct cgroup_root {
|
|
|
|
struct kernfs_root *kf_root;
|
|
|
|
|
|
|
|
/* The bitmask of subsystems attached to this hierarchy */
|
|
|
|
unsigned int subsys_mask;
|
|
|
|
|
|
|
|
/* Unique id for this hierarchy. */
|
|
|
|
int hierarchy_id;
|
|
|
|
|
2022-07-29 23:10:16 +00:00
|
|
|
/*
|
|
|
|
* The root cgroup. The containing cgroup_root will be destroyed on its
|
|
|
|
* release. cgrp->ancestors[0] will be used overflowing into the
|
|
|
|
* following field. cgrp_ancestor_storage must immediately follow.
|
|
|
|
*/
|
2015-05-13 19:38:40 +00:00
|
|
|
struct cgroup cgrp;
|
|
|
|
|
2022-07-29 23:10:16 +00:00
|
|
|
/* must follow cgrp for cgrp->ancestors[0], see above */
|
|
|
|
struct cgroup *cgrp_ancestor_storage;
|
2015-11-20 20:55:52 +00:00
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/* Number of cgroups in the hierarchy, used only for /proc/cgroups */
|
|
|
|
atomic_t nr_cgrps;
|
|
|
|
|
|
|
|
/* A list running through the active hierarchies */
|
|
|
|
struct list_head root_list;
|
|
|
|
|
|
|
|
/* Hierarchy-specific flags */
|
|
|
|
unsigned int flags;
|
|
|
|
|
|
|
|
/* The path to use for release notifications. */
|
|
|
|
char release_agent_path[PATH_MAX];
|
|
|
|
|
|
|
|
/* The name for this hierarchy - may be empty */
|
|
|
|
char name[MAX_CGROUP_ROOT_NAMELEN];
|
|
|
|
};
|
|
|
|
|
|
|
|
/*
|
|
|
|
* struct cftype: handler definitions for cgroup control files
|
|
|
|
*
|
|
|
|
* When reading/writing to a file:
|
|
|
|
* - the cgroup to use is file->f_path.dentry->d_parent->d_fsdata
|
|
|
|
* - the 'cftype' of the file is file->f_path.dentry->d_fsdata
|
|
|
|
*/
|
|
|
|
struct cftype {
|
|
|
|
/*
|
|
|
|
* By convention, the name should begin with the name of the
|
|
|
|
* subsystem, followed by a period. Zero length string indicates
|
|
|
|
* end of cftype array.
|
|
|
|
*/
|
|
|
|
char name[MAX_CFTYPE_NAME];
|
2015-08-11 17:35:42 +00:00
|
|
|
unsigned long private;
|
2015-05-13 19:38:40 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* The maximum length of string, excluding trailing nul, that can
|
|
|
|
* be passed to write. If < PAGE_SIZE-1, PAGE_SIZE-1 is assumed.
|
|
|
|
*/
|
|
|
|
size_t max_write_len;
|
|
|
|
|
|
|
|
/* CFTYPE_* flags */
|
|
|
|
unsigned int flags;
|
|
|
|
|
2015-09-18 21:54:23 +00:00
|
|
|
/*
|
|
|
|
* If non-zero, should contain the offset from the start of css to
|
|
|
|
* a struct cgroup_file field. cgroup will record the handle of
|
|
|
|
* the created file into it. The recorded handle can be used as
|
|
|
|
* long as the containing css remains accessible.
|
|
|
|
*/
|
|
|
|
unsigned int file_offset;
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/*
|
|
|
|
* Fields used for internal bookkeeping. Initialized automatically
|
|
|
|
* during registration.
|
|
|
|
*/
|
|
|
|
struct cgroup_subsys *ss; /* NULL for cgroup core files */
|
|
|
|
struct list_head node; /* anchored at ss->cfts */
|
|
|
|
struct kernfs_ops *kf_ops;
|
|
|
|
|
2016-12-27 19:49:03 +00:00
|
|
|
int (*open)(struct kernfs_open_file *of);
|
|
|
|
void (*release)(struct kernfs_open_file *of);
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/*
|
|
|
|
* read_u64() is a shortcut for the common case of returning a
|
|
|
|
* single integer. Use it in place of read()
|
|
|
|
*/
|
|
|
|
u64 (*read_u64)(struct cgroup_subsys_state *css, struct cftype *cft);
|
|
|
|
/*
|
|
|
|
* read_s64() is a signed version of read_u64()
|
|
|
|
*/
|
|
|
|
s64 (*read_s64)(struct cgroup_subsys_state *css, struct cftype *cft);
|
|
|
|
|
|
|
|
/* generic seq_file read interface */
|
|
|
|
int (*seq_show)(struct seq_file *sf, void *v);
|
|
|
|
|
|
|
|
/* optional ops, implement all or none */
|
|
|
|
void *(*seq_start)(struct seq_file *sf, loff_t *ppos);
|
|
|
|
void *(*seq_next)(struct seq_file *sf, void *v, loff_t *ppos);
|
|
|
|
void (*seq_stop)(struct seq_file *sf, void *v);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* write_u64() is a shortcut for the common case of accepting
|
|
|
|
* a single integer (as parsed by simple_strtoull) from
|
|
|
|
* userspace. Use in place of write(); return 0 or error.
|
|
|
|
*/
|
|
|
|
int (*write_u64)(struct cgroup_subsys_state *css, struct cftype *cft,
|
|
|
|
u64 val);
|
|
|
|
/*
|
|
|
|
* write_s64() is a signed version of write_u64()
|
|
|
|
*/
|
|
|
|
int (*write_s64)(struct cgroup_subsys_state *css, struct cftype *cft,
|
|
|
|
s64 val);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* write() is the generic write callback which maps directly to
|
|
|
|
* kernfs write operation and overrides all other operations.
|
|
|
|
* Maximum write size is determined by ->max_write_len. Use
|
|
|
|
* of_css/cft() to access the associated css and cft.
|
|
|
|
*/
|
|
|
|
ssize_t (*write)(struct kernfs_open_file *of,
|
|
|
|
char *buf, size_t nbytes, loff_t off);
|
|
|
|
|
2019-03-05 23:45:48 +00:00
|
|
|
__poll_t (*poll)(struct kernfs_open_file *of,
|
|
|
|
struct poll_table_struct *pt);
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
#ifdef CONFIG_DEBUG_LOCK_ALLOC
|
|
|
|
struct lock_class_key lockdep_key;
|
|
|
|
#endif
|
|
|
|
};
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Control Group subsystem type.
|
2019-06-27 16:08:35 +00:00
|
|
|
* See Documentation/admin-guide/cgroup-v1/cgroups.rst for details
|
2015-05-13 19:38:40 +00:00
|
|
|
*/
|
|
|
|
struct cgroup_subsys {
|
|
|
|
struct cgroup_subsys_state *(*css_alloc)(struct cgroup_subsys_state *parent_css);
|
|
|
|
int (*css_online)(struct cgroup_subsys_state *css);
|
|
|
|
void (*css_offline)(struct cgroup_subsys_state *css);
|
|
|
|
void (*css_released)(struct cgroup_subsys_state *css);
|
|
|
|
void (*css_free)(struct cgroup_subsys_state *css);
|
|
|
|
void (*css_reset)(struct cgroup_subsys_state *css);
|
2018-04-26 21:29:05 +00:00
|
|
|
void (*css_rstat_flush)(struct cgroup_subsys_state *css, int cpu);
|
2017-10-23 23:18:27 +00:00
|
|
|
int (*css_extra_stat_show)(struct seq_file *seq,
|
|
|
|
struct cgroup_subsys_state *css);
|
2023-06-20 18:32:47 +00:00
|
|
|
int (*css_local_stat_show)(struct seq_file *seq,
|
|
|
|
struct cgroup_subsys_state *css);
|
2015-05-13 19:38:40 +00:00
|
|
|
|
cgroup: fix handling of multi-destination migration from subtree_control enabling
Consider the following v2 hierarchy.
P0 (+memory) --- P1 (-memory) --- A
\- B
P0 has memory enabled in its subtree_control while P1 doesn't. If
both A and B contain processes, they would belong to the memory css of
P1. Now if memory is enabled on P1's subtree_control, memory csses
should be created on both A and B and A's processes should be moved to
the former and B's processes the latter. IOW, enabling controllers
can cause atomic migrations into different csses.
The core cgroup migration logic has been updated accordingly but the
controller migration methods haven't and still assume that all tasks
migrate to a single target css; furthermore, the methods were fed the
css in which subtree_control was updated which is the parent of the
target csses. pids controller depends on the migration methods to
move charges and this made the controller attribute charges to the
wrong csses often triggering the following warning by driving a
counter negative.
WARNING: CPU: 1 PID: 1 at kernel/cgroup_pids.c:97 pids_cancel.constprop.6+0x31/0x40()
Modules linked in:
CPU: 1 PID: 1 Comm: systemd Not tainted 4.4.0-rc1+ #29
...
ffffffff81f65382 ffff88007c043b90 ffffffff81551ffc 0000000000000000
ffff88007c043bc8 ffffffff810de202 ffff88007a752000 ffff88007a29ab00
ffff88007c043c80 ffff88007a1d8400 0000000000000001 ffff88007c043bd8
Call Trace:
[<ffffffff81551ffc>] dump_stack+0x4e/0x82
[<ffffffff810de202>] warn_slowpath_common+0x82/0xc0
[<ffffffff810de2fa>] warn_slowpath_null+0x1a/0x20
[<ffffffff8118e031>] pids_cancel.constprop.6+0x31/0x40
[<ffffffff8118e0fd>] pids_can_attach+0x6d/0xf0
[<ffffffff81188a4c>] cgroup_taskset_migrate+0x6c/0x330
[<ffffffff81188e05>] cgroup_migrate+0xf5/0x190
[<ffffffff81189016>] cgroup_attach_task+0x176/0x200
[<ffffffff8118949d>] __cgroup_procs_write+0x2ad/0x460
[<ffffffff81189684>] cgroup_procs_write+0x14/0x20
[<ffffffff811854e5>] cgroup_file_write+0x35/0x1c0
[<ffffffff812e26f1>] kernfs_fop_write+0x141/0x190
[<ffffffff81265f88>] __vfs_write+0x28/0xe0
[<ffffffff812666fc>] vfs_write+0xac/0x1a0
[<ffffffff81267019>] SyS_write+0x49/0xb0
[<ffffffff81bcef32>] entry_SYSCALL_64_fastpath+0x12/0x76
This patch fixes the bug by removing @css parameter from the three
migration methods, ->can_attach, ->cancel_attach() and ->attach() and
updating cgroup_taskset iteration helpers also return the destination
css in addition to the task being migrated. All controllers are
updated accordingly.
* Controllers which don't care whether there are one or multiple
target csses can be converted trivially. cpu, io, freezer, perf,
netclassid and netprio fall in this category.
* cpuset's current implementation assumes that there's single source
and destination and thus doesn't support v2 hierarchy already. The
only change made by this patchset is how that single destination css
is obtained.
* memory migration path already doesn't do anything on v2. How the
single destination css is obtained is updated and the prep stage of
mem_cgroup_can_attach() is reordered to accomodate the change.
* pids is the only controller which was affected by this bug. It now
correctly handles multi-destination migrations and no longer causes
counter underflow from incorrect accounting.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-and-tested-by: Daniel Wagner <daniel.wagner@bmw-carit.de>
Cc: Aleksa Sarai <cyphar@cyphar.com>
2015-12-03 15:18:21 +00:00
|
|
|
int (*can_attach)(struct cgroup_taskset *tset);
|
|
|
|
void (*cancel_attach)(struct cgroup_taskset *tset);
|
|
|
|
void (*attach)(struct cgroup_taskset *tset);
|
2016-04-21 23:06:48 +00:00
|
|
|
void (*post_attach)(void);
|
2020-02-05 13:26:22 +00:00
|
|
|
int (*can_fork)(struct task_struct *task,
|
|
|
|
struct css_set *cset);
|
|
|
|
void (*cancel_fork)(struct task_struct *task, struct css_set *cset);
|
2015-12-03 15:24:08 +00:00
|
|
|
void (*fork)(struct task_struct *task);
|
2015-10-15 20:41:53 +00:00
|
|
|
void (*exit)(struct task_struct *task);
|
2019-01-28 16:00:13 +00:00
|
|
|
void (*release)(struct task_struct *task);
|
2015-05-13 19:38:40 +00:00
|
|
|
void (*bind)(struct cgroup_subsys_state *root_css);
|
|
|
|
|
2016-02-23 15:00:50 +00:00
|
|
|
bool early_init:1;
|
2015-05-13 19:38:40 +00:00
|
|
|
|
2016-03-08 16:51:26 +00:00
|
|
|
/*
|
|
|
|
* If %true, the controller, on the default hierarchy, doesn't show
|
|
|
|
* up in "cgroup.controllers" or "cgroup.subtree_control", is
|
|
|
|
* implicitly enabled on all cgroups on the default hierarchy, and
|
|
|
|
* bypasses the "no internal process" constraint. This is for
|
|
|
|
* utility type controllers which is transparent to userland.
|
|
|
|
*
|
|
|
|
* An implicit controller can be stolen from the default hierarchy
|
|
|
|
* anytime and thus must be okay with offline csses from previous
|
|
|
|
* hierarchies coexisting with csses for the current one.
|
|
|
|
*/
|
|
|
|
bool implicit_on_dfl:1;
|
|
|
|
|
cgroup: implement cgroup v2 thread support
This patch implements cgroup v2 thread support. The goal of the
thread mode is supporting hierarchical accounting and control at
thread granularity while staying inside the resource domain model
which allows coordination across different resource controllers and
handling of anonymous resource consumptions.
A cgroup is always created as a domain and can be made threaded by
writing to the "cgroup.type" file. When a cgroup becomes threaded, it
becomes a member of a threaded subtree which is anchored at the
closest ancestor which isn't threaded.
The threads of the processes which are in a threaded subtree can be
placed anywhere without being restricted by process granularity or
no-internal-process constraint. Note that the threads aren't allowed
to escape to a different threaded subtree. To be used inside a
threaded subtree, a controller should explicitly support threaded mode
and be able to handle internal competition in the way which is
appropriate for the resource.
The root of a threaded subtree, the nearest ancestor which isn't
threaded, is called the threaded domain and serves as the resource
domain for the whole subtree. This is the last cgroup where domain
controllers are operational and where all the domain-level resource
consumptions in the subtree are accounted. This allows threaded
controllers to operate at thread granularity when requested while
staying inside the scope of system-level resource distribution.
As the root cgroup is exempt from the no-internal-process constraint,
it can serve as both a threaded domain and a parent to normal cgroups,
so, unlike non-root cgroups, the root cgroup can have both domain and
threaded children.
Internally, in a threaded subtree, each css_set has its ->dom_cset
pointing to a matching css_set which belongs to the threaded domain.
This ensures that thread root level cgroup_subsys_state for all
threaded controllers are readily accessible for domain-level
operations.
This patch enables threaded mode for the pids and perf_events
controllers. Neither has to worry about domain-level resource
consumptions and it's enough to simply set the flag.
For more details on the interface and behavior of the thread mode,
please refer to the section 2-2-2 in Documentation/cgroup-v2.txt added
by this patch.
v5: - Dropped silly no-op ->dom_cgrp init from cgroup_create().
Spotted by Waiman.
- Documentation updated as suggested by Waiman.
- cgroup.type content slightly reformatted.
- Mark the debug controller threaded.
v4: - Updated to the general idea of marking specific cgroups
domain/threaded as suggested by PeterZ.
v3: - Dropped "join" and always make mixed children join the parent's
threaded subtree.
v2: - After discussions with Waiman, support for mixed thread mode is
added. This should address the issue that Peter pointed out
where any nesting should be avoided for thread subtrees while
coexisting with other domain cgroups.
- Enabling / disabling thread mode now piggy backs on the existing
control mask update mechanism.
- Bug fixes and cleanup.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Waiman Long <longman@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
2017-07-21 15:14:51 +00:00
|
|
|
/*
|
|
|
|
* If %true, the controller, supports threaded mode on the default
|
|
|
|
* hierarchy. In a threaded subtree, both process granularity and
|
|
|
|
* no-internal-process constraint are ignored and a threaded
|
|
|
|
* controllers should be able to handle that.
|
|
|
|
*
|
|
|
|
* Note that as an implicit controller is automatically enabled on
|
|
|
|
* all cgroups on the default hierarchy, it should also be
|
|
|
|
* threaded. implicit && !threaded is not supported.
|
|
|
|
*/
|
|
|
|
bool threaded:1;
|
|
|
|
|
2021-05-24 08:29:43 +00:00
|
|
|
/* the following two fields are initialized automatically during boot */
|
2015-05-13 19:38:40 +00:00
|
|
|
int id;
|
|
|
|
const char *name;
|
|
|
|
|
2015-08-18 20:58:16 +00:00
|
|
|
/* optional, initialized automatically during boot if not set */
|
|
|
|
const char *legacy_name;
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
/* link to parent, protected by cgroup_lock() */
|
|
|
|
struct cgroup_root *root;
|
|
|
|
|
|
|
|
/* idr for css->id */
|
|
|
|
struct idr css_idr;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* List of cftypes. Each entry is the first entry of an array
|
|
|
|
* terminated by zero length name.
|
|
|
|
*/
|
|
|
|
struct list_head cfts;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Base cftypes which are automatically registered. The two can
|
|
|
|
* point to the same array.
|
|
|
|
*/
|
|
|
|
struct cftype *dfl_cftypes; /* for the default hierarchy */
|
|
|
|
struct cftype *legacy_cftypes; /* for the legacy hierarchies */
|
|
|
|
|
|
|
|
/*
|
|
|
|
* A subsystem may depend on other subsystems. When such subsystem
|
|
|
|
* is enabled on a cgroup, the depended-upon subsystems are enabled
|
|
|
|
* together if available. Subsystems enabled due to dependency are
|
|
|
|
* not visible to userland until explicitly enabled. The following
|
|
|
|
* specifies the mask of subsystems that this one depends on.
|
|
|
|
*/
|
|
|
|
unsigned int depends_on;
|
|
|
|
};
|
|
|
|
|
2015-09-16 16:53:17 +00:00
|
|
|
extern struct percpu_rw_semaphore cgroup_threadgroup_rwsem;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* cgroup_threadgroup_change_begin - threadgroup exclusion for cgroups
|
|
|
|
* @tsk: target task
|
|
|
|
*
|
2017-02-02 10:50:56 +00:00
|
|
|
* Allows cgroup operations to synchronize against threadgroup changes
|
|
|
|
* using a percpu_rw_semaphore.
|
2015-09-16 16:53:17 +00:00
|
|
|
*/
|
|
|
|
static inline void cgroup_threadgroup_change_begin(struct task_struct *tsk)
|
|
|
|
{
|
|
|
|
percpu_down_read(&cgroup_threadgroup_rwsem);
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* cgroup_threadgroup_change_end - threadgroup exclusion for cgroups
|
|
|
|
* @tsk: target task
|
|
|
|
*
|
2017-02-02 10:50:56 +00:00
|
|
|
* Counterpart of cgroup_threadcgroup_change_begin().
|
2015-09-16 16:53:17 +00:00
|
|
|
*/
|
|
|
|
static inline void cgroup_threadgroup_change_end(struct task_struct *tsk)
|
|
|
|
{
|
|
|
|
percpu_up_read(&cgroup_threadgroup_rwsem);
|
|
|
|
}
|
2015-05-13 20:35:16 +00:00
|
|
|
|
|
|
|
#else /* CONFIG_CGROUPS */
|
|
|
|
|
2015-06-06 00:02:14 +00:00
|
|
|
#define CGROUP_SUBSYS_COUNT 0
|
|
|
|
|
2017-02-02 10:50:56 +00:00
|
|
|
static inline void cgroup_threadgroup_change_begin(struct task_struct *tsk)
|
|
|
|
{
|
|
|
|
might_sleep();
|
|
|
|
}
|
|
|
|
|
2015-05-13 20:35:16 +00:00
|
|
|
static inline void cgroup_threadgroup_change_end(struct task_struct *tsk) {}
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
#endif /* CONFIG_CGROUPS */
|
2015-05-13 20:35:16 +00:00
|
|
|
|
2015-12-07 22:38:52 +00:00
|
|
|
#ifdef CONFIG_SOCK_CGROUP_DATA
|
|
|
|
|
sock, cgroup: add sock->sk_cgroup
In cgroup v1, dealing with cgroup membership was difficult because the
number of membership associations was unbound. As a result, cgroup v1
grew several controllers whose primary purpose is either tagging
membership or pull in configuration knobs from other subsystems so
that cgroup membership test can be avoided.
net_cls and net_prio controllers are examples of the latter. They
allow configuring network-specific attributes from cgroup side so that
network subsystem can avoid testing cgroup membership; unfortunately,
these are not only cumbersome but also problematic.
Both net_cls and net_prio aren't properly hierarchical. Both inherit
configuration from the parent on creation but there's no interaction
afterwards. An ancestor doesn't restrict the behavior in its subtree
in anyway and configuration changes aren't propagated downwards.
Especially when combined with cgroup delegation, this is problematic
because delegatees can mess up whatever network configuration
implemented at the system level. net_prio would allow the delegatees
to set whatever priority value regardless of CAP_NET_ADMIN and net_cls
the same for classid.
While it is possible to solve these issues from controller side by
implementing hierarchical allowable ranges in both controllers, it
would involve quite a bit of complexity in the controllers and further
obfuscate network configuration as it becomes even more difficult to
tell what's actually being configured looking from the network side.
While not much can be done for v1 at this point, as membership
handling is sane on cgroup v2, it'd be better to make cgroup matching
behave like other network matches and classifiers than introducing
further complications.
In preparation, this patch updates sock->sk_cgrp_data handling so that
it points to the v2 cgroup that sock was created in until either
net_prio or net_cls is used. Once either of the two is used,
sock->sk_cgrp_data reverts to its previous role of carrying prioidx
and classid. This is to avoid adding yet another cgroup related field
to struct sock.
As the mode switching can happen at most once per boot, the switching
mechanism is aimed at lowering hot path overhead. It may leak a
finite, likely small, number of cgroup refs and report spurious
prioidx or classid on switching; however, dynamic updates of prioidx
and classid have always been racy and lossy - socks between creation
and fd installation are never updated, config changes don't update
existing sockets at all, and prioidx may index with dead and recycled
cgroup IDs. Non-critical inaccuracies from small race windows won't
make any noticeable difference.
This patch doesn't make use of the pointer yet. The following patch
will implement netfilter match for cgroup2 membership.
v2: Use sock_cgroup_data to avoid inflating struct sock w/ another
cgroup specific field.
v3: Add comments explaining why sock_data_prioidx() and
sock_data_classid() use different fallback values.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Daniel Wagner <daniel.wagner@bmw-carit.de>
CC: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2015-12-07 22:38:53 +00:00
|
|
|
/*
|
|
|
|
* sock_cgroup_data is embedded at sock->sk_cgrp_data and contains
|
|
|
|
* per-socket cgroup information except for memcg association.
|
|
|
|
*
|
bpf, cgroups: Fix cgroup v2 fallback on v1/v2 mixed mode
Fix cgroup v1 interference when non-root cgroup v2 BPF programs are used.
Back in the days, commit bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
embedded per-socket cgroup information into sock->sk_cgrp_data and in order
to save 8 bytes in struct sock made both mutually exclusive, that is, when
cgroup v1 socket tagging (e.g. net_cls/net_prio) is used, then cgroup v2
falls back to the root cgroup in sock_cgroup_ptr() (&cgrp_dfl_root.cgrp).
The assumption made was "there is no reason to mix the two and this is in line
with how legacy and v2 compatibility is handled" as stated in bd1060a1d671.
However, with Kubernetes more widely supporting cgroups v2 as well nowadays,
this assumption no longer holds, and the possibility of the v1/v2 mixed mode
with the v2 root fallback being hit becomes a real security issue.
Many of the cgroup v2 BPF programs are also used for policy enforcement, just
to pick _one_ example, that is, to programmatically deny socket related system
calls like connect(2) or bind(2). A v2 root fallback would implicitly cause
a policy bypass for the affected Pods.
In production environments, we have recently seen this case due to various
circumstances: i) a different 3rd party agent and/or ii) a container runtime
such as [0] in the user's environment configuring legacy cgroup v1 net_cls
tags, which triggered implicitly mentioned root fallback. Another case is
Kubernetes projects like kind [1] which create Kubernetes nodes in a container
and also add cgroup namespaces to the mix, meaning programs which are attached
to the cgroup v2 root of the cgroup namespace get attached to a non-root
cgroup v2 path from init namespace point of view. And the latter's root is
out of reach for agents on a kind Kubernetes node to configure. Meaning, any
entity on the node setting cgroup v1 net_cls tag will trigger the bypass
despite cgroup v2 BPF programs attached to the namespace root.
Generally, this mutual exclusiveness does not hold anymore in today's user
environments and makes cgroup v2 usage from BPF side fragile and unreliable.
This fix adds proper struct cgroup pointer for the cgroup v2 case to struct
sock_cgroup_data in order to address these issues; this implicitly also fixes
the tradeoffs being made back then with regards to races and refcount leaks
as stated in bd1060a1d671, and removes the fallback, so that cgroup v2 BPF
programs always operate as expected.
[0] https://github.com/nestybox/sysbox/
[1] https://kind.sigs.k8s.io/
Fixes: bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Stanislav Fomichev <sdf@google.com>
Acked-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/bpf/20210913230759.2313-1-daniel@iogearbox.net
2021-09-13 23:07:57 +00:00
|
|
|
* On legacy hierarchies, net_prio and net_cls controllers directly
|
|
|
|
* set attributes on each sock which can then be tested by the network
|
|
|
|
* layer. On the default hierarchy, each sock is associated with the
|
|
|
|
* cgroup it was created in and the networking layer can match the
|
|
|
|
* cgroup directly.
|
sock, cgroup: add sock->sk_cgroup
In cgroup v1, dealing with cgroup membership was difficult because the
number of membership associations was unbound. As a result, cgroup v1
grew several controllers whose primary purpose is either tagging
membership or pull in configuration knobs from other subsystems so
that cgroup membership test can be avoided.
net_cls and net_prio controllers are examples of the latter. They
allow configuring network-specific attributes from cgroup side so that
network subsystem can avoid testing cgroup membership; unfortunately,
these are not only cumbersome but also problematic.
Both net_cls and net_prio aren't properly hierarchical. Both inherit
configuration from the parent on creation but there's no interaction
afterwards. An ancestor doesn't restrict the behavior in its subtree
in anyway and configuration changes aren't propagated downwards.
Especially when combined with cgroup delegation, this is problematic
because delegatees can mess up whatever network configuration
implemented at the system level. net_prio would allow the delegatees
to set whatever priority value regardless of CAP_NET_ADMIN and net_cls
the same for classid.
While it is possible to solve these issues from controller side by
implementing hierarchical allowable ranges in both controllers, it
would involve quite a bit of complexity in the controllers and further
obfuscate network configuration as it becomes even more difficult to
tell what's actually being configured looking from the network side.
While not much can be done for v1 at this point, as membership
handling is sane on cgroup v2, it'd be better to make cgroup matching
behave like other network matches and classifiers than introducing
further complications.
In preparation, this patch updates sock->sk_cgrp_data handling so that
it points to the v2 cgroup that sock was created in until either
net_prio or net_cls is used. Once either of the two is used,
sock->sk_cgrp_data reverts to its previous role of carrying prioidx
and classid. This is to avoid adding yet another cgroup related field
to struct sock.
As the mode switching can happen at most once per boot, the switching
mechanism is aimed at lowering hot path overhead. It may leak a
finite, likely small, number of cgroup refs and report spurious
prioidx or classid on switching; however, dynamic updates of prioidx
and classid have always been racy and lossy - socks between creation
and fd installation are never updated, config changes don't update
existing sockets at all, and prioidx may index with dead and recycled
cgroup IDs. Non-critical inaccuracies from small race windows won't
make any noticeable difference.
This patch doesn't make use of the pointer yet. The following patch
will implement netfilter match for cgroup2 membership.
v2: Use sock_cgroup_data to avoid inflating struct sock w/ another
cgroup specific field.
v3: Add comments explaining why sock_data_prioidx() and
sock_data_classid() use different fallback values.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Daniel Wagner <daniel.wagner@bmw-carit.de>
CC: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2015-12-07 22:38:53 +00:00
|
|
|
*/
|
2015-12-07 22:38:52 +00:00
|
|
|
struct sock_cgroup_data {
|
bpf, cgroups: Fix cgroup v2 fallback on v1/v2 mixed mode
Fix cgroup v1 interference when non-root cgroup v2 BPF programs are used.
Back in the days, commit bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
embedded per-socket cgroup information into sock->sk_cgrp_data and in order
to save 8 bytes in struct sock made both mutually exclusive, that is, when
cgroup v1 socket tagging (e.g. net_cls/net_prio) is used, then cgroup v2
falls back to the root cgroup in sock_cgroup_ptr() (&cgrp_dfl_root.cgrp).
The assumption made was "there is no reason to mix the two and this is in line
with how legacy and v2 compatibility is handled" as stated in bd1060a1d671.
However, with Kubernetes more widely supporting cgroups v2 as well nowadays,
this assumption no longer holds, and the possibility of the v1/v2 mixed mode
with the v2 root fallback being hit becomes a real security issue.
Many of the cgroup v2 BPF programs are also used for policy enforcement, just
to pick _one_ example, that is, to programmatically deny socket related system
calls like connect(2) or bind(2). A v2 root fallback would implicitly cause
a policy bypass for the affected Pods.
In production environments, we have recently seen this case due to various
circumstances: i) a different 3rd party agent and/or ii) a container runtime
such as [0] in the user's environment configuring legacy cgroup v1 net_cls
tags, which triggered implicitly mentioned root fallback. Another case is
Kubernetes projects like kind [1] which create Kubernetes nodes in a container
and also add cgroup namespaces to the mix, meaning programs which are attached
to the cgroup v2 root of the cgroup namespace get attached to a non-root
cgroup v2 path from init namespace point of view. And the latter's root is
out of reach for agents on a kind Kubernetes node to configure. Meaning, any
entity on the node setting cgroup v1 net_cls tag will trigger the bypass
despite cgroup v2 BPF programs attached to the namespace root.
Generally, this mutual exclusiveness does not hold anymore in today's user
environments and makes cgroup v2 usage from BPF side fragile and unreliable.
This fix adds proper struct cgroup pointer for the cgroup v2 case to struct
sock_cgroup_data in order to address these issues; this implicitly also fixes
the tradeoffs being made back then with regards to races and refcount leaks
as stated in bd1060a1d671, and removes the fallback, so that cgroup v2 BPF
programs always operate as expected.
[0] https://github.com/nestybox/sysbox/
[1] https://kind.sigs.k8s.io/
Fixes: bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Stanislav Fomichev <sdf@google.com>
Acked-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/bpf/20210913230759.2313-1-daniel@iogearbox.net
2021-09-13 23:07:57 +00:00
|
|
|
struct cgroup *cgroup; /* v2 */
|
|
|
|
#ifdef CONFIG_CGROUP_NET_CLASSID
|
|
|
|
u32 classid; /* v1 */
|
|
|
|
#endif
|
|
|
|
#ifdef CONFIG_CGROUP_NET_PRIO
|
|
|
|
u16 prioidx; /* v1 */
|
sock, cgroup: add sock->sk_cgroup
In cgroup v1, dealing with cgroup membership was difficult because the
number of membership associations was unbound. As a result, cgroup v1
grew several controllers whose primary purpose is either tagging
membership or pull in configuration knobs from other subsystems so
that cgroup membership test can be avoided.
net_cls and net_prio controllers are examples of the latter. They
allow configuring network-specific attributes from cgroup side so that
network subsystem can avoid testing cgroup membership; unfortunately,
these are not only cumbersome but also problematic.
Both net_cls and net_prio aren't properly hierarchical. Both inherit
configuration from the parent on creation but there's no interaction
afterwards. An ancestor doesn't restrict the behavior in its subtree
in anyway and configuration changes aren't propagated downwards.
Especially when combined with cgroup delegation, this is problematic
because delegatees can mess up whatever network configuration
implemented at the system level. net_prio would allow the delegatees
to set whatever priority value regardless of CAP_NET_ADMIN and net_cls
the same for classid.
While it is possible to solve these issues from controller side by
implementing hierarchical allowable ranges in both controllers, it
would involve quite a bit of complexity in the controllers and further
obfuscate network configuration as it becomes even more difficult to
tell what's actually being configured looking from the network side.
While not much can be done for v1 at this point, as membership
handling is sane on cgroup v2, it'd be better to make cgroup matching
behave like other network matches and classifiers than introducing
further complications.
In preparation, this patch updates sock->sk_cgrp_data handling so that
it points to the v2 cgroup that sock was created in until either
net_prio or net_cls is used. Once either of the two is used,
sock->sk_cgrp_data reverts to its previous role of carrying prioidx
and classid. This is to avoid adding yet another cgroup related field
to struct sock.
As the mode switching can happen at most once per boot, the switching
mechanism is aimed at lowering hot path overhead. It may leak a
finite, likely small, number of cgroup refs and report spurious
prioidx or classid on switching; however, dynamic updates of prioidx
and classid have always been racy and lossy - socks between creation
and fd installation are never updated, config changes don't update
existing sockets at all, and prioidx may index with dead and recycled
cgroup IDs. Non-critical inaccuracies from small race windows won't
make any noticeable difference.
This patch doesn't make use of the pointer yet. The following patch
will implement netfilter match for cgroup2 membership.
v2: Use sock_cgroup_data to avoid inflating struct sock w/ another
cgroup specific field.
v3: Add comments explaining why sock_data_prioidx() and
sock_data_classid() use different fallback values.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Daniel Wagner <daniel.wagner@bmw-carit.de>
CC: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2015-12-07 22:38:53 +00:00
|
|
|
#endif
|
2015-12-07 22:38:52 +00:00
|
|
|
};
|
|
|
|
|
2018-03-14 16:04:16 +00:00
|
|
|
static inline u16 sock_cgroup_prioidx(const struct sock_cgroup_data *skcd)
|
2015-12-07 22:38:52 +00:00
|
|
|
{
|
bpf, cgroups: Fix cgroup v2 fallback on v1/v2 mixed mode
Fix cgroup v1 interference when non-root cgroup v2 BPF programs are used.
Back in the days, commit bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
embedded per-socket cgroup information into sock->sk_cgrp_data and in order
to save 8 bytes in struct sock made both mutually exclusive, that is, when
cgroup v1 socket tagging (e.g. net_cls/net_prio) is used, then cgroup v2
falls back to the root cgroup in sock_cgroup_ptr() (&cgrp_dfl_root.cgrp).
The assumption made was "there is no reason to mix the two and this is in line
with how legacy and v2 compatibility is handled" as stated in bd1060a1d671.
However, with Kubernetes more widely supporting cgroups v2 as well nowadays,
this assumption no longer holds, and the possibility of the v1/v2 mixed mode
with the v2 root fallback being hit becomes a real security issue.
Many of the cgroup v2 BPF programs are also used for policy enforcement, just
to pick _one_ example, that is, to programmatically deny socket related system
calls like connect(2) or bind(2). A v2 root fallback would implicitly cause
a policy bypass for the affected Pods.
In production environments, we have recently seen this case due to various
circumstances: i) a different 3rd party agent and/or ii) a container runtime
such as [0] in the user's environment configuring legacy cgroup v1 net_cls
tags, which triggered implicitly mentioned root fallback. Another case is
Kubernetes projects like kind [1] which create Kubernetes nodes in a container
and also add cgroup namespaces to the mix, meaning programs which are attached
to the cgroup v2 root of the cgroup namespace get attached to a non-root
cgroup v2 path from init namespace point of view. And the latter's root is
out of reach for agents on a kind Kubernetes node to configure. Meaning, any
entity on the node setting cgroup v1 net_cls tag will trigger the bypass
despite cgroup v2 BPF programs attached to the namespace root.
Generally, this mutual exclusiveness does not hold anymore in today's user
environments and makes cgroup v2 usage from BPF side fragile and unreliable.
This fix adds proper struct cgroup pointer for the cgroup v2 case to struct
sock_cgroup_data in order to address these issues; this implicitly also fixes
the tradeoffs being made back then with regards to races and refcount leaks
as stated in bd1060a1d671, and removes the fallback, so that cgroup v2 BPF
programs always operate as expected.
[0] https://github.com/nestybox/sysbox/
[1] https://kind.sigs.k8s.io/
Fixes: bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Stanislav Fomichev <sdf@google.com>
Acked-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/bpf/20210913230759.2313-1-daniel@iogearbox.net
2021-09-13 23:07:57 +00:00
|
|
|
#ifdef CONFIG_CGROUP_NET_PRIO
|
|
|
|
return READ_ONCE(skcd->prioidx);
|
|
|
|
#else
|
|
|
|
return 1;
|
|
|
|
#endif
|
2015-12-07 22:38:52 +00:00
|
|
|
}
|
|
|
|
|
2018-03-14 16:04:16 +00:00
|
|
|
static inline u32 sock_cgroup_classid(const struct sock_cgroup_data *skcd)
|
2015-12-07 22:38:52 +00:00
|
|
|
{
|
bpf, cgroups: Fix cgroup v2 fallback on v1/v2 mixed mode
Fix cgroup v1 interference when non-root cgroup v2 BPF programs are used.
Back in the days, commit bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
embedded per-socket cgroup information into sock->sk_cgrp_data and in order
to save 8 bytes in struct sock made both mutually exclusive, that is, when
cgroup v1 socket tagging (e.g. net_cls/net_prio) is used, then cgroup v2
falls back to the root cgroup in sock_cgroup_ptr() (&cgrp_dfl_root.cgrp).
The assumption made was "there is no reason to mix the two and this is in line
with how legacy and v2 compatibility is handled" as stated in bd1060a1d671.
However, with Kubernetes more widely supporting cgroups v2 as well nowadays,
this assumption no longer holds, and the possibility of the v1/v2 mixed mode
with the v2 root fallback being hit becomes a real security issue.
Many of the cgroup v2 BPF programs are also used for policy enforcement, just
to pick _one_ example, that is, to programmatically deny socket related system
calls like connect(2) or bind(2). A v2 root fallback would implicitly cause
a policy bypass for the affected Pods.
In production environments, we have recently seen this case due to various
circumstances: i) a different 3rd party agent and/or ii) a container runtime
such as [0] in the user's environment configuring legacy cgroup v1 net_cls
tags, which triggered implicitly mentioned root fallback. Another case is
Kubernetes projects like kind [1] which create Kubernetes nodes in a container
and also add cgroup namespaces to the mix, meaning programs which are attached
to the cgroup v2 root of the cgroup namespace get attached to a non-root
cgroup v2 path from init namespace point of view. And the latter's root is
out of reach for agents on a kind Kubernetes node to configure. Meaning, any
entity on the node setting cgroup v1 net_cls tag will trigger the bypass
despite cgroup v2 BPF programs attached to the namespace root.
Generally, this mutual exclusiveness does not hold anymore in today's user
environments and makes cgroup v2 usage from BPF side fragile and unreliable.
This fix adds proper struct cgroup pointer for the cgroup v2 case to struct
sock_cgroup_data in order to address these issues; this implicitly also fixes
the tradeoffs being made back then with regards to races and refcount leaks
as stated in bd1060a1d671, and removes the fallback, so that cgroup v2 BPF
programs always operate as expected.
[0] https://github.com/nestybox/sysbox/
[1] https://kind.sigs.k8s.io/
Fixes: bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Stanislav Fomichev <sdf@google.com>
Acked-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/bpf/20210913230759.2313-1-daniel@iogearbox.net
2021-09-13 23:07:57 +00:00
|
|
|
#ifdef CONFIG_CGROUP_NET_CLASSID
|
|
|
|
return READ_ONCE(skcd->classid);
|
|
|
|
#else
|
|
|
|
return 0;
|
|
|
|
#endif
|
2015-12-07 22:38:52 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static inline void sock_cgroup_set_prioidx(struct sock_cgroup_data *skcd,
|
|
|
|
u16 prioidx)
|
|
|
|
{
|
bpf, cgroups: Fix cgroup v2 fallback on v1/v2 mixed mode
Fix cgroup v1 interference when non-root cgroup v2 BPF programs are used.
Back in the days, commit bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
embedded per-socket cgroup information into sock->sk_cgrp_data and in order
to save 8 bytes in struct sock made both mutually exclusive, that is, when
cgroup v1 socket tagging (e.g. net_cls/net_prio) is used, then cgroup v2
falls back to the root cgroup in sock_cgroup_ptr() (&cgrp_dfl_root.cgrp).
The assumption made was "there is no reason to mix the two and this is in line
with how legacy and v2 compatibility is handled" as stated in bd1060a1d671.
However, with Kubernetes more widely supporting cgroups v2 as well nowadays,
this assumption no longer holds, and the possibility of the v1/v2 mixed mode
with the v2 root fallback being hit becomes a real security issue.
Many of the cgroup v2 BPF programs are also used for policy enforcement, just
to pick _one_ example, that is, to programmatically deny socket related system
calls like connect(2) or bind(2). A v2 root fallback would implicitly cause
a policy bypass for the affected Pods.
In production environments, we have recently seen this case due to various
circumstances: i) a different 3rd party agent and/or ii) a container runtime
such as [0] in the user's environment configuring legacy cgroup v1 net_cls
tags, which triggered implicitly mentioned root fallback. Another case is
Kubernetes projects like kind [1] which create Kubernetes nodes in a container
and also add cgroup namespaces to the mix, meaning programs which are attached
to the cgroup v2 root of the cgroup namespace get attached to a non-root
cgroup v2 path from init namespace point of view. And the latter's root is
out of reach for agents on a kind Kubernetes node to configure. Meaning, any
entity on the node setting cgroup v1 net_cls tag will trigger the bypass
despite cgroup v2 BPF programs attached to the namespace root.
Generally, this mutual exclusiveness does not hold anymore in today's user
environments and makes cgroup v2 usage from BPF side fragile and unreliable.
This fix adds proper struct cgroup pointer for the cgroup v2 case to struct
sock_cgroup_data in order to address these issues; this implicitly also fixes
the tradeoffs being made back then with regards to races and refcount leaks
as stated in bd1060a1d671, and removes the fallback, so that cgroup v2 BPF
programs always operate as expected.
[0] https://github.com/nestybox/sysbox/
[1] https://kind.sigs.k8s.io/
Fixes: bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Stanislav Fomichev <sdf@google.com>
Acked-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/bpf/20210913230759.2313-1-daniel@iogearbox.net
2021-09-13 23:07:57 +00:00
|
|
|
#ifdef CONFIG_CGROUP_NET_PRIO
|
|
|
|
WRITE_ONCE(skcd->prioidx, prioidx);
|
|
|
|
#endif
|
2015-12-07 22:38:52 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static inline void sock_cgroup_set_classid(struct sock_cgroup_data *skcd,
|
|
|
|
u32 classid)
|
|
|
|
{
|
bpf, cgroups: Fix cgroup v2 fallback on v1/v2 mixed mode
Fix cgroup v1 interference when non-root cgroup v2 BPF programs are used.
Back in the days, commit bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
embedded per-socket cgroup information into sock->sk_cgrp_data and in order
to save 8 bytes in struct sock made both mutually exclusive, that is, when
cgroup v1 socket tagging (e.g. net_cls/net_prio) is used, then cgroup v2
falls back to the root cgroup in sock_cgroup_ptr() (&cgrp_dfl_root.cgrp).
The assumption made was "there is no reason to mix the two and this is in line
with how legacy and v2 compatibility is handled" as stated in bd1060a1d671.
However, with Kubernetes more widely supporting cgroups v2 as well nowadays,
this assumption no longer holds, and the possibility of the v1/v2 mixed mode
with the v2 root fallback being hit becomes a real security issue.
Many of the cgroup v2 BPF programs are also used for policy enforcement, just
to pick _one_ example, that is, to programmatically deny socket related system
calls like connect(2) or bind(2). A v2 root fallback would implicitly cause
a policy bypass for the affected Pods.
In production environments, we have recently seen this case due to various
circumstances: i) a different 3rd party agent and/or ii) a container runtime
such as [0] in the user's environment configuring legacy cgroup v1 net_cls
tags, which triggered implicitly mentioned root fallback. Another case is
Kubernetes projects like kind [1] which create Kubernetes nodes in a container
and also add cgroup namespaces to the mix, meaning programs which are attached
to the cgroup v2 root of the cgroup namespace get attached to a non-root
cgroup v2 path from init namespace point of view. And the latter's root is
out of reach for agents on a kind Kubernetes node to configure. Meaning, any
entity on the node setting cgroup v1 net_cls tag will trigger the bypass
despite cgroup v2 BPF programs attached to the namespace root.
Generally, this mutual exclusiveness does not hold anymore in today's user
environments and makes cgroup v2 usage from BPF side fragile and unreliable.
This fix adds proper struct cgroup pointer for the cgroup v2 case to struct
sock_cgroup_data in order to address these issues; this implicitly also fixes
the tradeoffs being made back then with regards to races and refcount leaks
as stated in bd1060a1d671, and removes the fallback, so that cgroup v2 BPF
programs always operate as expected.
[0] https://github.com/nestybox/sysbox/
[1] https://kind.sigs.k8s.io/
Fixes: bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Stanislav Fomichev <sdf@google.com>
Acked-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/bpf/20210913230759.2313-1-daniel@iogearbox.net
2021-09-13 23:07:57 +00:00
|
|
|
#ifdef CONFIG_CGROUP_NET_CLASSID
|
|
|
|
WRITE_ONCE(skcd->classid, classid);
|
|
|
|
#endif
|
2015-12-07 22:38:52 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
#else /* CONFIG_SOCK_CGROUP_DATA */
|
|
|
|
|
|
|
|
struct sock_cgroup_data {
|
|
|
|
};
|
|
|
|
|
|
|
|
#endif /* CONFIG_SOCK_CGROUP_DATA */
|
|
|
|
|
2015-05-13 19:38:40 +00:00
|
|
|
#endif /* _LINUX_CGROUP_DEFS_H */
|