lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
#ifndef LLIST_H
|
|
|
|
#define LLIST_H
|
|
|
|
/*
|
|
|
|
* Lock-less NULL terminated single linked list
|
|
|
|
*
|
2016-12-13 02:01:45 +00:00
|
|
|
* Cases where locking is not needed:
|
|
|
|
* If there are multiple producers and multiple consumers, llist_add can be
|
|
|
|
* used in producers and llist_del_all can be used in consumers simultaneously
|
|
|
|
* without locking. Also a single consumer can use llist_del_first while
|
|
|
|
* multiple producers simultaneously use llist_add, without any locking.
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
*
|
2016-12-13 02:01:45 +00:00
|
|
|
* Cases where locking is needed:
|
|
|
|
* If we have multiple consumers with llist_del_first used in one consumer, and
|
|
|
|
* llist_del_first or llist_del_all used in other consumers, then a lock is
|
|
|
|
* needed. This is because llist_del_first depends on list->first->next not
|
|
|
|
* changing, but without lock protection, there's no way to be sure about that
|
|
|
|
* if a preemption happens in the middle of the delete operation and on being
|
|
|
|
* preempted back, the list->first is the same as before causing the cmpxchg in
|
|
|
|
* llist_del_first to succeed. For example, while a llist_del_first operation
|
|
|
|
* is in progress in one consumer, then a llist_del_first, llist_add,
|
|
|
|
* llist_add (or llist_del_all, llist_add, llist_add) sequence in another
|
|
|
|
* consumer may cause violations.
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
*
|
2016-12-13 02:01:45 +00:00
|
|
|
* This can be summarized as follows:
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
*
|
|
|
|
* | add | del_first | del_all
|
|
|
|
* add | - | - | -
|
|
|
|
* del_first | | L | L
|
|
|
|
* del_all | | | -
|
|
|
|
*
|
2016-12-13 02:01:45 +00:00
|
|
|
* Where, a particular row's operation can happen concurrently with a column's
|
|
|
|
* operation, with "-" being no lock needed, while "L" being lock is needed.
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
*
|
|
|
|
* The list entries deleted via llist_del_all can be traversed with
|
|
|
|
* traversing function such as llist_for_each etc. But the list
|
|
|
|
* entries can not be traversed safely before deleted from the list.
|
|
|
|
* The order of deleted entries is from the newest to the oldest added
|
|
|
|
* one. If you want to traverse from the oldest to the newest, you
|
|
|
|
* must reverse the order by yourself before traversing.
|
|
|
|
*
|
|
|
|
* The basic atomic operation of this list is cmpxchg on long. On
|
|
|
|
* architectures that don't have NMI-safe cmpxchg implementation, the
|
2011-10-04 10:43:11 +00:00
|
|
|
* list can NOT be used in NMI handlers. So code that uses the list in
|
|
|
|
* an NMI handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
|
2011-09-08 06:00:42 +00:00
|
|
|
*
|
|
|
|
* Copyright 2010,2011 Intel Corp.
|
|
|
|
* Author: Huang Ying <ying.huang@intel.com>
|
|
|
|
*
|
|
|
|
* This program is free software; you can redistribute it and/or
|
|
|
|
* modify it under the terms of the GNU General Public License version
|
|
|
|
* 2 as published by the Free Software Foundation;
|
|
|
|
*
|
|
|
|
* This program is distributed in the hope that it will be useful,
|
|
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
* GNU General Public License for more details.
|
|
|
|
*
|
|
|
|
* You should have received a copy of the GNU General Public License
|
|
|
|
* along with this program; if not, write to the Free Software
|
|
|
|
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
*/
|
|
|
|
|
2015-08-06 16:54:43 +00:00
|
|
|
#include <linux/atomic.h>
|
2011-09-08 06:00:42 +00:00
|
|
|
#include <linux/kernel.h>
|
|
|
|
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
struct llist_head {
|
|
|
|
struct llist_node *first;
|
|
|
|
};
|
|
|
|
|
|
|
|
struct llist_node {
|
|
|
|
struct llist_node *next;
|
|
|
|
};
|
|
|
|
|
|
|
|
#define LLIST_HEAD_INIT(name) { NULL }
|
|
|
|
#define LLIST_HEAD(name) struct llist_head name = LLIST_HEAD_INIT(name)
|
|
|
|
|
|
|
|
/**
|
|
|
|
* init_llist_head - initialize lock-less list head
|
|
|
|
* @head: the head for your lock-less list
|
|
|
|
*/
|
|
|
|
static inline void init_llist_head(struct llist_head *list)
|
|
|
|
{
|
|
|
|
list->first = NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* llist_entry - get the struct of this entry
|
|
|
|
* @ptr: the &struct llist_node pointer.
|
|
|
|
* @type: the type of the struct this is embedded in.
|
|
|
|
* @member: the name of the llist_node within the struct.
|
|
|
|
*/
|
|
|
|
#define llist_entry(ptr, type, member) \
|
|
|
|
container_of(ptr, type, member)
|
|
|
|
|
2017-07-19 18:27:30 +00:00
|
|
|
/**
|
|
|
|
* member_address_is_nonnull - check whether the member address is not NULL
|
|
|
|
* @ptr: the object pointer (struct type * that contains the llist_node)
|
|
|
|
* @member: the name of the llist_node within the struct.
|
|
|
|
*
|
|
|
|
* This macro is conceptually the same as
|
|
|
|
* &ptr->member != NULL
|
|
|
|
* but it works around the fact that compilers can decide that taking a member
|
|
|
|
* address is never a NULL pointer.
|
|
|
|
*
|
|
|
|
* Real objects that start at a high address and have a member at NULL are
|
|
|
|
* unlikely to exist, but such pointers may be returned e.g. by the
|
|
|
|
* container_of() macro.
|
|
|
|
*/
|
|
|
|
#define member_address_is_nonnull(ptr, member) \
|
|
|
|
((uintptr_t)(ptr) + offsetof(typeof(*(ptr)), member) != 0)
|
|
|
|
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
/**
|
|
|
|
* llist_for_each - iterate over some deleted entries of a lock-less list
|
|
|
|
* @pos: the &struct llist_node to use as a loop cursor
|
|
|
|
* @node: the first entry of deleted list entries
|
|
|
|
*
|
|
|
|
* In general, some entries of the lock-less list can be traversed
|
|
|
|
* safely only after being deleted from list, so start with an entry
|
|
|
|
* instead of list head.
|
|
|
|
*
|
|
|
|
* If being used on entries deleted from lock-less list directly, the
|
|
|
|
* traverse order is from the newest to the oldest added entry. If
|
|
|
|
* you want to traverse from the oldest to the newest, you must
|
|
|
|
* reverse the order by yourself before traversing.
|
|
|
|
*/
|
|
|
|
#define llist_for_each(pos, node) \
|
|
|
|
for ((pos) = (node); pos; (pos) = (pos)->next)
|
|
|
|
|
2017-05-12 00:36:56 +00:00
|
|
|
/**
|
|
|
|
* llist_for_each_safe - iterate over some deleted entries of a lock-less list
|
|
|
|
* safe against removal of list entry
|
|
|
|
* @pos: the &struct llist_node to use as a loop cursor
|
|
|
|
* @n: another &struct llist_node to use as temporary storage
|
|
|
|
* @node: the first entry of deleted list entries
|
|
|
|
*
|
|
|
|
* In general, some entries of the lock-less list can be traversed
|
|
|
|
* safely only after being deleted from list, so start with an entry
|
|
|
|
* instead of list head.
|
|
|
|
*
|
|
|
|
* If being used on entries deleted from lock-less list directly, the
|
|
|
|
* traverse order is from the newest to the oldest added entry. If
|
|
|
|
* you want to traverse from the oldest to the newest, you must
|
|
|
|
* reverse the order by yourself before traversing.
|
|
|
|
*/
|
|
|
|
#define llist_for_each_safe(pos, n, node) \
|
|
|
|
for ((pos) = (node); (pos) && ((n) = (pos)->next, true); (pos) = (n))
|
|
|
|
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
/**
|
|
|
|
* llist_for_each_entry - iterate over some deleted entries of lock-less list of given type
|
|
|
|
* @pos: the type * to use as a loop cursor.
|
|
|
|
* @node: the fist entry of deleted list entries.
|
|
|
|
* @member: the name of the llist_node with the struct.
|
|
|
|
*
|
|
|
|
* In general, some entries of the lock-less list can be traversed
|
|
|
|
* safely only after being removed from list, so start with an entry
|
|
|
|
* instead of list head.
|
|
|
|
*
|
|
|
|
* If being used on entries deleted from lock-less list directly, the
|
|
|
|
* traverse order is from the newest to the oldest added entry. If
|
|
|
|
* you want to traverse from the oldest to the newest, you must
|
|
|
|
* reverse the order by yourself before traversing.
|
|
|
|
*/
|
|
|
|
#define llist_for_each_entry(pos, node, member) \
|
|
|
|
for ((pos) = llist_entry((node), typeof(*(pos)), member); \
|
2017-07-19 18:27:30 +00:00
|
|
|
member_address_is_nonnull(pos, member); \
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
(pos) = llist_entry((pos)->member.next, typeof(*(pos)), member))
|
|
|
|
|
2013-06-15 13:36:06 +00:00
|
|
|
/**
|
|
|
|
* llist_for_each_entry_safe - iterate over some deleted entries of lock-less list of given type
|
|
|
|
* safe against removal of list entry
|
|
|
|
* @pos: the type * to use as a loop cursor.
|
|
|
|
* @n: another type * to use as temporary storage
|
|
|
|
* @node: the first entry of deleted list entries.
|
|
|
|
* @member: the name of the llist_node with the struct.
|
|
|
|
*
|
|
|
|
* In general, some entries of the lock-less list can be traversed
|
|
|
|
* safely only after being removed from list, so start with an entry
|
|
|
|
* instead of list head.
|
|
|
|
*
|
|
|
|
* If being used on entries deleted from lock-less list directly, the
|
|
|
|
* traverse order is from the newest to the oldest added entry. If
|
|
|
|
* you want to traverse from the oldest to the newest, you must
|
|
|
|
* reverse the order by yourself before traversing.
|
|
|
|
*/
|
|
|
|
#define llist_for_each_entry_safe(pos, n, node, member) \
|
|
|
|
for (pos = llist_entry((node), typeof(*pos), member); \
|
2017-07-19 18:27:30 +00:00
|
|
|
member_address_is_nonnull(pos, member) && \
|
2013-06-15 13:36:06 +00:00
|
|
|
(n = llist_entry(pos->member.next, typeof(*n), member), true); \
|
|
|
|
pos = n)
|
|
|
|
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
/**
|
|
|
|
* llist_empty - tests whether a lock-less list is empty
|
|
|
|
* @head: the list to test
|
|
|
|
*
|
|
|
|
* Not guaranteed to be accurate or up to date. Just a quick way to
|
|
|
|
* test whether the list is empty without deleting something from the
|
|
|
|
* list.
|
|
|
|
*/
|
2011-09-08 06:00:42 +00:00
|
|
|
static inline bool llist_empty(const struct llist_head *head)
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
{
|
|
|
|
return ACCESS_ONCE(head->first) == NULL;
|
|
|
|
}
|
|
|
|
|
2011-09-12 11:12:28 +00:00
|
|
|
static inline struct llist_node *llist_next(struct llist_node *node)
|
|
|
|
{
|
|
|
|
return node->next;
|
|
|
|
}
|
|
|
|
|
2013-07-08 21:24:19 +00:00
|
|
|
extern bool llist_add_batch(struct llist_node *new_first,
|
|
|
|
struct llist_node *new_last,
|
|
|
|
struct llist_head *head);
|
2011-09-08 06:00:42 +00:00
|
|
|
/**
|
|
|
|
* llist_add - add a new entry
|
|
|
|
* @new: new entry to be added
|
|
|
|
* @head: the head for your lock-less list
|
2011-09-08 06:00:45 +00:00
|
|
|
*
|
2011-11-01 00:13:08 +00:00
|
|
|
* Returns true if the list was empty prior to adding this entry.
|
2011-09-08 06:00:42 +00:00
|
|
|
*/
|
2011-09-08 06:00:45 +00:00
|
|
|
static inline bool llist_add(struct llist_node *new, struct llist_head *head)
|
2011-09-08 06:00:42 +00:00
|
|
|
{
|
2013-07-08 21:24:19 +00:00
|
|
|
return llist_add_batch(new, new, head);
|
2011-09-08 06:00:42 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* llist_del_all - delete all entries from lock-less list
|
|
|
|
* @head: the head of lock-less list to delete all entries
|
|
|
|
*
|
|
|
|
* If list is empty, return NULL, otherwise, delete all entries and
|
|
|
|
* return the pointer to the first entry. The order of entries
|
|
|
|
* deleted is from the newest to the oldest added one.
|
|
|
|
*/
|
|
|
|
static inline struct llist_node *llist_del_all(struct llist_head *head)
|
|
|
|
{
|
|
|
|
return xchg(&head->first, NULL);
|
|
|
|
}
|
2011-10-05 06:25:28 +00:00
|
|
|
|
|
|
|
extern struct llist_node *llist_del_first(struct llist_head *head);
|
|
|
|
|
2013-11-14 22:32:11 +00:00
|
|
|
struct llist_node *llist_reverse_order(struct llist_node *head);
|
|
|
|
|
lib, Add lock-less NULL terminated single list
Cmpxchg is used to implement adding new entry to the list, deleting
all entries from the list, deleting first entry of the list and some
other operations.
Because this is a single list, so the tail can not be accessed in O(1).
If there are multiple producers and multiple consumers, llist_add can
be used in producers and llist_del_all can be used in consumers. They
can work simultaneously without lock. But llist_del_first can not be
used here. Because llist_del_first depends on list->first->next does
not changed if list->first is not changed during its operation, but
llist_del_first, llist_add, llist_add (or llist_del_all, llist_add,
llist_add) sequence in another consumer may violate that.
If there are multiple producers and one consumer, llist_add can be
used in producers and llist_del_all or llist_del_first can be used in
the consumer.
This can be summarized as follow:
| add | del_first | del_all
add | - | - | -
del_first | | L | L
del_all | | | -
Where "-" stands for no lock is needed, while "L" stands for lock is
needed.
The list entries deleted via llist_del_all can be traversed with
traversing function such as llist_for_each etc. But the list entries
can not be traversed safely before deleted from the list. The order
of deleted entries is from the newest to the oldest added one. If you
want to traverse from the oldest to the newest, you must reverse the
order by yourself before traversing.
The basic atomic operation of this list is cmpxchg on long. On
architectures that don't have NMI-safe cmpxchg implementation, the
list can NOT be used in NMI handler. So code uses the list in NMI
handler should depend on CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Len Brown <len.brown@intel.com>
2011-07-13 05:14:23 +00:00
|
|
|
#endif /* LLIST_H */
|