2019-05-27 06:55:01 +00:00
|
|
|
// SPDX-License-Identifier: GPL-2.0-or-later
|
2005-04-16 22:20:36 +00:00
|
|
|
/*
|
|
|
|
* Linux INET6 implementation
|
|
|
|
* FIB front-end.
|
|
|
|
*
|
|
|
|
* Authors:
|
2007-02-09 14:24:49 +00:00
|
|
|
* Pedro Roque <roque@di.fc.ul.pt>
|
2005-04-16 22:20:36 +00:00
|
|
|
*/
|
|
|
|
|
|
|
|
/* Changes:
|
|
|
|
*
|
|
|
|
* YOSHIFUJI Hideaki @USAGI
|
|
|
|
* reworked default router selection.
|
|
|
|
* - respect outgoing interface
|
|
|
|
* - select from (probably) reachable routers (i.e.
|
|
|
|
* routers in REACHABLE, STALE, DELAY or PROBE states).
|
|
|
|
* - always select the same router if it is (probably)
|
|
|
|
* reachable. otherwise, round-robin the list.
|
2006-08-24 00:23:25 +00:00
|
|
|
* Ville Nuorvala
|
|
|
|
* Fixed routing subtrees.
|
2005-04-16 22:20:36 +00:00
|
|
|
*/
|
|
|
|
|
2012-05-15 14:11:53 +00:00
|
|
|
#define pr_fmt(fmt) "IPv6: " fmt
|
|
|
|
|
2006-01-11 20:17:47 +00:00
|
|
|
#include <linux/capability.h>
|
2005-04-16 22:20:36 +00:00
|
|
|
#include <linux/errno.h>
|
2011-07-15 15:47:34 +00:00
|
|
|
#include <linux/export.h>
|
2005-04-16 22:20:36 +00:00
|
|
|
#include <linux/types.h>
|
|
|
|
#include <linux/times.h>
|
|
|
|
#include <linux/socket.h>
|
|
|
|
#include <linux/sockios.h>
|
|
|
|
#include <linux/net.h>
|
|
|
|
#include <linux/route.h>
|
|
|
|
#include <linux/netdevice.h>
|
|
|
|
#include <linux/in6.h>
|
2008-04-03 00:22:53 +00:00
|
|
|
#include <linux/mroute6.h>
|
2005-04-16 22:20:36 +00:00
|
|
|
#include <linux/init.h>
|
|
|
|
#include <linux/if_arp.h>
|
|
|
|
#include <linux/proc_fs.h>
|
|
|
|
#include <linux/seq_file.h>
|
2008-03-04 07:28:58 +00:00
|
|
|
#include <linux/nsproxy.h>
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 08:04:11 +00:00
|
|
|
#include <linux/slab.h>
|
2017-10-06 19:05:57 +00:00
|
|
|
#include <linux/jhash.h>
|
2021-08-25 23:17:28 +00:00
|
|
|
#include <linux/siphash.h>
|
2007-09-12 10:01:34 +00:00
|
|
|
#include <net/net_namespace.h>
|
2005-04-16 22:20:36 +00:00
|
|
|
#include <net/snmp.h>
|
|
|
|
#include <net/ipv6.h>
|
|
|
|
#include <net/ip6_fib.h>
|
|
|
|
#include <net/ip6_route.h>
|
|
|
|
#include <net/ndisc.h>
|
|
|
|
#include <net/addrconf.h>
|
|
|
|
#include <net/tcp.h>
|
|
|
|
#include <linux/rtnetlink.h>
|
|
|
|
#include <net/dst.h>
|
2015-08-20 11:56:31 +00:00
|
|
|
#include <net/dst_metadata.h>
|
2005-04-16 22:20:36 +00:00
|
|
|
#include <net/xfrm.h>
|
2006-07-31 03:43:36 +00:00
|
|
|
#include <net/netevent.h>
|
2006-08-15 07:35:24 +00:00
|
|
|
#include <net/netlink.h>
|
2019-04-20 16:28:20 +00:00
|
|
|
#include <net/rtnh.h>
|
2015-07-21 08:43:48 +00:00
|
|
|
#include <net/lwtunnel.h>
|
2015-08-20 11:56:31 +00:00
|
|
|
#include <net/ip_tunnels.h>
|
2015-10-12 18:47:10 +00:00
|
|
|
#include <net/l3mdev.h>
|
2018-05-22 21:03:28 +00:00
|
|
|
#include <net/ip.h>
|
2016-12-24 19:46:01 +00:00
|
|
|
#include <linux/uaccess.h>
|
2020-07-20 16:34:03 +00:00
|
|
|
#include <linux/btf_ids.h>
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
#ifdef CONFIG_SYSCTL
|
|
|
|
#include <linux/sysctl.h>
|
|
|
|
#endif
|
|
|
|
|
2018-05-24 00:08:48 +00:00
|
|
|
static int ip6_rt_type_to_error(u8 fib6_type);
|
|
|
|
|
|
|
|
#define CREATE_TRACE_POINTS
|
|
|
|
#include <trace/events/fib6.h>
|
|
|
|
EXPORT_TRACEPOINT_SYMBOL_GPL(fib6_table_lookup);
|
|
|
|
#undef CREATE_TRACE_POINTS
|
|
|
|
|
2013-07-11 10:43:42 +00:00
|
|
|
enum rt6_nud_state {
|
2013-12-11 12:48:20 +00:00
|
|
|
RT6_NUD_FAIL_HARD = -3,
|
|
|
|
RT6_NUD_FAIL_PROBE = -2,
|
|
|
|
RT6_NUD_FAIL_DO_RR = -1,
|
2013-07-11 10:43:42 +00:00
|
|
|
RT6_NUD_SUCCEED = 1
|
|
|
|
};
|
|
|
|
|
2021-02-01 17:41:32 +00:00
|
|
|
INDIRECT_CALLABLE_SCOPE
|
|
|
|
struct dst_entry *ip6_dst_check(struct dst_entry *dst, u32 cookie);
|
2010-12-13 20:52:14 +00:00
|
|
|
static unsigned int ip6_default_advmss(const struct dst_entry *dst);
|
2021-02-01 17:41:31 +00:00
|
|
|
INDIRECT_CALLABLE_SCOPE
|
|
|
|
unsigned int ip6_mtu(const struct dst_entry *dst);
|
2005-04-16 22:20:36 +00:00
|
|
|
static struct dst_entry *ip6_negative_advice(struct dst_entry *);
|
|
|
|
static void ip6_dst_destroy(struct dst_entry *);
|
|
|
|
static void ip6_dst_ifdown(struct dst_entry *,
|
|
|
|
struct net_device *dev, int how);
|
2008-01-18 11:56:57 +00:00
|
|
|
static int ip6_dst_gc(struct dst_ops *ops);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
static int ip6_pkt_discard(struct sk_buff *skb);
|
2015-10-07 21:48:47 +00:00
|
|
|
static int ip6_pkt_discard_out(struct net *net, struct sock *sk, struct sk_buff *skb);
|
2013-12-02 14:25:21 +00:00
|
|
|
static int ip6_pkt_prohibit(struct sk_buff *skb);
|
2015-10-07 21:48:47 +00:00
|
|
|
static int ip6_pkt_prohibit_out(struct net *net, struct sock *sk, struct sk_buff *skb);
|
2005-04-16 22:20:36 +00:00
|
|
|
static void ip6_link_failure(struct sk_buff *skb);
|
2012-07-17 10:29:28 +00:00
|
|
|
static void ip6_rt_update_pmtu(struct dst_entry *dst, struct sock *sk,
|
2019-12-22 02:51:09 +00:00
|
|
|
struct sk_buff *skb, u32 mtu,
|
|
|
|
bool confirm_neigh);
|
2012-07-17 10:29:28 +00:00
|
|
|
static void rt6_do_redirect(struct dst_entry *dst, struct sock *sk,
|
|
|
|
struct sk_buff *skb);
|
2019-04-09 21:41:13 +00:00
|
|
|
static int rt6_score_route(const struct fib6_nh *nh, u32 fib6_flags, int oif,
|
|
|
|
int strict);
|
2019-06-08 21:53:26 +00:00
|
|
|
static size_t rt6_nlmsg_size(struct fib6_info *f6i);
|
2018-04-18 00:33:16 +00:00
|
|
|
static int rt6_fill_node(struct net *net, struct sk_buff *skb,
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt, struct dst_entry *dst,
|
2018-04-18 00:33:16 +00:00
|
|
|
struct in6_addr *dest, struct in6_addr *src,
|
2017-02-02 20:37:11 +00:00
|
|
|
int iif, int type, u32 portid, u32 seq,
|
|
|
|
unsigned int flags);
|
2019-04-16 21:36:00 +00:00
|
|
|
static struct rt6_info *rt6_find_cached_rt(const struct fib6_result *res,
|
2019-05-16 20:30:54 +00:00
|
|
|
const struct in6_addr *daddr,
|
|
|
|
const struct in6_addr *saddr);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2006-03-21 01:06:24 +00:00
|
|
|
#ifdef CONFIG_IPV6_ROUTE_INFO
|
2018-04-18 00:33:26 +00:00
|
|
|
static struct fib6_info *rt6_add_route_info(struct net *net,
|
2011-04-22 04:53:02 +00:00
|
|
|
const struct in6_addr *prefix, int prefixlen,
|
2016-10-24 17:52:35 +00:00
|
|
|
const struct in6_addr *gwaddr,
|
|
|
|
struct net_device *dev,
|
2012-04-15 05:58:06 +00:00
|
|
|
unsigned int pref);
|
2018-04-18 00:33:26 +00:00
|
|
|
static struct fib6_info *rt6_get_route_info(struct net *net,
|
2011-04-22 04:53:02 +00:00
|
|
|
const struct in6_addr *prefix, int prefixlen,
|
2016-10-24 17:52:35 +00:00
|
|
|
const struct in6_addr *gwaddr,
|
|
|
|
struct net_device *dev);
|
2006-03-21 01:06:24 +00:00
|
|
|
#endif
|
|
|
|
|
2015-05-23 03:56:04 +00:00
|
|
|
struct uncached_list {
|
|
|
|
spinlock_t lock;
|
|
|
|
struct list_head head;
|
|
|
|
};
|
|
|
|
|
|
|
|
static DEFINE_PER_CPU_ALIGNED(struct uncached_list, rt6_uncached_list);
|
|
|
|
|
xfrm: reuse uncached_list to track xdsts
In early time, when freeing a xdst, it would be inserted into
dst_garbage.list first. Then if it's refcnt was still held
somewhere, later it would be put into dst_busy_list in
dst_gc_task().
When one dev was being unregistered, the dev of these dsts in
dst_busy_list would be set with loopback_dev and put this dev.
So that this dev's removal wouldn't get blocked, and avoid the
kmsg warning:
kernel:unregister_netdevice: waiting for veth0 to become \
free. Usage count = 2
However after Commit 52df157f17e5 ("xfrm: take refcnt of dst
when creating struct xfrm_dst bundle"), the xdst will not be
freed with dst gc, and this warning happens.
To fix it, we need to find these xdsts that are still held by
others when removing the dev, and free xdst's dev and set it
with loopback_dev.
But unfortunately after flow_cache for xfrm was deleted, no
list tracks them anymore. So we need to save these xdsts
somewhere to release the xdst's dev later.
To make this easier, this patch is to reuse uncached_list to
track xdsts, so that the dev refcnt can be released in the
event NETDEV_UNREGISTER process of fib_netdev_notifier.
Thanks to Florian, we could move forward this fix quickly.
Fixes: 52df157f17e5 ("xfrm: take refcnt of dst when creating struct xfrm_dst bundle")
Reported-by: Jianlin Shi <jishi@redhat.com>
Reported-by: Hangbin Liu <liuhangbin@gmail.com>
Tested-by: Eyal Birger <eyal.birger@gmail.com>
Signed-off-by: Xin Long <lucien.xin@gmail.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
2018-02-14 11:06:02 +00:00
|
|
|
void rt6_uncached_list_add(struct rt6_info *rt)
|
2015-05-23 03:56:04 +00:00
|
|
|
{
|
|
|
|
struct uncached_list *ul = raw_cpu_ptr(&rt6_uncached_list);
|
|
|
|
|
|
|
|
rt->rt6i_uncached_list = ul;
|
|
|
|
|
|
|
|
spin_lock_bh(&ul->lock);
|
|
|
|
list_add_tail(&rt->rt6i_uncached, &ul->head);
|
|
|
|
spin_unlock_bh(&ul->lock);
|
|
|
|
}
|
|
|
|
|
xfrm: reuse uncached_list to track xdsts
In early time, when freeing a xdst, it would be inserted into
dst_garbage.list first. Then if it's refcnt was still held
somewhere, later it would be put into dst_busy_list in
dst_gc_task().
When one dev was being unregistered, the dev of these dsts in
dst_busy_list would be set with loopback_dev and put this dev.
So that this dev's removal wouldn't get blocked, and avoid the
kmsg warning:
kernel:unregister_netdevice: waiting for veth0 to become \
free. Usage count = 2
However after Commit 52df157f17e5 ("xfrm: take refcnt of dst
when creating struct xfrm_dst bundle"), the xdst will not be
freed with dst gc, and this warning happens.
To fix it, we need to find these xdsts that are still held by
others when removing the dev, and free xdst's dev and set it
with loopback_dev.
But unfortunately after flow_cache for xfrm was deleted, no
list tracks them anymore. So we need to save these xdsts
somewhere to release the xdst's dev later.
To make this easier, this patch is to reuse uncached_list to
track xdsts, so that the dev refcnt can be released in the
event NETDEV_UNREGISTER process of fib_netdev_notifier.
Thanks to Florian, we could move forward this fix quickly.
Fixes: 52df157f17e5 ("xfrm: take refcnt of dst when creating struct xfrm_dst bundle")
Reported-by: Jianlin Shi <jishi@redhat.com>
Reported-by: Hangbin Liu <liuhangbin@gmail.com>
Tested-by: Eyal Birger <eyal.birger@gmail.com>
Signed-off-by: Xin Long <lucien.xin@gmail.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
2018-02-14 11:06:02 +00:00
|
|
|
void rt6_uncached_list_del(struct rt6_info *rt)
|
2015-05-23 03:56:04 +00:00
|
|
|
{
|
|
|
|
if (!list_empty(&rt->rt6i_uncached)) {
|
|
|
|
struct uncached_list *ul = rt->rt6i_uncached_list;
|
2017-10-06 19:06:11 +00:00
|
|
|
struct net *net = dev_net(rt->dst.dev);
|
2015-05-23 03:56:04 +00:00
|
|
|
|
|
|
|
spin_lock_bh(&ul->lock);
|
|
|
|
list_del(&rt->rt6i_uncached);
|
2017-10-06 19:06:11 +00:00
|
|
|
atomic_dec(&net->ipv6.rt6_stats->fib_rt_uncache);
|
2015-05-23 03:56:04 +00:00
|
|
|
spin_unlock_bh(&ul->lock);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
static void rt6_uncached_list_flush_dev(struct net *net, struct net_device *dev)
|
|
|
|
{
|
|
|
|
struct net_device *loopback_dev = net->loopback_dev;
|
|
|
|
int cpu;
|
|
|
|
|
2015-10-12 16:02:08 +00:00
|
|
|
if (dev == loopback_dev)
|
|
|
|
return;
|
|
|
|
|
2015-05-23 03:56:04 +00:00
|
|
|
for_each_possible_cpu(cpu) {
|
|
|
|
struct uncached_list *ul = per_cpu_ptr(&rt6_uncached_list, cpu);
|
|
|
|
struct rt6_info *rt;
|
|
|
|
|
|
|
|
spin_lock_bh(&ul->lock);
|
|
|
|
list_for_each_entry(rt, &ul->head, rt6i_uncached) {
|
|
|
|
struct inet6_dev *rt_idev = rt->rt6i_idev;
|
|
|
|
struct net_device *rt_dev = rt->dst.dev;
|
|
|
|
|
2015-10-12 16:02:08 +00:00
|
|
|
if (rt_idev->dev == dev) {
|
2015-05-23 03:56:04 +00:00
|
|
|
rt->rt6i_idev = in6_dev_get(loopback_dev);
|
|
|
|
in6_dev_put(rt_idev);
|
|
|
|
}
|
|
|
|
|
2015-10-12 16:02:08 +00:00
|
|
|
if (rt_dev == dev) {
|
2019-07-01 21:38:57 +00:00
|
|
|
rt->dst.dev = blackhole_netdev;
|
2021-12-05 04:22:03 +00:00
|
|
|
dev_replace_track(rt_dev, blackhole_netdev,
|
|
|
|
&rt->dst.dev_tracker,
|
|
|
|
GFP_ATOMIC);
|
2015-05-23 03:56:04 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
spin_unlock_bh(&ul->lock);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:21 +00:00
|
|
|
static inline const void *choose_neigh_daddr(const struct in6_addr *p,
|
2012-07-03 04:52:24 +00:00
|
|
|
struct sk_buff *skb,
|
|
|
|
const void *daddr)
|
2012-01-26 20:22:32 +00:00
|
|
|
{
|
2012-01-26 21:29:16 +00:00
|
|
|
if (!ipv6_addr_any(p))
|
2012-01-26 20:22:32 +00:00
|
|
|
return (const void *) p;
|
2012-07-03 04:52:24 +00:00
|
|
|
else if (skb)
|
|
|
|
return &ipv6_hdr(skb)->daddr;
|
2012-01-26 20:22:32 +00:00
|
|
|
return daddr;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:21 +00:00
|
|
|
struct neighbour *ip6_neigh_lookup(const struct in6_addr *gw,
|
|
|
|
struct net_device *dev,
|
|
|
|
struct sk_buff *skb,
|
|
|
|
const void *daddr)
|
2011-07-18 07:40:17 +00:00
|
|
|
{
|
2012-01-26 20:22:32 +00:00
|
|
|
struct neighbour *n;
|
|
|
|
|
2018-04-18 00:33:21 +00:00
|
|
|
daddr = choose_neigh_daddr(gw, skb, daddr);
|
|
|
|
n = __ipv6_neigh_lookup(dev, daddr);
|
2011-12-28 20:41:23 +00:00
|
|
|
if (n)
|
|
|
|
return n;
|
2019-01-02 12:29:27 +00:00
|
|
|
|
|
|
|
n = neigh_create(&nd_tbl, daddr, dev);
|
|
|
|
return IS_ERR(n) ? NULL : n;
|
2018-04-18 00:33:21 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static struct neighbour *ip6_dst_neigh_lookup(const struct dst_entry *dst,
|
|
|
|
struct sk_buff *skb,
|
|
|
|
const void *daddr)
|
|
|
|
{
|
|
|
|
const struct rt6_info *rt = container_of(dst, struct rt6_info, dst);
|
|
|
|
|
ipv6: fix neighbour resolution with raw socket
The scenario is the following: the user uses a raw socket to send an ipv6
packet, destinated to a not-connected network, and specify a connected nh.
Here is the corresponding python script to reproduce this scenario:
import socket
IPPROTO_RAW = 255
send_s = socket.socket(socket.AF_INET6, socket.SOCK_RAW, IPPROTO_RAW)
# scapy
# p = IPv6(src='fd00:100::1', dst='fd00:200::fa')/ICMPv6EchoRequest()
# str(p)
req = b'`\x00\x00\x00\x00\x08:@\xfd\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\xfd\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xfa\x80\x00\x81\xc0\x00\x00\x00\x00'
send_s.sendto(req, ('fd00:175::2', 0, 0, 0))
fd00:175::/64 is a connected route and fd00:200::fa is not a connected
host.
With this scenario, the kernel starts by sending a NS to resolve
fd00:175::2. When it receives the NA, it flushes its queue and try to send
the initial packet. But instead of sending it, it sends another NS to
resolve fd00:200::fa, which obvioulsy fails, thus the packet is dropped. If
the user sends again the packet, it now uses the right nh (fd00:175::2).
The problem is that ip6_dst_lookup_neigh() uses the rt6i_gateway, which is
:: because the associated route is a connected route, thus it uses the dst
addr of the packet. Let's use rt6_nexthop() to choose the right nh.
Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 14:01:09 +00:00
|
|
|
return ip6_neigh_lookup(rt6_nexthop(rt, &in6addr_any),
|
|
|
|
dst->dev, skb, daddr);
|
2011-12-28 20:41:23 +00:00
|
|
|
}
|
|
|
|
|
2017-02-06 21:14:15 +00:00
|
|
|
static void ip6_confirm_neigh(const struct dst_entry *dst, const void *daddr)
|
|
|
|
{
|
|
|
|
struct net_device *dev = dst->dev;
|
|
|
|
struct rt6_info *rt = (struct rt6_info *)dst;
|
|
|
|
|
ipv6: Don't use dst gateway directly in ip6_confirm_neigh()
This is the equivalent of commit 2c6b55f45d53 ("ipv6: fix neighbour
resolution with raw socket") for ip6_confirm_neigh(): we can send a
packet with MSG_CONFIRM on a raw socket for a connected route, so the
gateway would be :: here, and we should pick the next hop using
rt6_nexthop() instead.
This was found by code review and, to the best of my knowledge, doesn't
actually fix a practical issue: the destination address from the packet
is not considered while confirming a neighbour, as ip6_confirm_neigh()
calls choose_neigh_daddr() without passing the packet, so there are no
similar issues as the one fixed by said commit.
A possible source of issues with the existing implementation might come
from the fact that, if we have a cached dst, we won't consider it,
while rt6_nexthop() takes care of that. I might just not be creative
enough to find a practical problem here: the only way to affect this
with cached routes is to have one coming from an ICMPv6 redirect, but
if the next hop is a directly connected host, there should be no
topology for which a redirect applies here, and tests with redirected
routes show no differences for MSG_CONFIRM (and MSG_PROBE) packets on
raw sockets destined to a directly connected host.
However, directly using the dst gateway here is not consistent anymore
with neighbour resolution, and, in general, as we want the next hop,
using rt6_nexthop() looks like the only sane way to fetch it.
Reported-by: Guillaume Nault <gnault@redhat.com>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Acked-by: Guillaume Nault <gnault@redhat.com>
Acked-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-09-09 20:44:06 +00:00
|
|
|
daddr = choose_neigh_daddr(rt6_nexthop(rt, &in6addr_any), NULL, daddr);
|
2017-02-06 21:14:15 +00:00
|
|
|
if (!daddr)
|
|
|
|
return;
|
|
|
|
if (dev->flags & (IFF_NOARP | IFF_LOOPBACK))
|
|
|
|
return;
|
|
|
|
if (ipv6_addr_is_multicast((const struct in6_addr *)daddr))
|
|
|
|
return;
|
|
|
|
__ipv6_confirm_neigh(dev, daddr);
|
|
|
|
}
|
|
|
|
|
2008-03-04 21:48:53 +00:00
|
|
|
static struct dst_ops ip6_dst_ops_template = {
|
2005-04-16 22:20:36 +00:00
|
|
|
.family = AF_INET6,
|
|
|
|
.gc = ip6_dst_gc,
|
|
|
|
.gc_thresh = 1024,
|
|
|
|
.check = ip6_dst_check,
|
2010-12-13 20:52:14 +00:00
|
|
|
.default_advmss = ip6_default_advmss,
|
2011-11-23 02:12:51 +00:00
|
|
|
.mtu = ip6_mtu,
|
2018-04-18 00:33:16 +00:00
|
|
|
.cow_metrics = dst_cow_metrics_generic,
|
2005-04-16 22:20:36 +00:00
|
|
|
.destroy = ip6_dst_destroy,
|
|
|
|
.ifdown = ip6_dst_ifdown,
|
|
|
|
.negative_advice = ip6_negative_advice,
|
|
|
|
.link_failure = ip6_link_failure,
|
|
|
|
.update_pmtu = ip6_rt_update_pmtu,
|
2012-07-12 07:05:02 +00:00
|
|
|
.redirect = rt6_do_redirect,
|
2015-10-07 21:48:39 +00:00
|
|
|
.local_out = __ip6_local_out,
|
2018-04-18 00:33:21 +00:00
|
|
|
.neigh_lookup = ip6_dst_neigh_lookup,
|
2017-02-06 21:14:15 +00:00
|
|
|
.confirm_neigh = ip6_confirm_neigh,
|
2005-04-16 22:20:36 +00:00
|
|
|
};
|
|
|
|
|
2007-05-25 01:17:54 +00:00
|
|
|
static struct dst_ops ip6_dst_blackhole_ops = {
|
2021-03-10 00:38:09 +00:00
|
|
|
.family = AF_INET6,
|
|
|
|
.default_advmss = ip6_default_advmss,
|
|
|
|
.neigh_lookup = ip6_dst_neigh_lookup,
|
|
|
|
.check = ip6_dst_check,
|
|
|
|
.destroy = ip6_dst_destroy,
|
|
|
|
.cow_metrics = dst_cow_metrics_generic,
|
|
|
|
.update_pmtu = dst_blackhole_update_pmtu,
|
|
|
|
.redirect = dst_blackhole_redirect,
|
|
|
|
.mtu = dst_blackhole_mtu,
|
2007-05-25 01:17:54 +00:00
|
|
|
};
|
|
|
|
|
net: Implement read-only protection and COW'ing of metrics.
Routing metrics are now copy-on-write.
Initially a route entry points it's metrics at a read-only location.
If a routing table entry exists, it will point there. Else it will
point at the all zero metric place-holder called 'dst_default_metrics'.
The writeability state of the metrics is stored in the low bits of the
metrics pointer, we have two bits left to spare if we want to store
more states.
For the initial implementation, COW is implemented simply via kmalloc.
However future enhancements will change this to place the writable
metrics somewhere else, in order to increase sharing. Very likely
this "somewhere else" will be the inetpeer cache.
Note also that this means that metrics updates may transiently fail
if we cannot COW the metrics successfully.
But even by itself, this patch should decrease memory usage and
increase cache locality especially for routing workloads. In those
cases the read-only metric copies stay in place and never get written
to.
TCP workloads where metrics get updated, and those rare cases where
PMTU triggers occur, will take a very slight performance hit. But
that hit will be alleviated when the long-term writable metrics
move to a more sharable location.
Since the metrics storage went from a u32 array of RTAX_MAX entries to
what is essentially a pointer, some retooling of the dst_entry layout
was necessary.
Most importantly, we need to preserve the alignment of the reference
count so that it doesn't share cache lines with the read-mostly state,
as per Eric Dumazet's alignment assertion checks.
The only non-trivial bit here is the move of the 'flags' member into
the writeable cacheline. This is OK since we are always accessing the
flags around the same moment when we made a modification to the
reference count.
Signed-off-by: David S. Miller <davem@davemloft.net>
2011-01-27 04:51:05 +00:00
|
|
|
static const u32 ip6_template_metrics[RTAX_MAX] = {
|
2012-10-24 06:01:18 +00:00
|
|
|
[RTAX_HOPLIMIT - 1] = 0,
|
net: Implement read-only protection and COW'ing of metrics.
Routing metrics are now copy-on-write.
Initially a route entry points it's metrics at a read-only location.
If a routing table entry exists, it will point there. Else it will
point at the all zero metric place-holder called 'dst_default_metrics'.
The writeability state of the metrics is stored in the low bits of the
metrics pointer, we have two bits left to spare if we want to store
more states.
For the initial implementation, COW is implemented simply via kmalloc.
However future enhancements will change this to place the writable
metrics somewhere else, in order to increase sharing. Very likely
this "somewhere else" will be the inetpeer cache.
Note also that this means that metrics updates may transiently fail
if we cannot COW the metrics successfully.
But even by itself, this patch should decrease memory usage and
increase cache locality especially for routing workloads. In those
cases the read-only metric copies stay in place and never get written
to.
TCP workloads where metrics get updated, and those rare cases where
PMTU triggers occur, will take a very slight performance hit. But
that hit will be alleviated when the long-term writable metrics
move to a more sharable location.
Since the metrics storage went from a u32 array of RTAX_MAX entries to
what is essentially a pointer, some retooling of the dst_entry layout
was necessary.
Most importantly, we need to preserve the alignment of the reference
count so that it doesn't share cache lines with the read-mostly state,
as per Eric Dumazet's alignment assertion checks.
The only non-trivial bit here is the move of the 'flags' member into
the writeable cacheline. This is OK since we are always accessing the
flags around the same moment when we made a modification to the
reference count.
Signed-off-by: David S. Miller <davem@davemloft.net>
2011-01-27 04:51:05 +00:00
|
|
|
};
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static const struct fib6_info fib6_null_entry_template = {
|
2018-04-18 22:38:59 +00:00
|
|
|
.fib6_flags = (RTF_REJECT | RTF_NONEXTHOP),
|
|
|
|
.fib6_protocol = RTPROT_KERNEL,
|
|
|
|
.fib6_metric = ~(u32)0,
|
2019-04-23 01:35:03 +00:00
|
|
|
.fib6_ref = REFCOUNT_INIT(1),
|
2018-04-18 00:33:18 +00:00
|
|
|
.fib6_type = RTN_UNREACHABLE,
|
|
|
|
.fib6_metrics = (struct dst_metrics *)&dst_default_metrics,
|
|
|
|
};
|
|
|
|
|
2012-09-11 21:47:51 +00:00
|
|
|
static const struct rt6_info ip6_null_entry_template = {
|
2010-06-11 06:31:35 +00:00
|
|
|
.dst = {
|
|
|
|
.__refcnt = ATOMIC_INIT(1),
|
|
|
|
.__use = 1,
|
2012-09-10 22:09:47 +00:00
|
|
|
.obsolete = DST_OBSOLETE_FORCE_CHK,
|
2010-06-11 06:31:35 +00:00
|
|
|
.error = -ENETUNREACH,
|
|
|
|
.input = ip6_pkt_discard,
|
|
|
|
.output = ip6_pkt_discard_out,
|
2005-04-16 22:20:36 +00:00
|
|
|
},
|
|
|
|
.rt6i_flags = (RTF_REJECT | RTF_NONEXTHOP),
|
|
|
|
};
|
|
|
|
|
2006-08-04 10:39:02 +00:00
|
|
|
#ifdef CONFIG_IPV6_MULTIPLE_TABLES
|
|
|
|
|
2012-09-11 21:47:51 +00:00
|
|
|
static const struct rt6_info ip6_prohibit_entry_template = {
|
2010-06-11 06:31:35 +00:00
|
|
|
.dst = {
|
|
|
|
.__refcnt = ATOMIC_INIT(1),
|
|
|
|
.__use = 1,
|
2012-09-10 22:09:47 +00:00
|
|
|
.obsolete = DST_OBSOLETE_FORCE_CHK,
|
2010-06-11 06:31:35 +00:00
|
|
|
.error = -EACCES,
|
|
|
|
.input = ip6_pkt_prohibit,
|
|
|
|
.output = ip6_pkt_prohibit_out,
|
2006-08-04 10:39:02 +00:00
|
|
|
},
|
|
|
|
.rt6i_flags = (RTF_REJECT | RTF_NONEXTHOP),
|
|
|
|
};
|
|
|
|
|
2012-09-11 21:47:51 +00:00
|
|
|
static const struct rt6_info ip6_blk_hole_entry_template = {
|
2010-06-11 06:31:35 +00:00
|
|
|
.dst = {
|
|
|
|
.__refcnt = ATOMIC_INIT(1),
|
|
|
|
.__use = 1,
|
2012-09-10 22:09:47 +00:00
|
|
|
.obsolete = DST_OBSOLETE_FORCE_CHK,
|
2010-06-11 06:31:35 +00:00
|
|
|
.error = -EINVAL,
|
|
|
|
.input = dst_discard,
|
2015-10-07 21:48:47 +00:00
|
|
|
.output = dst_discard_out,
|
2006-08-04 10:39:02 +00:00
|
|
|
},
|
|
|
|
.rt6i_flags = (RTF_REJECT | RTF_NONEXTHOP),
|
|
|
|
};
|
|
|
|
|
|
|
|
#endif
|
|
|
|
|
2015-10-15 23:39:57 +00:00
|
|
|
static void rt6_info_init(struct rt6_info *rt)
|
|
|
|
{
|
2021-11-18 20:32:41 +00:00
|
|
|
memset_after(rt, 0, dst);
|
2015-10-15 23:39:57 +00:00
|
|
|
INIT_LIST_HEAD(&rt->rt6i_uncached);
|
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/* allocate dst with ip6_dst_ops */
|
2018-04-18 00:33:25 +00:00
|
|
|
struct rt6_info *ip6_dst_alloc(struct net *net, struct net_device *dev,
|
|
|
|
int flags)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2012-06-10 05:36:36 +00:00
|
|
|
struct rt6_info *rt = dst_alloc(&net->ipv6.ip6_dst_ops, dev,
|
2017-06-17 17:42:41 +00:00
|
|
|
1, DST_OBSOLETE_FORCE_CHK, flags);
|
2011-04-28 21:31:47 +00:00
|
|
|
|
2017-10-06 19:06:11 +00:00
|
|
|
if (rt) {
|
2015-10-15 23:39:57 +00:00
|
|
|
rt6_info_init(rt);
|
2017-10-06 19:06:11 +00:00
|
|
|
atomic_inc(&net->ipv6.rt6_stats->fib_rt_alloc);
|
|
|
|
}
|
2012-07-05 23:37:09 +00:00
|
|
|
|
2011-04-28 21:31:47 +00:00
|
|
|
return rt;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2016-04-07 18:10:06 +00:00
|
|
|
EXPORT_SYMBOL(ip6_dst_alloc);
|
2015-05-23 03:56:06 +00:00
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
static void ip6_dst_destroy(struct dst_entry *dst)
|
|
|
|
{
|
|
|
|
struct rt6_info *rt = (struct rt6_info *)dst;
|
2018-04-20 22:38:02 +00:00
|
|
|
struct fib6_info *from;
|
2015-05-23 03:56:04 +00:00
|
|
|
struct inet6_dev *idev;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-10-05 03:07:54 +00:00
|
|
|
ip_dst_metrics_put(dst);
|
2015-05-23 03:56:04 +00:00
|
|
|
rt6_uncached_list_del(rt);
|
|
|
|
|
|
|
|
idev = rt->rt6i_idev;
|
2011-12-03 23:02:47 +00:00
|
|
|
if (idev) {
|
2005-04-16 22:20:36 +00:00
|
|
|
rt->rt6i_idev = NULL;
|
|
|
|
in6_dev_put(idev);
|
2007-02-09 14:24:49 +00:00
|
|
|
}
|
2012-04-06 00:13:10 +00:00
|
|
|
|
2019-04-28 19:22:25 +00:00
|
|
|
from = xchg((__force struct fib6_info **)&rt->from, NULL);
|
2018-04-18 00:33:25 +00:00
|
|
|
fib6_info_release(from);
|
2010-11-30 20:27:11 +00:00
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
static void ip6_dst_ifdown(struct dst_entry *dst, struct net_device *dev,
|
|
|
|
int how)
|
|
|
|
{
|
|
|
|
struct rt6_info *rt = (struct rt6_info *)dst;
|
|
|
|
struct inet6_dev *idev = rt->rt6i_idev;
|
2007-12-07 08:38:10 +00:00
|
|
|
struct net_device *loopback_dev =
|
2008-03-25 12:47:49 +00:00
|
|
|
dev_net(dev)->loopback_dev;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2017-08-14 17:44:59 +00:00
|
|
|
if (idev && idev->dev != loopback_dev) {
|
|
|
|
struct inet6_dev *loopback_idev = in6_dev_get(loopback_dev);
|
|
|
|
if (loopback_idev) {
|
|
|
|
rt->rt6i_idev = loopback_idev;
|
|
|
|
in6_dev_put(idev);
|
2012-07-03 05:43:47 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2015-11-11 19:51:07 +00:00
|
|
|
static bool __rt6_check_expired(const struct rt6_info *rt)
|
|
|
|
{
|
|
|
|
if (rt->rt6i_flags & RTF_EXPIRES)
|
|
|
|
return time_after(jiffies, rt->dst.expires);
|
|
|
|
else
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2012-05-18 18:57:34 +00:00
|
|
|
static bool rt6_check_expired(const struct rt6_info *rt)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2018-04-20 22:38:02 +00:00
|
|
|
struct fib6_info *from;
|
|
|
|
|
|
|
|
from = rcu_dereference(rt->from);
|
|
|
|
|
2012-04-06 00:13:10 +00:00
|
|
|
if (rt->rt6i_flags & RTF_EXPIRES) {
|
|
|
|
if (time_after(jiffies, rt->dst.expires))
|
2012-05-18 18:57:34 +00:00
|
|
|
return true;
|
2018-04-20 22:38:02 +00:00
|
|
|
} else if (from) {
|
2017-08-26 12:10:10 +00:00
|
|
|
return rt->dst.obsolete != DST_OBSOLETE_FORCE_CHK ||
|
2018-04-20 22:38:02 +00:00
|
|
|
fib6_check_expired(from);
|
2012-04-06 00:13:10 +00:00
|
|
|
}
|
2012-05-18 18:57:34 +00:00
|
|
|
return false;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2019-04-16 21:35:59 +00:00
|
|
|
void fib6_select_path(const struct net *net, struct fib6_result *res,
|
|
|
|
struct flowi6 *fl6, int oif, bool have_oif_match,
|
|
|
|
const struct sk_buff *skb, int strict)
|
2012-10-22 03:42:09 +00:00
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *sibling, *next_sibling;
|
2019-04-16 21:35:59 +00:00
|
|
|
struct fib6_info *match = res->f6i;
|
|
|
|
|
2020-07-06 17:45:07 +00:00
|
|
|
if (!match->nh && (!match->fib6_nsiblings || have_oif_match))
|
2019-04-16 21:35:59 +00:00
|
|
|
goto out;
|
2012-10-22 03:42:09 +00:00
|
|
|
|
2020-07-06 17:45:07 +00:00
|
|
|
if (match->nh && have_oif_match && res->nh)
|
|
|
|
return;
|
|
|
|
|
2017-08-23 07:58:31 +00:00
|
|
|
/* We might have already computed the hash for ICMPv6 errors. In such
|
|
|
|
* case it will always be non-zero. Otherwise now is the time to do it.
|
|
|
|
*/
|
2019-06-04 03:19:52 +00:00
|
|
|
if (!fl6->mp_hash &&
|
|
|
|
(!match->nh || nexthop_is_multipath(match->nh)))
|
2018-03-02 16:32:18 +00:00
|
|
|
fl6->mp_hash = rt6_multipath_hash(net, fl6, skb, NULL);
|
2017-08-23 07:58:31 +00:00
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
if (unlikely(match->nh)) {
|
|
|
|
nexthop_path_fib6_result(res, fl6->mp_hash);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:27:59 +00:00
|
|
|
if (fl6->mp_hash <= atomic_read(&match->fib6_nh->fib_nh_upper_bound))
|
2019-04-16 21:35:59 +00:00
|
|
|
goto out;
|
2018-01-09 14:40:27 +00:00
|
|
|
|
2018-04-18 22:38:59 +00:00
|
|
|
list_for_each_entry_safe(sibling, next_sibling, &match->fib6_siblings,
|
|
|
|
fib6_siblings) {
|
2019-05-23 03:27:59 +00:00
|
|
|
const struct fib6_nh *nh = sibling->fib6_nh;
|
2018-04-18 00:33:14 +00:00
|
|
|
int nh_upper_bound;
|
|
|
|
|
2019-04-09 21:41:13 +00:00
|
|
|
nh_upper_bound = atomic_read(&nh->fib_nh_upper_bound);
|
2018-04-18 00:33:14 +00:00
|
|
|
if (fl6->mp_hash > nh_upper_bound)
|
2018-01-09 14:40:27 +00:00
|
|
|
continue;
|
2019-04-09 21:41:13 +00:00
|
|
|
if (rt6_score_route(nh, sibling->fib6_flags, oif, strict) < 0)
|
2018-01-09 14:40:27 +00:00
|
|
|
break;
|
|
|
|
match = sibling;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
|
2019-04-16 21:35:59 +00:00
|
|
|
out:
|
|
|
|
res->f6i = match;
|
2019-05-23 03:27:59 +00:00
|
|
|
res->nh = match->fib6_nh;
|
2012-10-22 03:42:09 +00:00
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/*
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
* Route lookup. rcu_read_lock() should be held.
|
2005-04-16 22:20:36 +00:00
|
|
|
*/
|
|
|
|
|
2019-04-09 21:41:18 +00:00
|
|
|
static bool __rt6_device_match(struct net *net, const struct fib6_nh *nh,
|
|
|
|
const struct in6_addr *saddr, int oif, int flags)
|
|
|
|
{
|
|
|
|
const struct net_device *dev;
|
|
|
|
|
|
|
|
if (nh->fib_nh_flags & RTNH_F_DEAD)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
dev = nh->fib_nh_dev;
|
|
|
|
if (oif) {
|
|
|
|
if (dev->ifindex == oif)
|
|
|
|
return true;
|
|
|
|
} else {
|
|
|
|
if (ipv6_chk_addr(net, saddr, dev,
|
|
|
|
flags & RT6_LOOKUP_F_IFACE))
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2019-06-08 21:53:24 +00:00
|
|
|
struct fib6_nh_dm_arg {
|
|
|
|
struct net *net;
|
|
|
|
const struct in6_addr *saddr;
|
|
|
|
int oif;
|
|
|
|
int flags;
|
|
|
|
struct fib6_nh *nh;
|
|
|
|
};
|
|
|
|
|
|
|
|
static int __rt6_nh_dev_match(struct fib6_nh *nh, void *_arg)
|
|
|
|
{
|
|
|
|
struct fib6_nh_dm_arg *arg = _arg;
|
|
|
|
|
|
|
|
arg->nh = nh;
|
|
|
|
return __rt6_device_match(arg->net, nh, arg->saddr, arg->oif,
|
|
|
|
arg->flags);
|
|
|
|
}
|
|
|
|
|
|
|
|
/* returns fib6_nh from nexthop or NULL */
|
|
|
|
static struct fib6_nh *rt6_nh_dev_match(struct net *net, struct nexthop *nh,
|
|
|
|
struct fib6_result *res,
|
|
|
|
const struct in6_addr *saddr,
|
|
|
|
int oif, int flags)
|
|
|
|
{
|
|
|
|
struct fib6_nh_dm_arg arg = {
|
|
|
|
.net = net,
|
|
|
|
.saddr = saddr,
|
|
|
|
.oif = oif,
|
|
|
|
.flags = flags,
|
|
|
|
};
|
|
|
|
|
|
|
|
if (nexthop_is_blackhole(nh))
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
if (nexthop_for_each_fib6_nh(nh, __rt6_nh_dev_match, &arg))
|
|
|
|
return arg.nh;
|
|
|
|
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:07 +00:00
|
|
|
static void rt6_device_match(struct net *net, struct fib6_result *res,
|
|
|
|
const struct in6_addr *saddr, int oif, int flags)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2019-04-16 21:36:07 +00:00
|
|
|
struct fib6_info *f6i = res->f6i;
|
|
|
|
struct fib6_info *spf6i;
|
|
|
|
struct fib6_nh *nh;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-04-16 21:36:07 +00:00
|
|
|
if (!oif && ipv6_addr_any(saddr)) {
|
2019-06-04 03:19:52 +00:00
|
|
|
if (unlikely(f6i->nh)) {
|
|
|
|
nh = nexthop_fib6_nh(f6i->nh);
|
|
|
|
if (nexthop_is_blackhole(f6i->nh))
|
|
|
|
goto out_blackhole;
|
|
|
|
} else {
|
|
|
|
nh = f6i->fib6_nh;
|
|
|
|
}
|
2019-04-16 21:36:11 +00:00
|
|
|
if (!(nh->fib_nh_flags & RTNH_F_DEAD))
|
|
|
|
goto out;
|
2019-04-16 21:36:07 +00:00
|
|
|
}
|
2008-07-02 09:30:18 +00:00
|
|
|
|
2019-04-16 21:36:07 +00:00
|
|
|
for (spf6i = f6i; spf6i; spf6i = rcu_dereference(spf6i->fib6_next)) {
|
2019-06-08 21:53:24 +00:00
|
|
|
bool matched = false;
|
|
|
|
|
|
|
|
if (unlikely(spf6i->nh)) {
|
|
|
|
nh = rt6_nh_dev_match(net, spf6i->nh, res, saddr,
|
|
|
|
oif, flags);
|
|
|
|
if (nh)
|
|
|
|
matched = true;
|
|
|
|
} else {
|
|
|
|
nh = spf6i->fib6_nh;
|
|
|
|
if (__rt6_device_match(net, nh, saddr, oif, flags))
|
|
|
|
matched = true;
|
|
|
|
}
|
|
|
|
if (matched) {
|
2019-04-16 21:36:07 +00:00
|
|
|
res->f6i = spf6i;
|
2019-04-16 21:36:11 +00:00
|
|
|
goto out;
|
2019-04-16 21:36:07 +00:00
|
|
|
}
|
2008-07-02 09:30:18 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-04-16 21:36:07 +00:00
|
|
|
if (oif && flags & RT6_LOOKUP_F_IFACE) {
|
|
|
|
res->f6i = net->ipv6.fib6_null_entry;
|
2019-05-23 03:27:59 +00:00
|
|
|
nh = res->f6i->fib6_nh;
|
2019-04-16 21:36:11 +00:00
|
|
|
goto out;
|
2019-04-16 21:36:07 +00:00
|
|
|
}
|
2018-01-07 10:45:09 +00:00
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
if (unlikely(f6i->nh)) {
|
|
|
|
nh = nexthop_fib6_nh(f6i->nh);
|
|
|
|
if (nexthop_is_blackhole(f6i->nh))
|
|
|
|
goto out_blackhole;
|
|
|
|
} else {
|
|
|
|
nh = f6i->fib6_nh;
|
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:11 +00:00
|
|
|
if (nh->fib_nh_flags & RTNH_F_DEAD) {
|
2019-04-16 21:36:07 +00:00
|
|
|
res->f6i = net->ipv6.fib6_null_entry;
|
2019-05-23 03:27:59 +00:00
|
|
|
nh = res->f6i->fib6_nh;
|
2019-04-16 21:36:07 +00:00
|
|
|
}
|
2019-04-16 21:36:11 +00:00
|
|
|
out:
|
|
|
|
res->nh = nh;
|
|
|
|
res->fib6_type = res->f6i->fib6_type;
|
|
|
|
res->fib6_flags = res->f6i->fib6_flags;
|
2019-06-04 03:19:52 +00:00
|
|
|
return;
|
|
|
|
|
|
|
|
out_blackhole:
|
|
|
|
res->fib6_flags |= RTF_REJECT;
|
|
|
|
res->fib6_type = RTN_BLACKHOLE;
|
|
|
|
res->nh = nh;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2006-03-21 01:05:13 +00:00
|
|
|
#ifdef CONFIG_IPV6_ROUTER_PREF
|
2013-10-21 04:17:15 +00:00
|
|
|
struct __rt6_probe_work {
|
|
|
|
struct work_struct work;
|
|
|
|
struct in6_addr target;
|
|
|
|
struct net_device *dev;
|
2021-12-05 04:22:04 +00:00
|
|
|
netdevice_tracker dev_tracker;
|
2013-10-21 04:17:15 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
static void rt6_probe_deferred(struct work_struct *w)
|
|
|
|
{
|
|
|
|
struct in6_addr mcaddr;
|
|
|
|
struct __rt6_probe_work *work =
|
|
|
|
container_of(w, struct __rt6_probe_work, work);
|
|
|
|
|
|
|
|
addrconf_addr_solict_mult(&work->target, &mcaddr);
|
2016-12-02 22:00:08 +00:00
|
|
|
ndisc_send_ns(work->dev, &work->target, &mcaddr, NULL, 0);
|
2021-12-05 04:22:04 +00:00
|
|
|
dev_put_track(work->dev, &work->dev_tracker);
|
2015-02-08 09:14:07 +00:00
|
|
|
kfree(work);
|
2013-10-21 04:17:15 +00:00
|
|
|
}
|
|
|
|
|
2019-04-09 21:41:12 +00:00
|
|
|
static void rt6_probe(struct fib6_nh *fib6_nh)
|
2006-03-21 01:05:13 +00:00
|
|
|
{
|
2018-10-12 14:22:47 +00:00
|
|
|
struct __rt6_probe_work *work = NULL;
|
2018-04-18 00:33:14 +00:00
|
|
|
const struct in6_addr *nh_gw;
|
2019-11-07 17:26:19 +00:00
|
|
|
unsigned long last_probe;
|
2011-07-29 19:00:53 +00:00
|
|
|
struct neighbour *neigh;
|
2018-04-18 00:33:14 +00:00
|
|
|
struct net_device *dev;
|
2018-10-12 14:22:47 +00:00
|
|
|
struct inet6_dev *idev;
|
2018-04-18 00:33:14 +00:00
|
|
|
|
2006-03-21 01:05:13 +00:00
|
|
|
/*
|
|
|
|
* Okay, this does not seem to be appropriate
|
|
|
|
* for now, however, we need to check if it
|
|
|
|
* is really so; aka Router Reachability Probing.
|
|
|
|
*
|
|
|
|
* Router Reachability Probe MUST be rate-limited
|
|
|
|
* to no more than one per minute.
|
|
|
|
*/
|
2019-11-20 07:39:06 +00:00
|
|
|
if (!fib6_nh->fib_nh_gw_family)
|
2013-01-17 12:53:02 +00:00
|
|
|
return;
|
2018-04-18 00:33:14 +00:00
|
|
|
|
2019-04-09 21:41:12 +00:00
|
|
|
nh_gw = &fib6_nh->fib_nh_gw6;
|
|
|
|
dev = fib6_nh->fib_nh_dev;
|
2013-01-17 12:53:43 +00:00
|
|
|
rcu_read_lock_bh();
|
2019-11-07 17:26:19 +00:00
|
|
|
last_probe = READ_ONCE(fib6_nh->last_probe);
|
2018-10-12 14:22:47 +00:00
|
|
|
idev = __in6_dev_get(dev);
|
2018-04-18 00:33:14 +00:00
|
|
|
neigh = __ipv6_neigh_lookup_noref(dev, nh_gw);
|
2013-01-17 12:53:43 +00:00
|
|
|
if (neigh) {
|
2015-07-24 16:57:43 +00:00
|
|
|
if (neigh->nud_state & NUD_VALID)
|
|
|
|
goto out;
|
|
|
|
|
2013-01-17 12:53:43 +00:00
|
|
|
write_lock(&neigh->lock);
|
2015-07-24 16:57:42 +00:00
|
|
|
if (!(neigh->nud_state & NUD_VALID) &&
|
|
|
|
time_after(jiffies,
|
2018-04-18 22:39:05 +00:00
|
|
|
neigh->updated + idev->cnf.rtr_probe_interval)) {
|
2015-07-24 16:57:42 +00:00
|
|
|
work = kmalloc(sizeof(*work), GFP_ATOMIC);
|
|
|
|
if (work)
|
|
|
|
__neigh_set_probe_once(neigh);
|
2013-10-21 04:17:15 +00:00
|
|
|
}
|
2013-01-17 12:53:43 +00:00
|
|
|
write_unlock(&neigh->lock);
|
2019-11-07 17:26:19 +00:00
|
|
|
} else if (time_after(jiffies, last_probe +
|
2018-10-12 14:22:47 +00:00
|
|
|
idev->cnf.rtr_probe_interval)) {
|
2015-07-24 16:57:42 +00:00
|
|
|
work = kmalloc(sizeof(*work), GFP_ATOMIC);
|
2011-07-29 19:00:53 +00:00
|
|
|
}
|
2015-07-24 16:57:42 +00:00
|
|
|
|
2019-11-07 17:26:19 +00:00
|
|
|
if (!work || cmpxchg(&fib6_nh->last_probe,
|
|
|
|
last_probe, jiffies) != last_probe) {
|
|
|
|
kfree(work);
|
|
|
|
} else {
|
2015-07-24 16:57:42 +00:00
|
|
|
INIT_WORK(&work->work, rt6_probe_deferred);
|
2018-04-18 00:33:14 +00:00
|
|
|
work->target = *nh_gw;
|
2021-12-14 02:58:06 +00:00
|
|
|
dev_hold_track(dev, &work->dev_tracker, GFP_ATOMIC);
|
2018-04-18 00:33:14 +00:00
|
|
|
work->dev = dev;
|
2015-07-24 16:57:42 +00:00
|
|
|
schedule_work(&work->work);
|
|
|
|
}
|
|
|
|
|
2015-07-24 16:57:43 +00:00
|
|
|
out:
|
2013-01-17 12:53:43 +00:00
|
|
|
rcu_read_unlock_bh();
|
2006-03-21 01:05:13 +00:00
|
|
|
}
|
|
|
|
#else
|
2019-04-09 21:41:12 +00:00
|
|
|
static inline void rt6_probe(struct fib6_nh *fib6_nh)
|
2006-03-21 01:05:13 +00:00
|
|
|
{
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/*
|
2006-03-21 01:00:26 +00:00
|
|
|
* Default Router Selection (RFC 2461 6.3.6)
|
2005-04-16 22:20:36 +00:00
|
|
|
*/
|
2019-04-09 21:41:10 +00:00
|
|
|
static enum rt6_nud_state rt6_check_neigh(const struct fib6_nh *fib6_nh)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2013-07-11 10:43:42 +00:00
|
|
|
enum rt6_nud_state ret = RT6_NUD_FAIL_HARD;
|
2018-04-18 00:33:14 +00:00
|
|
|
struct neighbour *neigh;
|
2011-07-29 19:00:53 +00:00
|
|
|
|
2013-01-17 12:53:38 +00:00
|
|
|
rcu_read_lock_bh();
|
2019-04-09 21:41:10 +00:00
|
|
|
neigh = __ipv6_neigh_lookup_noref(fib6_nh->fib_nh_dev,
|
|
|
|
&fib6_nh->fib_nh_gw6);
|
2013-01-17 12:53:38 +00:00
|
|
|
if (neigh) {
|
|
|
|
read_lock(&neigh->lock);
|
2006-03-21 01:00:26 +00:00
|
|
|
if (neigh->nud_state & NUD_VALID)
|
2013-07-11 10:43:42 +00:00
|
|
|
ret = RT6_NUD_SUCCEED;
|
2008-01-19 08:35:16 +00:00
|
|
|
#ifdef CONFIG_IPV6_ROUTER_PREF
|
2012-12-03 10:26:54 +00:00
|
|
|
else if (!(neigh->nud_state & NUD_FAILED))
|
2013-07-11 10:43:42 +00:00
|
|
|
ret = RT6_NUD_SUCCEED;
|
2013-12-11 12:48:20 +00:00
|
|
|
else
|
|
|
|
ret = RT6_NUD_FAIL_PROBE;
|
2008-01-19 08:35:16 +00:00
|
|
|
#endif
|
2013-01-17 12:53:38 +00:00
|
|
|
read_unlock(&neigh->lock);
|
2013-07-11 10:43:42 +00:00
|
|
|
} else {
|
|
|
|
ret = IS_ENABLED(CONFIG_IPV6_ROUTER_PREF) ?
|
2013-12-11 12:48:20 +00:00
|
|
|
RT6_NUD_SUCCEED : RT6_NUD_FAIL_DO_RR;
|
2012-12-03 10:26:54 +00:00
|
|
|
}
|
2013-01-17 12:53:38 +00:00
|
|
|
rcu_read_unlock_bh();
|
|
|
|
|
2012-12-03 10:26:54 +00:00
|
|
|
return ret;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2019-04-09 21:41:13 +00:00
|
|
|
static int rt6_score_route(const struct fib6_nh *nh, u32 fib6_flags, int oif,
|
|
|
|
int strict)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2019-04-09 21:41:11 +00:00
|
|
|
int m = 0;
|
|
|
|
|
|
|
|
if (!oif || nh->fib_nh_dev->ifindex == oif)
|
|
|
|
m = 2;
|
2007-02-09 14:24:49 +00:00
|
|
|
|
2006-08-24 00:25:05 +00:00
|
|
|
if (!m && (strict & RT6_LOOKUP_F_IFACE))
|
2013-07-11 10:43:42 +00:00
|
|
|
return RT6_NUD_FAIL_HARD;
|
2006-03-21 01:04:53 +00:00
|
|
|
#ifdef CONFIG_IPV6_ROUTER_PREF
|
2019-04-09 21:41:13 +00:00
|
|
|
m |= IPV6_DECODE_PREF(IPV6_EXTRACT_PREF(fib6_flags)) << 2;
|
2006-03-21 01:04:53 +00:00
|
|
|
#endif
|
2019-04-09 21:41:10 +00:00
|
|
|
if ((strict & RT6_LOOKUP_F_REACHABLE) &&
|
2019-04-09 21:41:13 +00:00
|
|
|
!(fib6_flags & RTF_NONEXTHOP) && nh->fib_nh_gw_family) {
|
2019-04-09 21:41:10 +00:00
|
|
|
int n = rt6_check_neigh(nh);
|
2013-07-11 10:43:42 +00:00
|
|
|
if (n < 0)
|
|
|
|
return n;
|
|
|
|
}
|
2006-03-21 01:00:26 +00:00
|
|
|
return m;
|
|
|
|
}
|
|
|
|
|
2019-04-09 21:41:14 +00:00
|
|
|
static bool find_match(struct fib6_nh *nh, u32 fib6_flags,
|
|
|
|
int oif, int strict, int *mpri, bool *do_rr)
|
2006-03-21 01:00:26 +00:00
|
|
|
{
|
2013-07-11 10:43:42 +00:00
|
|
|
bool match_do_rr = false;
|
2019-04-09 21:41:14 +00:00
|
|
|
bool rc = false;
|
|
|
|
int m;
|
2015-08-13 14:39:01 +00:00
|
|
|
|
2019-04-09 21:41:14 +00:00
|
|
|
if (nh->fib_nh_flags & RTNH_F_DEAD)
|
2018-01-07 10:45:09 +00:00
|
|
|
goto out;
|
|
|
|
|
2019-04-09 21:41:14 +00:00
|
|
|
if (ip6_ignore_linkdown(nh->fib_nh_dev) &&
|
|
|
|
nh->fib_nh_flags & RTNH_F_LINKDOWN &&
|
2016-10-24 19:27:23 +00:00
|
|
|
!(strict & RT6_LOOKUP_F_IGNORE_LINKSTATE))
|
2015-08-13 14:39:01 +00:00
|
|
|
goto out;
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
|
2019-04-09 21:41:14 +00:00
|
|
|
m = rt6_score_route(nh, fib6_flags, oif, strict);
|
2013-12-11 12:48:20 +00:00
|
|
|
if (m == RT6_NUD_FAIL_DO_RR) {
|
2013-07-11 10:43:42 +00:00
|
|
|
match_do_rr = true;
|
|
|
|
m = 0; /* lowest valid score */
|
2013-12-11 12:48:20 +00:00
|
|
|
} else if (m == RT6_NUD_FAIL_HARD) {
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
goto out;
|
2013-07-11 10:43:42 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if (strict & RT6_LOOKUP_F_REACHABLE)
|
2019-04-09 21:41:14 +00:00
|
|
|
rt6_probe(nh);
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
|
2013-12-11 12:48:20 +00:00
|
|
|
/* note that m can be RT6_NUD_FAIL_PROBE at this point */
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
if (m > *mpri) {
|
2013-07-11 10:43:42 +00:00
|
|
|
*do_rr = match_do_rr;
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
*mpri = m;
|
2019-04-09 21:41:14 +00:00
|
|
|
rc = true;
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
}
|
|
|
|
out:
|
2019-04-09 21:41:14 +00:00
|
|
|
return rc;
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
}
|
|
|
|
|
2019-06-08 21:53:25 +00:00
|
|
|
struct fib6_nh_frl_arg {
|
|
|
|
u32 flags;
|
|
|
|
int oif;
|
|
|
|
int strict;
|
|
|
|
int *mpri;
|
|
|
|
bool *do_rr;
|
|
|
|
struct fib6_nh *nh;
|
|
|
|
};
|
|
|
|
|
|
|
|
static int rt6_nh_find_match(struct fib6_nh *nh, void *_arg)
|
|
|
|
{
|
|
|
|
struct fib6_nh_frl_arg *arg = _arg;
|
|
|
|
|
|
|
|
arg->nh = nh;
|
|
|
|
return find_match(nh, arg->flags, arg->oif, arg->strict,
|
|
|
|
arg->mpri, arg->do_rr);
|
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
static void __find_rr_leaf(struct fib6_info *f6i_start,
|
2019-04-09 21:41:15 +00:00
|
|
|
struct fib6_info *nomatch, u32 metric,
|
2019-04-16 21:36:08 +00:00
|
|
|
struct fib6_result *res, struct fib6_info **cont,
|
2019-04-09 21:41:15 +00:00
|
|
|
int oif, int strict, bool *do_rr, int *mpri)
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
{
|
2019-04-16 21:36:08 +00:00
|
|
|
struct fib6_info *f6i;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
for (f6i = f6i_start;
|
|
|
|
f6i && f6i != nomatch;
|
|
|
|
f6i = rcu_dereference(f6i->fib6_next)) {
|
2019-06-08 21:53:25 +00:00
|
|
|
bool matched = false;
|
2019-04-09 21:41:15 +00:00
|
|
|
struct fib6_nh *nh;
|
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
if (cont && f6i->fib6_metric != metric) {
|
|
|
|
*cont = f6i;
|
2019-04-09 21:41:15 +00:00
|
|
|
return;
|
2015-04-28 20:03:04 +00:00
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
if (fib6_check_expired(f6i))
|
2019-04-09 21:41:14 +00:00
|
|
|
continue;
|
|
|
|
|
2019-06-08 21:53:25 +00:00
|
|
|
if (unlikely(f6i->nh)) {
|
|
|
|
struct fib6_nh_frl_arg arg = {
|
|
|
|
.flags = f6i->fib6_flags,
|
|
|
|
.oif = oif,
|
|
|
|
.strict = strict,
|
|
|
|
.mpri = mpri,
|
|
|
|
.do_rr = do_rr
|
|
|
|
};
|
|
|
|
|
|
|
|
if (nexthop_is_blackhole(f6i->nh)) {
|
|
|
|
res->fib6_flags = RTF_REJECT;
|
|
|
|
res->fib6_type = RTN_BLACKHOLE;
|
|
|
|
res->f6i = f6i;
|
|
|
|
res->nh = nexthop_fib6_nh(f6i->nh);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
if (nexthop_for_each_fib6_nh(f6i->nh, rt6_nh_find_match,
|
|
|
|
&arg)) {
|
|
|
|
matched = true;
|
|
|
|
nh = arg.nh;
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
nh = f6i->fib6_nh;
|
|
|
|
if (find_match(nh, f6i->fib6_flags, oif, strict,
|
|
|
|
mpri, do_rr))
|
|
|
|
matched = true;
|
|
|
|
}
|
|
|
|
if (matched) {
|
2019-04-16 21:36:08 +00:00
|
|
|
res->f6i = f6i;
|
|
|
|
res->nh = nh;
|
2019-04-16 21:36:11 +00:00
|
|
|
res->fib6_flags = f6i->fib6_flags;
|
|
|
|
res->fib6_type = f6i->fib6_type;
|
2019-04-16 21:36:08 +00:00
|
|
|
}
|
2015-04-28 20:03:04 +00:00
|
|
|
}
|
2019-04-09 21:41:15 +00:00
|
|
|
}
|
2015-04-28 20:03:04 +00:00
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
static void find_rr_leaf(struct fib6_node *fn, struct fib6_info *leaf,
|
|
|
|
struct fib6_info *rr_head, int oif, int strict,
|
|
|
|
bool *do_rr, struct fib6_result *res)
|
2019-04-09 21:41:15 +00:00
|
|
|
{
|
2019-04-16 21:36:08 +00:00
|
|
|
u32 metric = rr_head->fib6_metric;
|
|
|
|
struct fib6_info *cont = NULL;
|
2019-04-09 21:41:15 +00:00
|
|
|
int mpri = -1;
|
2015-04-28 20:03:04 +00:00
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
__find_rr_leaf(rr_head, NULL, metric, res, &cont,
|
2019-04-09 21:41:15 +00:00
|
|
|
oif, strict, do_rr, &mpri);
|
2019-04-09 21:41:14 +00:00
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
__find_rr_leaf(leaf, rr_head, metric, res, &cont,
|
2019-04-09 21:41:15 +00:00
|
|
|
oif, strict, do_rr, &mpri);
|
2015-04-28 20:03:04 +00:00
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
if (res->f6i || !cont)
|
|
|
|
return;
|
2015-04-28 20:03:04 +00:00
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
__find_rr_leaf(cont, NULL, metric, res, NULL,
|
2019-04-09 21:41:15 +00:00
|
|
|
oif, strict, do_rr, &mpri);
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
static void rt6_select(struct net *net, struct fib6_node *fn, int oif,
|
|
|
|
struct fib6_result *res, int strict)
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *leaf = rcu_dereference(fn->leaf);
|
2019-04-16 21:36:08 +00:00
|
|
|
struct fib6_info *rt0;
|
2013-07-11 10:43:42 +00:00
|
|
|
bool do_rr = false;
|
2017-10-06 19:06:09 +00:00
|
|
|
int key_plen;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
/* make sure this function or its helpers sets f6i */
|
|
|
|
res->f6i = NULL;
|
|
|
|
|
2018-04-18 00:33:18 +00:00
|
|
|
if (!leaf || leaf == net->ipv6.fib6_null_entry)
|
2019-04-16 21:36:08 +00:00
|
|
|
goto out;
|
2017-10-06 19:06:08 +00:00
|
|
|
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rt0 = rcu_dereference(fn->rr_ptr);
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
if (!rt0)
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rt0 = leaf;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2017-10-06 19:06:09 +00:00
|
|
|
/* Double check to make sure fn is not an intermediate node
|
|
|
|
* and fn->leaf does not points to its child's leaf
|
|
|
|
* (This might happen if all routes under fn are deleted from
|
|
|
|
* the tree and fib6_repair_tree() is called on the node.)
|
|
|
|
*/
|
2018-04-18 22:38:59 +00:00
|
|
|
key_plen = rt0->fib6_dst.plen;
|
2017-10-06 19:06:09 +00:00
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
2018-04-18 22:38:59 +00:00
|
|
|
if (rt0->fib6_src.plen)
|
|
|
|
key_plen = rt0->fib6_src.plen;
|
2017-10-06 19:06:09 +00:00
|
|
|
#endif
|
|
|
|
if (fn->fn_bit != key_plen)
|
2019-04-16 21:36:08 +00:00
|
|
|
goto out;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
find_rr_leaf(fn, leaf, rt0, oif, strict, &do_rr, res);
|
2013-07-11 10:43:42 +00:00
|
|
|
if (do_rr) {
|
2018-05-04 20:54:24 +00:00
|
|
|
struct fib6_info *next = rcu_dereference(rt0->fib6_next);
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
|
2006-03-21 01:00:26 +00:00
|
|
|
/* no entries matched; do round-robin */
|
2018-04-18 22:38:59 +00:00
|
|
|
if (!next || next->fib6_metric != rt0->fib6_metric)
|
2017-10-06 19:06:08 +00:00
|
|
|
next = leaf;
|
[IPV6]: Fix routing round-robin locking.
As per RFC2461, section 6.3.6, item #2, when no routers on the
matching list are known to be reachable or probably reachable we
do round robin on those available routes so that we make sure
to probe as many of them as possible to detect when one becomes
reachable faster.
Each routing table has a rwlock protecting the tree and the linked
list of routes at each leaf. The round robin code executes during
lookup and thus with the rwlock taken as a reader. A small local
spinlock tries to provide protection but this does not work at all
for two reasons:
1) The round-robin list manipulation, as coded, goes like this (with
read lock held):
walk routes finding head and tail
spin_lock();
rotate list using head and tail
spin_unlock();
While one thread is rotating the list, another thread can
end up with stale values of head and tail and then proceed
to corrupt the list when it gets the lock. This ends up causing
the OOPS in fib6_add() later onthat many people have been hitting.
2) All the other code paths that run with the rwlock held as
a reader do not expect the list to change on them, they
expect it to remain completely fixed while they hold the
lock in that way.
So, simply stated, it is impossible to implement this correctly using
a manipulation of the list without violating the rwlock locking
semantics.
Reimplement using a per-fib6_node round-robin pointer. This way we
don't need to manipulate the list at all, and since the round-robin
pointer can only ever point to real existing entries we don't need
to perform any locking on the changing of the round-robin pointer
itself. We only need to reset the round-robin pointer to NULL when
the entry it is pointing to is removed.
The idea is from Thomas Graf and it is very similar to how this
was implemented before the advanced router selection code when in.
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-03-25 03:36:25 +00:00
|
|
|
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
if (next != rt0) {
|
2018-04-18 22:38:59 +00:00
|
|
|
spin_lock_bh(&leaf->fib6_table->tb6_lock);
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
/* make sure next is not being deleted from the tree */
|
2018-04-18 22:38:59 +00:00
|
|
|
if (next->fib6_node)
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_assign_pointer(fn->rr_ptr, next);
|
2018-04-18 22:38:59 +00:00
|
|
|
spin_unlock_bh(&leaf->fib6_table->tb6_lock);
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:08 +00:00
|
|
|
out:
|
|
|
|
if (!res->f6i) {
|
|
|
|
res->f6i = net->ipv6.fib6_null_entry;
|
2019-05-23 03:27:59 +00:00
|
|
|
res->nh = res->f6i->fib6_nh;
|
2019-04-16 21:36:11 +00:00
|
|
|
res->fib6_flags = res->f6i->fib6_flags;
|
|
|
|
res->fib6_type = res->f6i->fib6_type;
|
2019-04-16 21:36:08 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:01 +00:00
|
|
|
static bool rt6_is_gw_or_nonexthop(const struct fib6_result *res)
|
2015-05-23 03:55:59 +00:00
|
|
|
{
|
2019-04-16 21:36:01 +00:00
|
|
|
return (res->f6i->fib6_flags & RTF_NONEXTHOP) ||
|
|
|
|
res->nh->fib_nh_gw_family;
|
2015-05-23 03:55:59 +00:00
|
|
|
}
|
|
|
|
|
2006-03-21 01:06:24 +00:00
|
|
|
#ifdef CONFIG_IPV6_ROUTE_INFO
|
|
|
|
int rt6_route_rcv(struct net_device *dev, u8 *opt, int len,
|
2011-04-22 04:53:02 +00:00
|
|
|
const struct in6_addr *gwaddr)
|
2006-03-21 01:06:24 +00:00
|
|
|
{
|
2008-03-25 12:47:49 +00:00
|
|
|
struct net *net = dev_net(dev);
|
2006-03-21 01:06:24 +00:00
|
|
|
struct route_info *rinfo = (struct route_info *) opt;
|
|
|
|
struct in6_addr prefix_buf, *prefix;
|
|
|
|
unsigned int pref;
|
2008-05-27 08:37:49 +00:00
|
|
|
unsigned long lifetime;
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt;
|
2006-03-21 01:06:24 +00:00
|
|
|
|
|
|
|
if (len < sizeof(struct route_info)) {
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Sanity check for prefix_len and length */
|
|
|
|
if (rinfo->length > 3) {
|
|
|
|
return -EINVAL;
|
|
|
|
} else if (rinfo->prefix_len > 128) {
|
|
|
|
return -EINVAL;
|
|
|
|
} else if (rinfo->prefix_len > 64) {
|
|
|
|
if (rinfo->length < 2) {
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
} else if (rinfo->prefix_len > 0) {
|
|
|
|
if (rinfo->length < 1) {
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
pref = rinfo->route_pref;
|
|
|
|
if (pref == ICMPV6_ROUTER_PREF_INVALID)
|
2009-09-10 06:25:11 +00:00
|
|
|
return -EINVAL;
|
2006-03-21 01:06:24 +00:00
|
|
|
|
2008-05-27 08:37:49 +00:00
|
|
|
lifetime = addrconf_timeout_fixup(ntohl(rinfo->lifetime), HZ);
|
2006-03-21 01:06:24 +00:00
|
|
|
|
|
|
|
if (rinfo->length == 3)
|
|
|
|
prefix = (struct in6_addr *)rinfo->prefix;
|
|
|
|
else {
|
|
|
|
/* this function is safe */
|
|
|
|
ipv6_addr_prefix(&prefix_buf,
|
|
|
|
(struct in6_addr *)rinfo->prefix,
|
|
|
|
rinfo->prefix_len);
|
|
|
|
prefix = &prefix_buf;
|
|
|
|
}
|
|
|
|
|
2013-11-08 01:56:53 +00:00
|
|
|
if (rinfo->prefix_len == 0)
|
2018-04-18 00:33:11 +00:00
|
|
|
rt = rt6_get_dflt_router(net, gwaddr, dev);
|
2013-11-08 01:56:53 +00:00
|
|
|
else
|
|
|
|
rt = rt6_get_route_info(net, prefix, rinfo->prefix_len,
|
2016-10-24 17:52:35 +00:00
|
|
|
gwaddr, dev);
|
2006-03-21 01:06:24 +00:00
|
|
|
|
|
|
|
if (rt && !lifetime) {
|
2020-04-27 20:56:45 +00:00
|
|
|
ip6_del_rt(net, rt, false);
|
2006-03-21 01:06:24 +00:00
|
|
|
rt = NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (!rt && lifetime)
|
2016-10-24 17:52:35 +00:00
|
|
|
rt = rt6_add_route_info(net, prefix, rinfo->prefix_len, gwaddr,
|
|
|
|
dev, pref);
|
2006-03-21 01:06:24 +00:00
|
|
|
else if (rt)
|
2018-04-18 22:38:59 +00:00
|
|
|
rt->fib6_flags = RTF_ROUTEINFO |
|
|
|
|
(rt->fib6_flags & ~RTF_PREF_MASK) | RTF_PREF(pref);
|
2006-03-21 01:06:24 +00:00
|
|
|
|
|
|
|
if (rt) {
|
2012-04-06 00:13:10 +00:00
|
|
|
if (!addrconf_finite_timeout(lifetime))
|
2018-04-18 00:33:17 +00:00
|
|
|
fib6_clean_expires(rt);
|
2012-04-06 00:13:10 +00:00
|
|
|
else
|
2018-04-18 00:33:17 +00:00
|
|
|
fib6_set_expires(rt, jiffies + HZ * lifetime);
|
2012-04-06 00:13:10 +00:00
|
|
|
|
2018-04-18 00:33:25 +00:00
|
|
|
fib6_info_release(rt);
|
2006-03-21 01:06:24 +00:00
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2018-04-18 00:33:12 +00:00
|
|
|
/*
|
|
|
|
* Misc support functions
|
|
|
|
*/
|
|
|
|
|
|
|
|
/* called with rcu_lock held */
|
2019-04-16 21:36:04 +00:00
|
|
|
static struct net_device *ip6_rt_get_dev_rcu(const struct fib6_result *res)
|
2018-04-18 00:33:12 +00:00
|
|
|
{
|
2019-04-16 21:36:04 +00:00
|
|
|
struct net_device *dev = res->nh->fib_nh_dev;
|
2018-04-18 00:33:12 +00:00
|
|
|
|
2019-04-16 21:36:11 +00:00
|
|
|
if (res->fib6_flags & (RTF_LOCAL | RTF_ANYCAST)) {
|
2018-04-18 00:33:12 +00:00
|
|
|
/* for copies of local routes, dst->dev needs to be the
|
|
|
|
* device if it is a master device, the master device if
|
|
|
|
* device is enslaved, and the loopback as the default
|
|
|
|
*/
|
|
|
|
if (netif_is_l3_slave(dev) &&
|
2019-04-16 21:36:11 +00:00
|
|
|
!rt6_need_strict(&res->f6i->fib6_dst.addr))
|
2018-04-18 00:33:12 +00:00
|
|
|
dev = l3mdev_master_dev_rcu(dev);
|
|
|
|
else if (!netif_is_l3_master(dev))
|
|
|
|
dev = dev_net(dev)->loopback_dev;
|
|
|
|
/* last case is netif_is_l3_master(dev) is true in which
|
|
|
|
* case we want dev returned to be dev
|
|
|
|
*/
|
|
|
|
}
|
|
|
|
|
|
|
|
return dev;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:15 +00:00
|
|
|
static const int fib6_prop[RTN_MAX + 1] = {
|
|
|
|
[RTN_UNSPEC] = 0,
|
|
|
|
[RTN_UNICAST] = 0,
|
|
|
|
[RTN_LOCAL] = 0,
|
|
|
|
[RTN_BROADCAST] = 0,
|
|
|
|
[RTN_ANYCAST] = 0,
|
|
|
|
[RTN_MULTICAST] = 0,
|
|
|
|
[RTN_BLACKHOLE] = -EINVAL,
|
|
|
|
[RTN_UNREACHABLE] = -EHOSTUNREACH,
|
|
|
|
[RTN_PROHIBIT] = -EACCES,
|
|
|
|
[RTN_THROW] = -EAGAIN,
|
|
|
|
[RTN_NAT] = -EINVAL,
|
|
|
|
[RTN_XRESOLVE] = -EINVAL,
|
|
|
|
};
|
|
|
|
|
|
|
|
static int ip6_rt_type_to_error(u8 fib6_type)
|
|
|
|
{
|
|
|
|
return fib6_prop[fib6_type];
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static unsigned short fib6_info_dst_flags(struct fib6_info *rt)
|
2018-04-18 00:33:20 +00:00
|
|
|
{
|
|
|
|
unsigned short flags = 0;
|
|
|
|
|
|
|
|
if (rt->dst_nocount)
|
|
|
|
flags |= DST_NOCOUNT;
|
|
|
|
if (rt->dst_nopolicy)
|
|
|
|
flags |= DST_NOPOLICY;
|
|
|
|
|
|
|
|
return flags;
|
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:11 +00:00
|
|
|
static void ip6_rt_init_dst_reject(struct rt6_info *rt, u8 fib6_type)
|
2018-04-18 00:33:15 +00:00
|
|
|
{
|
2019-04-16 21:36:11 +00:00
|
|
|
rt->dst.error = ip6_rt_type_to_error(fib6_type);
|
2018-04-18 00:33:15 +00:00
|
|
|
|
2019-04-16 21:36:11 +00:00
|
|
|
switch (fib6_type) {
|
2018-04-18 00:33:15 +00:00
|
|
|
case RTN_BLACKHOLE:
|
|
|
|
rt->dst.output = dst_discard_out;
|
|
|
|
rt->dst.input = dst_discard;
|
|
|
|
break;
|
|
|
|
case RTN_PROHIBIT:
|
|
|
|
rt->dst.output = ip6_pkt_prohibit_out;
|
|
|
|
rt->dst.input = ip6_pkt_prohibit;
|
|
|
|
break;
|
|
|
|
case RTN_THROW:
|
|
|
|
case RTN_UNREACHABLE:
|
|
|
|
default:
|
|
|
|
rt->dst.output = ip6_pkt_discard_out;
|
|
|
|
rt->dst.input = ip6_pkt_discard;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:04 +00:00
|
|
|
static void ip6_rt_init_dst(struct rt6_info *rt, const struct fib6_result *res)
|
2018-04-18 00:33:15 +00:00
|
|
|
{
|
2019-04-16 21:36:11 +00:00
|
|
|
struct fib6_info *f6i = res->f6i;
|
2019-04-16 21:36:04 +00:00
|
|
|
|
2019-04-16 21:36:11 +00:00
|
|
|
if (res->fib6_flags & RTF_REJECT) {
|
|
|
|
ip6_rt_init_dst_reject(rt, res->fib6_type);
|
2018-04-18 00:33:15 +00:00
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
rt->dst.error = 0;
|
|
|
|
rt->dst.output = ip6_output;
|
|
|
|
|
2019-04-16 21:36:11 +00:00
|
|
|
if (res->fib6_type == RTN_LOCAL || res->fib6_type == RTN_ANYCAST) {
|
2018-04-18 00:33:15 +00:00
|
|
|
rt->dst.input = ip6_input;
|
2019-04-16 21:36:11 +00:00
|
|
|
} else if (ipv6_addr_type(&f6i->fib6_dst.addr) & IPV6_ADDR_MULTICAST) {
|
2018-04-18 00:33:15 +00:00
|
|
|
rt->dst.input = ip6_mc_input;
|
|
|
|
} else {
|
|
|
|
rt->dst.input = ip6_forward;
|
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:04 +00:00
|
|
|
if (res->nh->fib_nh_lws) {
|
|
|
|
rt->dst.lwtstate = lwtstate_get(res->nh->fib_nh_lws);
|
2018-04-18 00:33:15 +00:00
|
|
|
lwtunnel_set_redirect(&rt->dst);
|
|
|
|
}
|
|
|
|
|
|
|
|
rt->dst.lastuse = jiffies;
|
|
|
|
}
|
|
|
|
|
2018-07-22 03:56:32 +00:00
|
|
|
/* Caller must already hold reference to @from */
|
2018-04-18 00:33:26 +00:00
|
|
|
static void rt6_set_from(struct rt6_info *rt, struct fib6_info *from)
|
2018-04-18 00:33:12 +00:00
|
|
|
{
|
|
|
|
rt->rt6i_flags &= ~RTF_EXPIRES;
|
2018-04-20 22:38:02 +00:00
|
|
|
rcu_assign_pointer(rt->from, from);
|
2018-10-05 03:07:53 +00:00
|
|
|
ip_dst_init_metrics(&rt->dst, from->fib6_metrics);
|
2018-04-18 00:33:12 +00:00
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:04 +00:00
|
|
|
/* Caller must already hold reference to f6i in result */
|
|
|
|
static void ip6_rt_copy_init(struct rt6_info *rt, const struct fib6_result *res)
|
2018-04-18 00:33:12 +00:00
|
|
|
{
|
2019-04-16 21:36:04 +00:00
|
|
|
const struct fib6_nh *nh = res->nh;
|
|
|
|
const struct net_device *dev = nh->fib_nh_dev;
|
|
|
|
struct fib6_info *f6i = res->f6i;
|
2018-04-18 22:39:05 +00:00
|
|
|
|
2019-04-16 21:36:04 +00:00
|
|
|
ip6_rt_init_dst(rt, res);
|
2018-04-18 00:33:15 +00:00
|
|
|
|
2019-04-16 21:36:04 +00:00
|
|
|
rt->rt6i_dst = f6i->fib6_dst;
|
2018-04-18 22:39:05 +00:00
|
|
|
rt->rt6i_idev = dev ? in6_dev_get(dev) : NULL;
|
2019-04-16 21:36:11 +00:00
|
|
|
rt->rt6i_flags = res->fib6_flags;
|
2019-04-16 21:36:04 +00:00
|
|
|
if (nh->fib_nh_gw_family) {
|
|
|
|
rt->rt6i_gateway = nh->fib_nh_gw6;
|
2019-03-28 03:53:52 +00:00
|
|
|
rt->rt6i_flags |= RTF_GATEWAY;
|
|
|
|
}
|
2019-04-16 21:36:04 +00:00
|
|
|
rt6_set_from(rt, f6i);
|
2018-04-18 00:33:12 +00:00
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
2019-04-16 21:36:04 +00:00
|
|
|
rt->rt6i_src = f6i->fib6_src;
|
2018-04-18 00:33:12 +00:00
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
2014-10-20 20:42:43 +00:00
|
|
|
static struct fib6_node* fib6_backtrack(struct fib6_node *fn,
|
|
|
|
struct in6_addr *saddr)
|
|
|
|
{
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
struct fib6_node *pn, *sn;
|
2014-10-20 20:42:43 +00:00
|
|
|
while (1) {
|
|
|
|
if (fn->fn_flags & RTN_TL_ROOT)
|
|
|
|
return NULL;
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
pn = rcu_dereference(fn->parent);
|
|
|
|
sn = FIB6_SUBTREE(pn);
|
|
|
|
if (sn && sn != fn)
|
2018-05-10 03:34:19 +00:00
|
|
|
fn = fib6_node_lookup(sn, NULL, saddr);
|
2014-10-20 20:42:43 +00:00
|
|
|
else
|
|
|
|
fn = pn;
|
|
|
|
if (fn->fn_flags & RTN_RTINFO)
|
|
|
|
return fn;
|
|
|
|
}
|
|
|
|
}
|
2006-08-05 06:20:06 +00:00
|
|
|
|
2019-03-20 16:24:50 +00:00
|
|
|
static bool ip6_hold_safe(struct net *net, struct rt6_info **prt)
|
2017-10-06 19:06:06 +00:00
|
|
|
{
|
|
|
|
struct rt6_info *rt = *prt;
|
|
|
|
|
|
|
|
if (dst_hold_safe(&rt->dst))
|
|
|
|
return true;
|
2019-03-20 16:24:50 +00:00
|
|
|
if (net) {
|
2017-10-06 19:06:06 +00:00
|
|
|
rt = net->ipv6.ip6_null_entry;
|
|
|
|
dst_hold(&rt->dst);
|
|
|
|
} else {
|
|
|
|
rt = NULL;
|
|
|
|
}
|
|
|
|
*prt = rt;
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:19 +00:00
|
|
|
/* called with rcu_lock held */
|
2019-04-16 21:36:02 +00:00
|
|
|
static struct rt6_info *ip6_create_rt_rcu(const struct fib6_result *res)
|
2018-04-18 00:33:19 +00:00
|
|
|
{
|
2019-04-16 21:36:02 +00:00
|
|
|
struct net_device *dev = res->nh->fib_nh_dev;
|
|
|
|
struct fib6_info *f6i = res->f6i;
|
|
|
|
unsigned short flags;
|
2018-04-18 00:33:19 +00:00
|
|
|
struct rt6_info *nrt;
|
|
|
|
|
2019-04-16 21:36:02 +00:00
|
|
|
if (!fib6_info_hold_safe(f6i))
|
2019-03-20 06:45:48 +00:00
|
|
|
goto fallback;
|
2018-07-22 03:56:32 +00:00
|
|
|
|
2019-04-16 21:36:02 +00:00
|
|
|
flags = fib6_info_dst_flags(f6i);
|
2018-04-18 00:33:25 +00:00
|
|
|
nrt = ip6_dst_alloc(dev_net(dev), dev, flags);
|
2019-03-20 06:45:48 +00:00
|
|
|
if (!nrt) {
|
2019-04-16 21:36:02 +00:00
|
|
|
fib6_info_release(f6i);
|
2019-03-20 06:45:48 +00:00
|
|
|
goto fallback;
|
|
|
|
}
|
2018-04-18 00:33:19 +00:00
|
|
|
|
2019-04-16 21:36:04 +00:00
|
|
|
ip6_rt_copy_init(nrt, res);
|
2019-03-20 06:45:48 +00:00
|
|
|
return nrt;
|
|
|
|
|
|
|
|
fallback:
|
|
|
|
nrt = dev_net(dev)->ipv6.ip6_null_entry;
|
|
|
|
dst_hold(&nrt->dst);
|
2018-04-18 00:33:19 +00:00
|
|
|
return nrt;
|
|
|
|
}
|
|
|
|
|
2020-06-23 16:42:32 +00:00
|
|
|
INDIRECT_CALLABLE_SCOPE struct rt6_info *ip6_pol_route_lookup(struct net *net,
|
2008-03-04 21:48:30 +00:00
|
|
|
struct fib6_table *table,
|
2018-03-02 16:32:17 +00:00
|
|
|
struct flowi6 *fl6,
|
|
|
|
const struct sk_buff *skb,
|
|
|
|
int flags)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2019-04-16 21:35:59 +00:00
|
|
|
struct fib6_result res = {};
|
2005-04-16 22:20:36 +00:00
|
|
|
struct fib6_node *fn;
|
2018-04-18 00:33:23 +00:00
|
|
|
struct rt6_info *rt;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-03-30 00:44:57 +00:00
|
|
|
if (fl6->flowi6_flags & FLOWI_FLAG_SKIP_NH_OIF)
|
|
|
|
flags &= ~RT6_LOOKUP_F_IFACE;
|
|
|
|
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_lock();
|
2018-05-10 03:34:19 +00:00
|
|
|
fn = fib6_node_lookup(&table->tb6_root, &fl6->daddr, &fl6->saddr);
|
2006-08-05 06:20:06 +00:00
|
|
|
restart:
|
2019-04-16 21:35:59 +00:00
|
|
|
res.f6i = rcu_dereference(fn->leaf);
|
|
|
|
if (!res.f6i)
|
|
|
|
res.f6i = net->ipv6.fib6_null_entry;
|
2019-04-09 21:41:16 +00:00
|
|
|
else
|
2019-04-16 21:36:07 +00:00
|
|
|
rt6_device_match(net, &res, &fl6->saddr, fl6->flowi6_oif,
|
|
|
|
flags);
|
2019-04-09 21:41:16 +00:00
|
|
|
|
2019-04-16 21:35:59 +00:00
|
|
|
if (res.f6i == net->ipv6.fib6_null_entry) {
|
2014-10-20 20:42:43 +00:00
|
|
|
fn = fib6_backtrack(fn, &fl6->saddr);
|
|
|
|
if (fn)
|
|
|
|
goto restart;
|
2017-10-06 19:06:03 +00:00
|
|
|
|
2019-04-09 21:41:16 +00:00
|
|
|
rt = net->ipv6.ip6_null_entry;
|
|
|
|
dst_hold(&rt->dst);
|
|
|
|
goto out;
|
2019-06-04 03:19:52 +00:00
|
|
|
} else if (res.fib6_flags & RTF_REJECT) {
|
|
|
|
goto do_create;
|
2019-04-09 21:41:16 +00:00
|
|
|
}
|
2017-10-06 19:06:06 +00:00
|
|
|
|
2019-04-16 21:35:59 +00:00
|
|
|
fib6_select_path(net, &res, fl6, fl6->flowi6_oif,
|
|
|
|
fl6->flowi6_oif != 0, skb, flags);
|
|
|
|
|
2017-10-06 19:06:03 +00:00
|
|
|
/* Search through exception table */
|
2019-04-16 21:36:00 +00:00
|
|
|
rt = rt6_find_cached_rt(&res, &fl6->daddr, &fl6->saddr);
|
2018-04-18 00:33:23 +00:00
|
|
|
if (rt) {
|
2019-03-20 16:24:50 +00:00
|
|
|
if (ip6_hold_safe(net, &rt))
|
2018-04-18 00:33:19 +00:00
|
|
|
dst_use_noref(&rt->dst, jiffies);
|
2018-04-18 00:33:23 +00:00
|
|
|
} else {
|
2019-06-04 03:19:52 +00:00
|
|
|
do_create:
|
2019-04-16 21:36:02 +00:00
|
|
|
rt = ip6_create_rt_rcu(&res);
|
2018-04-18 00:33:19 +00:00
|
|
|
}
|
2015-11-19 20:24:22 +00:00
|
|
|
|
2019-04-09 21:41:16 +00:00
|
|
|
out:
|
2019-04-16 21:36:09 +00:00
|
|
|
trace_fib6_table_lookup(net, &res, table, fl6);
|
2019-04-09 21:41:16 +00:00
|
|
|
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_unlock();
|
2015-11-19 20:24:22 +00:00
|
|
|
|
2006-08-05 06:20:06 +00:00
|
|
|
return rt;
|
|
|
|
}
|
|
|
|
|
2014-08-24 20:53:10 +00:00
|
|
|
struct dst_entry *ip6_route_lookup(struct net *net, struct flowi6 *fl6,
|
2018-03-02 16:32:17 +00:00
|
|
|
const struct sk_buff *skb, int flags)
|
2011-09-05 14:05:44 +00:00
|
|
|
{
|
2018-03-02 16:32:17 +00:00
|
|
|
return fib6_rule_lookup(net, fl6, skb, flags, ip6_pol_route_lookup);
|
2011-09-05 14:05:44 +00:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(ip6_route_lookup);
|
|
|
|
|
[IPV6]: Make address arguments const.
- net/ipv6/addrconf.c:
ipv6_get_ifaddr(), ipv6_dev_get_saddr()
- net/ipv6/mcast.c:
ipv6_sock_mc_join(), ipv6_sock_mc_drop(),
inet6_mc_check(),
ipv6_dev_mc_inc(), __ipv6_dev_mc_dec(), ipv6_dev_mc_dec(),
ipv6_chk_mcast_addr()
- net/ipv6/route.c:
rt6_lookup(), icmp6_dst_alloc()
- net/ipv6/ip6_output.c:
ip6_nd_hdr()
- net/ipv6/ndisc.c:
ndisc_send_ns(), ndisc_send_rs(), ndisc_send_redirect(),
ndisc_get_neigh(), __ndisc_send()
Signed-off-by: YOSHIFUJI Hideaki <yoshfuji@linux-ipv6.org>
2008-04-10 06:42:10 +00:00
|
|
|
struct rt6_info *rt6_lookup(struct net *net, const struct in6_addr *daddr,
|
2018-03-02 16:32:17 +00:00
|
|
|
const struct in6_addr *saddr, int oif,
|
|
|
|
const struct sk_buff *skb, int strict)
|
2006-08-05 06:20:06 +00:00
|
|
|
{
|
2011-03-12 21:22:43 +00:00
|
|
|
struct flowi6 fl6 = {
|
|
|
|
.flowi6_oif = oif,
|
|
|
|
.daddr = *daddr,
|
2006-08-05 06:20:06 +00:00
|
|
|
};
|
|
|
|
struct dst_entry *dst;
|
2006-08-24 00:25:05 +00:00
|
|
|
int flags = strict ? RT6_LOOKUP_F_IFACE : 0;
|
2006-08-05 06:20:06 +00:00
|
|
|
|
2006-10-13 22:01:03 +00:00
|
|
|
if (saddr) {
|
2011-03-12 21:22:43 +00:00
|
|
|
memcpy(&fl6.saddr, saddr, sizeof(*saddr));
|
2006-10-13 22:01:03 +00:00
|
|
|
flags |= RT6_LOOKUP_F_HAS_SADDR;
|
|
|
|
}
|
|
|
|
|
2018-03-02 16:32:17 +00:00
|
|
|
dst = fib6_rule_lookup(net, &fl6, skb, flags, ip6_pol_route_lookup);
|
2006-08-05 06:20:06 +00:00
|
|
|
if (dst->error == 0)
|
|
|
|
return (struct rt6_info *) dst;
|
|
|
|
|
|
|
|
dst_release(dst);
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
return NULL;
|
|
|
|
}
|
2007-02-22 13:05:40 +00:00
|
|
|
EXPORT_SYMBOL(rt6_lookup);
|
|
|
|
|
2006-08-05 06:20:06 +00:00
|
|
|
/* ip6_ins_rt is called with FREE table->tb6_lock.
|
2017-06-17 17:42:33 +00:00
|
|
|
* It takes new route entry, the addition fails by any reason the
|
|
|
|
* route is released.
|
|
|
|
* Caller must hold dst before calling it.
|
2005-04-16 22:20:36 +00:00
|
|
|
*/
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static int __ip6_ins_rt(struct fib6_info *rt, struct nl_info *info,
|
2017-05-21 16:12:04 +00:00
|
|
|
struct netlink_ext_ack *extack)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
|
|
|
int err;
|
2006-08-05 06:20:06 +00:00
|
|
|
struct fib6_table *table;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-04-18 22:38:59 +00:00
|
|
|
table = rt->fib6_table;
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
spin_lock_bh(&table->tb6_lock);
|
2018-04-18 00:33:16 +00:00
|
|
|
err = fib6_add(&table->tb6_root, rt, info, extack);
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
spin_unlock_bh(&table->tb6_lock);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
int ip6_ins_rt(struct net *net, struct fib6_info *rt)
|
2006-08-22 07:00:45 +00:00
|
|
|
{
|
2018-04-18 00:33:11 +00:00
|
|
|
struct nl_info info = { .nl_net = net, };
|
2015-01-05 22:57:44 +00:00
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
return __ip6_ins_rt(rt, &info, NULL);
|
2006-08-22 07:00:45 +00:00
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:01 +00:00
|
|
|
static struct rt6_info *ip6_rt_cache_alloc(const struct fib6_result *res,
|
2015-05-23 03:55:59 +00:00
|
|
|
const struct in6_addr *daddr,
|
|
|
|
const struct in6_addr *saddr)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2019-04-16 21:36:01 +00:00
|
|
|
struct fib6_info *f6i = res->f6i;
|
2017-08-17 19:17:20 +00:00
|
|
|
struct net_device *dev;
|
2005-04-16 22:20:36 +00:00
|
|
|
struct rt6_info *rt;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Clone the route.
|
|
|
|
*/
|
|
|
|
|
2019-04-16 21:36:01 +00:00
|
|
|
if (!fib6_info_hold_safe(f6i))
|
2018-07-22 03:56:32 +00:00
|
|
|
return NULL;
|
|
|
|
|
2019-04-16 21:36:04 +00:00
|
|
|
dev = ip6_rt_get_dev_rcu(res);
|
2018-04-18 00:33:25 +00:00
|
|
|
rt = ip6_dst_alloc(dev_net(dev), dev, 0);
|
2018-07-22 03:56:32 +00:00
|
|
|
if (!rt) {
|
2019-04-16 21:36:01 +00:00
|
|
|
fib6_info_release(f6i);
|
2015-05-23 03:56:05 +00:00
|
|
|
return NULL;
|
2018-07-22 03:56:32 +00:00
|
|
|
}
|
2015-05-23 03:56:05 +00:00
|
|
|
|
2019-04-16 21:36:04 +00:00
|
|
|
ip6_rt_copy_init(rt, res);
|
2015-05-23 03:56:05 +00:00
|
|
|
rt->rt6i_flags |= RTF_CACHE;
|
|
|
|
rt->rt6i_dst.addr = *daddr;
|
|
|
|
rt->rt6i_dst.plen = 128;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-04-16 21:36:01 +00:00
|
|
|
if (!rt6_is_gw_or_nonexthop(res)) {
|
|
|
|
if (f6i->fib6_dst.plen != 128 &&
|
|
|
|
ipv6_addr_equal(&f6i->fib6_dst.addr, daddr))
|
2015-05-23 03:56:05 +00:00
|
|
|
rt->rt6i_flags |= RTF_ANYCAST;
|
2005-04-16 22:20:36 +00:00
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
2015-05-23 03:56:05 +00:00
|
|
|
if (rt->rt6i_src.plen && saddr) {
|
|
|
|
rt->rt6i_src.addr = *saddr;
|
|
|
|
rt->rt6i_src.plen = 128;
|
2015-05-23 03:55:59 +00:00
|
|
|
}
|
2015-05-23 03:56:05 +00:00
|
|
|
#endif
|
2006-03-21 00:55:51 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2006-03-21 00:55:51 +00:00
|
|
|
return rt;
|
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-04-16 21:36:03 +00:00
|
|
|
static struct rt6_info *ip6_rt_pcpu_alloc(const struct fib6_result *res)
|
2015-05-23 03:56:06 +00:00
|
|
|
{
|
2019-04-16 21:36:03 +00:00
|
|
|
struct fib6_info *f6i = res->f6i;
|
|
|
|
unsigned short flags = fib6_info_dst_flags(f6i);
|
2017-08-17 19:17:20 +00:00
|
|
|
struct net_device *dev;
|
2015-05-23 03:56:06 +00:00
|
|
|
struct rt6_info *pcpu_rt;
|
|
|
|
|
2019-04-16 21:36:03 +00:00
|
|
|
if (!fib6_info_hold_safe(f6i))
|
2018-07-22 03:56:32 +00:00
|
|
|
return NULL;
|
|
|
|
|
2017-08-17 19:17:20 +00:00
|
|
|
rcu_read_lock();
|
2019-04-16 21:36:04 +00:00
|
|
|
dev = ip6_rt_get_dev_rcu(res);
|
2020-05-08 14:34:14 +00:00
|
|
|
pcpu_rt = ip6_dst_alloc(dev_net(dev), dev, flags | DST_NOCOUNT);
|
2017-08-17 19:17:20 +00:00
|
|
|
rcu_read_unlock();
|
2018-07-22 03:56:32 +00:00
|
|
|
if (!pcpu_rt) {
|
2019-04-16 21:36:03 +00:00
|
|
|
fib6_info_release(f6i);
|
2015-05-23 03:56:06 +00:00
|
|
|
return NULL;
|
2018-07-22 03:56:32 +00:00
|
|
|
}
|
2019-04-16 21:36:04 +00:00
|
|
|
ip6_rt_copy_init(pcpu_rt, res);
|
2015-05-23 03:56:06 +00:00
|
|
|
pcpu_rt->rt6i_flags |= RTF_PCPU;
|
ipv6: Use global sernum for dst validation with nexthop objects
Nik reported a bug with pcpu dst cache when nexthop objects are
used illustrated by the following:
$ ip netns add foo
$ ip -netns foo li set lo up
$ ip -netns foo addr add 2001:db8:11::1/128 dev lo
$ ip netns exec foo sysctl net.ipv6.conf.all.forwarding=1
$ ip li add veth1 type veth peer name veth2
$ ip li set veth1 up
$ ip addr add 2001:db8:10::1/64 dev veth1
$ ip li set dev veth2 netns foo
$ ip -netns foo li set veth2 up
$ ip -netns foo addr add 2001:db8:10::2/64 dev veth2
$ ip -6 nexthop add id 100 via 2001:db8:10::2 dev veth1
$ ip -6 route add 2001:db8:11::1/128 nhid 100
Create a pcpu entry on cpu 0:
$ taskset -a -c 0 ip -6 route get 2001:db8:11::1
Re-add the route entry:
$ ip -6 ro del 2001:db8:11::1
$ ip -6 route add 2001:db8:11::1/128 nhid 100
Route get on cpu 0 returns the stale pcpu:
$ taskset -a -c 0 ip -6 route get 2001:db8:11::1
RTNETLINK answers: Network is unreachable
While cpu 1 works:
$ taskset -a -c 1 ip -6 route get 2001:db8:11::1
2001:db8:11::1 from :: via 2001:db8:10::2 dev veth1 src 2001:db8:10::1 metric 1024 pref medium
Conversion of FIB entries to work with external nexthop objects
missed an important difference between IPv4 and IPv6 - how dst
entries are invalidated when the FIB changes. IPv4 has a per-network
namespace generation id (rt_genid) that is bumped on changes to the FIB.
Checking if a dst_entry is still valid means comparing rt_genid in the
rtable to the current value of rt_genid for the namespace.
IPv6 also has a per network namespace counter, fib6_sernum, but the
count is saved per fib6_node. With the per-node counter only dst_entries
based on fib entries under the node are invalidated when changes are
made to the routes - limiting the scope of invalidations. IPv6 uses a
reference in the rt6_info, 'from', to track the corresponding fib entry
used to create the dst_entry. When validating a dst_entry, the 'from'
is used to backtrack to the fib6_node and check the sernum of it to the
cookie passed to the dst_check operation.
With the inline format (nexthop definition inline with the fib6_info),
dst_entries cached in the fib6_nh have a 1:1 correlation between fib
entries, nexthop data and dst_entries. With external nexthops, IPv6
looks more like IPv4 which means multiple fib entries across disparate
fib6_nodes can all reference the same fib6_nh. That means validation
of dst_entries based on external nexthops needs to use the IPv4 format
- the per-network namespace counter.
Add sernum to rt6_info and set it when creating a pcpu dst entry. Update
rt6_get_cookie to return sernum if it is set and update dst_check for
IPv6 to look for sernum set and based the check on it if so. Finally,
rt6_get_pcpu_route needs to validate the cached entry before returning
a pcpu entry (similar to the rt_cache_valid calls in __mkroute_input and
__mkroute_output for IPv4).
This problem only affects routes using the new, external nexthops.
Thanks to the kbuild test robot for catching the IS_ENABLED needed
around rt_genid_ipv6 before I sent this out.
Fixes: 5b98324ebe29 ("ipv6: Allow routes to use nexthop objects")
Reported-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Signed-off-by: David Ahern <dsahern@kernel.org>
Reviewed-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Tested-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2020-05-01 14:53:08 +00:00
|
|
|
|
|
|
|
if (f6i->nh)
|
|
|
|
pcpu_rt->sernum = rt_genid_ipv6(dev_net(dev));
|
|
|
|
|
2015-05-23 03:56:06 +00:00
|
|
|
return pcpu_rt;
|
|
|
|
}
|
|
|
|
|
ipv6: Use global sernum for dst validation with nexthop objects
Nik reported a bug with pcpu dst cache when nexthop objects are
used illustrated by the following:
$ ip netns add foo
$ ip -netns foo li set lo up
$ ip -netns foo addr add 2001:db8:11::1/128 dev lo
$ ip netns exec foo sysctl net.ipv6.conf.all.forwarding=1
$ ip li add veth1 type veth peer name veth2
$ ip li set veth1 up
$ ip addr add 2001:db8:10::1/64 dev veth1
$ ip li set dev veth2 netns foo
$ ip -netns foo li set veth2 up
$ ip -netns foo addr add 2001:db8:10::2/64 dev veth2
$ ip -6 nexthop add id 100 via 2001:db8:10::2 dev veth1
$ ip -6 route add 2001:db8:11::1/128 nhid 100
Create a pcpu entry on cpu 0:
$ taskset -a -c 0 ip -6 route get 2001:db8:11::1
Re-add the route entry:
$ ip -6 ro del 2001:db8:11::1
$ ip -6 route add 2001:db8:11::1/128 nhid 100
Route get on cpu 0 returns the stale pcpu:
$ taskset -a -c 0 ip -6 route get 2001:db8:11::1
RTNETLINK answers: Network is unreachable
While cpu 1 works:
$ taskset -a -c 1 ip -6 route get 2001:db8:11::1
2001:db8:11::1 from :: via 2001:db8:10::2 dev veth1 src 2001:db8:10::1 metric 1024 pref medium
Conversion of FIB entries to work with external nexthop objects
missed an important difference between IPv4 and IPv6 - how dst
entries are invalidated when the FIB changes. IPv4 has a per-network
namespace generation id (rt_genid) that is bumped on changes to the FIB.
Checking if a dst_entry is still valid means comparing rt_genid in the
rtable to the current value of rt_genid for the namespace.
IPv6 also has a per network namespace counter, fib6_sernum, but the
count is saved per fib6_node. With the per-node counter only dst_entries
based on fib entries under the node are invalidated when changes are
made to the routes - limiting the scope of invalidations. IPv6 uses a
reference in the rt6_info, 'from', to track the corresponding fib entry
used to create the dst_entry. When validating a dst_entry, the 'from'
is used to backtrack to the fib6_node and check the sernum of it to the
cookie passed to the dst_check operation.
With the inline format (nexthop definition inline with the fib6_info),
dst_entries cached in the fib6_nh have a 1:1 correlation between fib
entries, nexthop data and dst_entries. With external nexthops, IPv6
looks more like IPv4 which means multiple fib entries across disparate
fib6_nodes can all reference the same fib6_nh. That means validation
of dst_entries based on external nexthops needs to use the IPv4 format
- the per-network namespace counter.
Add sernum to rt6_info and set it when creating a pcpu dst entry. Update
rt6_get_cookie to return sernum if it is set and update dst_check for
IPv6 to look for sernum set and based the check on it if so. Finally,
rt6_get_pcpu_route needs to validate the cached entry before returning
a pcpu entry (similar to the rt_cache_valid calls in __mkroute_input and
__mkroute_output for IPv4).
This problem only affects routes using the new, external nexthops.
Thanks to the kbuild test robot for catching the IS_ENABLED needed
around rt_genid_ipv6 before I sent this out.
Fixes: 5b98324ebe29 ("ipv6: Allow routes to use nexthop objects")
Reported-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Signed-off-by: David Ahern <dsahern@kernel.org>
Reviewed-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Tested-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2020-05-01 14:53:08 +00:00
|
|
|
static bool rt6_is_valid(const struct rt6_info *rt6)
|
|
|
|
{
|
|
|
|
return rt6->sernum == rt_genid_ipv6(dev_net(rt6->dst.dev));
|
|
|
|
}
|
|
|
|
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
/* It should be called with rcu_read_lock() acquired */
|
2019-04-16 21:36:03 +00:00
|
|
|
static struct rt6_info *rt6_get_pcpu_route(const struct fib6_result *res)
|
2015-05-23 03:56:06 +00:00
|
|
|
{
|
2019-06-01 01:11:25 +00:00
|
|
|
struct rt6_info *pcpu_rt;
|
2015-05-23 03:56:06 +00:00
|
|
|
|
2019-06-01 01:11:25 +00:00
|
|
|
pcpu_rt = this_cpu_read(*res->nh->rt6i_pcpu);
|
2015-05-23 03:56:06 +00:00
|
|
|
|
ipv6: Use global sernum for dst validation with nexthop objects
Nik reported a bug with pcpu dst cache when nexthop objects are
used illustrated by the following:
$ ip netns add foo
$ ip -netns foo li set lo up
$ ip -netns foo addr add 2001:db8:11::1/128 dev lo
$ ip netns exec foo sysctl net.ipv6.conf.all.forwarding=1
$ ip li add veth1 type veth peer name veth2
$ ip li set veth1 up
$ ip addr add 2001:db8:10::1/64 dev veth1
$ ip li set dev veth2 netns foo
$ ip -netns foo li set veth2 up
$ ip -netns foo addr add 2001:db8:10::2/64 dev veth2
$ ip -6 nexthop add id 100 via 2001:db8:10::2 dev veth1
$ ip -6 route add 2001:db8:11::1/128 nhid 100
Create a pcpu entry on cpu 0:
$ taskset -a -c 0 ip -6 route get 2001:db8:11::1
Re-add the route entry:
$ ip -6 ro del 2001:db8:11::1
$ ip -6 route add 2001:db8:11::1/128 nhid 100
Route get on cpu 0 returns the stale pcpu:
$ taskset -a -c 0 ip -6 route get 2001:db8:11::1
RTNETLINK answers: Network is unreachable
While cpu 1 works:
$ taskset -a -c 1 ip -6 route get 2001:db8:11::1
2001:db8:11::1 from :: via 2001:db8:10::2 dev veth1 src 2001:db8:10::1 metric 1024 pref medium
Conversion of FIB entries to work with external nexthop objects
missed an important difference between IPv4 and IPv6 - how dst
entries are invalidated when the FIB changes. IPv4 has a per-network
namespace generation id (rt_genid) that is bumped on changes to the FIB.
Checking if a dst_entry is still valid means comparing rt_genid in the
rtable to the current value of rt_genid for the namespace.
IPv6 also has a per network namespace counter, fib6_sernum, but the
count is saved per fib6_node. With the per-node counter only dst_entries
based on fib entries under the node are invalidated when changes are
made to the routes - limiting the scope of invalidations. IPv6 uses a
reference in the rt6_info, 'from', to track the corresponding fib entry
used to create the dst_entry. When validating a dst_entry, the 'from'
is used to backtrack to the fib6_node and check the sernum of it to the
cookie passed to the dst_check operation.
With the inline format (nexthop definition inline with the fib6_info),
dst_entries cached in the fib6_nh have a 1:1 correlation between fib
entries, nexthop data and dst_entries. With external nexthops, IPv6
looks more like IPv4 which means multiple fib entries across disparate
fib6_nodes can all reference the same fib6_nh. That means validation
of dst_entries based on external nexthops needs to use the IPv4 format
- the per-network namespace counter.
Add sernum to rt6_info and set it when creating a pcpu dst entry. Update
rt6_get_cookie to return sernum if it is set and update dst_check for
IPv6 to look for sernum set and based the check on it if so. Finally,
rt6_get_pcpu_route needs to validate the cached entry before returning
a pcpu entry (similar to the rt_cache_valid calls in __mkroute_input and
__mkroute_output for IPv4).
This problem only affects routes using the new, external nexthops.
Thanks to the kbuild test robot for catching the IS_ENABLED needed
around rt_genid_ipv6 before I sent this out.
Fixes: 5b98324ebe29 ("ipv6: Allow routes to use nexthop objects")
Reported-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Signed-off-by: David Ahern <dsahern@kernel.org>
Reviewed-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Tested-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2020-05-01 14:53:08 +00:00
|
|
|
if (pcpu_rt && pcpu_rt->sernum && !rt6_is_valid(pcpu_rt)) {
|
|
|
|
struct rt6_info *prev, **p;
|
|
|
|
|
|
|
|
p = this_cpu_ptr(res->nh->rt6i_pcpu);
|
|
|
|
prev = xchg(p, NULL);
|
|
|
|
if (prev) {
|
|
|
|
dst_dev_put(&prev->dst);
|
|
|
|
dst_release(&prev->dst);
|
|
|
|
}
|
|
|
|
|
|
|
|
pcpu_rt = NULL;
|
|
|
|
}
|
|
|
|
|
2015-08-14 18:05:53 +00:00
|
|
|
return pcpu_rt;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:11 +00:00
|
|
|
static struct rt6_info *rt6_make_pcpu_route(struct net *net,
|
2019-04-16 21:36:03 +00:00
|
|
|
const struct fib6_result *res)
|
2015-08-14 18:05:53 +00:00
|
|
|
{
|
|
|
|
struct rt6_info *pcpu_rt, *prev, **p;
|
2015-05-23 03:56:06 +00:00
|
|
|
|
2019-04-16 21:36:03 +00:00
|
|
|
pcpu_rt = ip6_rt_pcpu_alloc(res);
|
2019-06-21 00:36:37 +00:00
|
|
|
if (!pcpu_rt)
|
|
|
|
return NULL;
|
2015-05-23 03:56:06 +00:00
|
|
|
|
2019-05-23 03:27:55 +00:00
|
|
|
p = this_cpu_ptr(res->nh->rt6i_pcpu);
|
2017-10-06 19:06:04 +00:00
|
|
|
prev = cmpxchg(p, NULL, pcpu_rt);
|
2017-10-09 04:07:18 +00:00
|
|
|
BUG_ON(prev);
|
2017-10-06 19:06:04 +00:00
|
|
|
|
2019-05-16 02:39:52 +00:00
|
|
|
if (res->f6i->fib6_destroying) {
|
|
|
|
struct fib6_info *from;
|
|
|
|
|
|
|
|
from = xchg((__force struct fib6_info **)&pcpu_rt->from, NULL);
|
|
|
|
fib6_info_release(from);
|
|
|
|
}
|
|
|
|
|
2015-05-23 03:56:06 +00:00
|
|
|
return pcpu_rt;
|
|
|
|
}
|
|
|
|
|
2017-10-06 19:05:57 +00:00
|
|
|
/* exception hash table implementation
|
|
|
|
*/
|
|
|
|
static DEFINE_SPINLOCK(rt6_exception_lock);
|
|
|
|
|
|
|
|
/* Remove rt6_ex from hash table and free the memory
|
|
|
|
* Caller must hold rt6_exception_lock
|
|
|
|
*/
|
|
|
|
static void rt6_remove_exception(struct rt6_exception_bucket *bucket,
|
|
|
|
struct rt6_exception *rt6_ex)
|
|
|
|
{
|
2019-02-20 17:18:12 +00:00
|
|
|
struct fib6_info *from;
|
2017-10-10 17:01:16 +00:00
|
|
|
struct net *net;
|
2017-10-06 19:06:11 +00:00
|
|
|
|
2017-10-06 19:05:57 +00:00
|
|
|
if (!bucket || !rt6_ex)
|
|
|
|
return;
|
2017-10-10 17:01:16 +00:00
|
|
|
|
|
|
|
net = dev_net(rt6_ex->rt6i->dst.dev);
|
2019-02-20 17:18:12 +00:00
|
|
|
net->ipv6.rt6_stats->fib_rt_cache--;
|
|
|
|
|
|
|
|
/* purge completely the exception to allow releasing the held resources:
|
|
|
|
* some [sk] cache may keep the dst around for unlimited time
|
|
|
|
*/
|
2019-04-28 19:22:25 +00:00
|
|
|
from = xchg((__force struct fib6_info **)&rt6_ex->rt6i->from, NULL);
|
2019-02-20 17:18:12 +00:00
|
|
|
fib6_info_release(from);
|
|
|
|
dst_dev_put(&rt6_ex->rt6i->dst);
|
|
|
|
|
2017-10-06 19:05:57 +00:00
|
|
|
hlist_del_rcu(&rt6_ex->hlist);
|
2018-04-18 00:33:27 +00:00
|
|
|
dst_release(&rt6_ex->rt6i->dst);
|
2017-10-06 19:05:57 +00:00
|
|
|
kfree_rcu(rt6_ex, rcu);
|
|
|
|
WARN_ON_ONCE(!bucket->depth);
|
|
|
|
bucket->depth--;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Remove oldest rt6_ex in bucket and free the memory
|
|
|
|
* Caller must hold rt6_exception_lock
|
|
|
|
*/
|
|
|
|
static void rt6_exception_remove_oldest(struct rt6_exception_bucket *bucket)
|
|
|
|
{
|
|
|
|
struct rt6_exception *rt6_ex, *oldest = NULL;
|
|
|
|
|
|
|
|
if (!bucket)
|
|
|
|
return;
|
|
|
|
|
|
|
|
hlist_for_each_entry(rt6_ex, &bucket->chain, hlist) {
|
|
|
|
if (!oldest || time_before(rt6_ex->stamp, oldest->stamp))
|
|
|
|
oldest = rt6_ex;
|
|
|
|
}
|
|
|
|
rt6_remove_exception(bucket, oldest);
|
|
|
|
}
|
|
|
|
|
|
|
|
static u32 rt6_exception_hash(const struct in6_addr *dst,
|
|
|
|
const struct in6_addr *src)
|
|
|
|
{
|
2021-11-15 17:23:03 +00:00
|
|
|
static siphash_aligned_key_t rt6_exception_key;
|
2021-08-25 23:17:28 +00:00
|
|
|
struct {
|
|
|
|
struct in6_addr dst;
|
|
|
|
struct in6_addr src;
|
|
|
|
} __aligned(SIPHASH_ALIGNMENT) combined = {
|
|
|
|
.dst = *dst,
|
|
|
|
};
|
|
|
|
u64 val;
|
2017-10-06 19:05:57 +00:00
|
|
|
|
2021-08-25 23:17:28 +00:00
|
|
|
net_get_random_once(&rt6_exception_key, sizeof(rt6_exception_key));
|
2017-10-06 19:05:57 +00:00
|
|
|
|
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
|
|
|
if (src)
|
2021-08-25 23:17:28 +00:00
|
|
|
combined.src = *src;
|
2017-10-06 19:05:57 +00:00
|
|
|
#endif
|
2021-08-25 23:17:28 +00:00
|
|
|
val = siphash(&combined, sizeof(combined), &rt6_exception_key);
|
|
|
|
|
|
|
|
return hash_64(val, FIB6_EXCEPTION_BUCKET_SIZE_SHIFT);
|
2017-10-06 19:05:57 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Helper function to find the cached rt in the hash table
|
|
|
|
* and update bucket pointer to point to the bucket for this
|
|
|
|
* (daddr, saddr) pair
|
|
|
|
* Caller must hold rt6_exception_lock
|
|
|
|
*/
|
|
|
|
static struct rt6_exception *
|
|
|
|
__rt6_find_exception_spinlock(struct rt6_exception_bucket **bucket,
|
|
|
|
const struct in6_addr *daddr,
|
|
|
|
const struct in6_addr *saddr)
|
|
|
|
{
|
|
|
|
struct rt6_exception *rt6_ex;
|
|
|
|
u32 hval;
|
|
|
|
|
|
|
|
if (!(*bucket) || !daddr)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
hval = rt6_exception_hash(daddr, saddr);
|
|
|
|
*bucket += hval;
|
|
|
|
|
|
|
|
hlist_for_each_entry(rt6_ex, &(*bucket)->chain, hlist) {
|
|
|
|
struct rt6_info *rt6 = rt6_ex->rt6i;
|
|
|
|
bool matched = ipv6_addr_equal(daddr, &rt6->rt6i_dst.addr);
|
|
|
|
|
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
|
|
|
if (matched && saddr)
|
|
|
|
matched = ipv6_addr_equal(saddr, &rt6->rt6i_src.addr);
|
|
|
|
#endif
|
|
|
|
if (matched)
|
|
|
|
return rt6_ex;
|
|
|
|
}
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Helper function to find the cached rt in the hash table
|
|
|
|
* and update bucket pointer to point to the bucket for this
|
|
|
|
* (daddr, saddr) pair
|
|
|
|
* Caller must hold rcu_read_lock()
|
|
|
|
*/
|
|
|
|
static struct rt6_exception *
|
|
|
|
__rt6_find_exception_rcu(struct rt6_exception_bucket **bucket,
|
|
|
|
const struct in6_addr *daddr,
|
|
|
|
const struct in6_addr *saddr)
|
|
|
|
{
|
|
|
|
struct rt6_exception *rt6_ex;
|
|
|
|
u32 hval;
|
|
|
|
|
|
|
|
WARN_ON_ONCE(!rcu_read_lock_held());
|
|
|
|
|
|
|
|
if (!(*bucket) || !daddr)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
hval = rt6_exception_hash(daddr, saddr);
|
|
|
|
*bucket += hval;
|
|
|
|
|
|
|
|
hlist_for_each_entry_rcu(rt6_ex, &(*bucket)->chain, hlist) {
|
|
|
|
struct rt6_info *rt6 = rt6_ex->rt6i;
|
|
|
|
bool matched = ipv6_addr_equal(daddr, &rt6->rt6i_dst.addr);
|
|
|
|
|
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
|
|
|
if (matched && saddr)
|
|
|
|
matched = ipv6_addr_equal(saddr, &rt6->rt6i_src.addr);
|
|
|
|
#endif
|
|
|
|
if (matched)
|
|
|
|
return rt6_ex;
|
|
|
|
}
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:06 +00:00
|
|
|
static unsigned int fib6_mtu(const struct fib6_result *res)
|
2018-04-18 00:33:16 +00:00
|
|
|
{
|
2019-04-16 21:36:06 +00:00
|
|
|
const struct fib6_nh *nh = res->nh;
|
2018-04-18 00:33:16 +00:00
|
|
|
unsigned int mtu;
|
|
|
|
|
2019-04-16 21:36:06 +00:00
|
|
|
if (res->f6i->fib6_pmtu) {
|
|
|
|
mtu = res->f6i->fib6_pmtu;
|
2018-04-18 22:39:05 +00:00
|
|
|
} else {
|
2019-04-16 21:36:06 +00:00
|
|
|
struct net_device *dev = nh->fib_nh_dev;
|
2018-04-18 22:39:05 +00:00
|
|
|
struct inet6_dev *idev;
|
|
|
|
|
|
|
|
rcu_read_lock();
|
|
|
|
idev = __in6_dev_get(dev);
|
|
|
|
mtu = idev->cnf.mtu6;
|
|
|
|
rcu_read_unlock();
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
mtu = min_t(unsigned int, mtu, IP6_MAX_MTU);
|
|
|
|
|
2019-04-16 21:36:06 +00:00
|
|
|
return mtu - lwtunnel_headroom(nh->fib_nh_lws, mtu);
|
2018-04-18 00:33:16 +00:00
|
|
|
}
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
#define FIB6_EXCEPTION_BUCKET_FLUSHED 0x1UL
|
|
|
|
|
|
|
|
/* used when the flushed bit is not relevant, only access to the bucket
|
|
|
|
* (ie., all bucket users except rt6_insert_exception);
|
|
|
|
*
|
|
|
|
* called under rcu lock; sometimes called with rt6_exception_lock held
|
|
|
|
*/
|
|
|
|
static
|
|
|
|
struct rt6_exception_bucket *fib6_nh_get_excptn_bucket(const struct fib6_nh *nh,
|
|
|
|
spinlock_t *lock)
|
|
|
|
{
|
|
|
|
struct rt6_exception_bucket *bucket;
|
|
|
|
|
|
|
|
if (lock)
|
|
|
|
bucket = rcu_dereference_protected(nh->rt6i_exception_bucket,
|
|
|
|
lockdep_is_held(lock));
|
|
|
|
else
|
|
|
|
bucket = rcu_dereference(nh->rt6i_exception_bucket);
|
|
|
|
|
|
|
|
/* remove bucket flushed bit if set */
|
|
|
|
if (bucket) {
|
|
|
|
unsigned long p = (unsigned long)bucket;
|
|
|
|
|
|
|
|
p &= ~FIB6_EXCEPTION_BUCKET_FLUSHED;
|
|
|
|
bucket = (struct rt6_exception_bucket *)p;
|
|
|
|
}
|
|
|
|
|
|
|
|
return bucket;
|
|
|
|
}
|
|
|
|
|
|
|
|
static bool fib6_nh_excptn_bucket_flushed(struct rt6_exception_bucket *bucket)
|
|
|
|
{
|
|
|
|
unsigned long p = (unsigned long)bucket;
|
|
|
|
|
|
|
|
return !!(p & FIB6_EXCEPTION_BUCKET_FLUSHED);
|
|
|
|
}
|
|
|
|
|
|
|
|
/* called with rt6_exception_lock held */
|
|
|
|
static void fib6_nh_excptn_bucket_set_flushed(struct fib6_nh *nh,
|
|
|
|
spinlock_t *lock)
|
|
|
|
{
|
|
|
|
struct rt6_exception_bucket *bucket;
|
|
|
|
unsigned long p;
|
|
|
|
|
|
|
|
bucket = rcu_dereference_protected(nh->rt6i_exception_bucket,
|
|
|
|
lockdep_is_held(lock));
|
|
|
|
|
|
|
|
p = (unsigned long)bucket;
|
|
|
|
p |= FIB6_EXCEPTION_BUCKET_FLUSHED;
|
|
|
|
bucket = (struct rt6_exception_bucket *)p;
|
|
|
|
rcu_assign_pointer(nh->rt6i_exception_bucket, bucket);
|
|
|
|
}
|
|
|
|
|
2017-10-06 19:05:57 +00:00
|
|
|
static int rt6_insert_exception(struct rt6_info *nrt,
|
2019-04-16 21:36:05 +00:00
|
|
|
const struct fib6_result *res)
|
2017-10-06 19:05:57 +00:00
|
|
|
{
|
2018-04-18 00:33:14 +00:00
|
|
|
struct net *net = dev_net(nrt->dst.dev);
|
2017-10-06 19:05:57 +00:00
|
|
|
struct rt6_exception_bucket *bucket;
|
2019-05-23 03:27:58 +00:00
|
|
|
struct fib6_info *f6i = res->f6i;
|
2017-10-06 19:05:57 +00:00
|
|
|
struct in6_addr *src_key = NULL;
|
|
|
|
struct rt6_exception *rt6_ex;
|
2019-05-23 03:27:58 +00:00
|
|
|
struct fib6_nh *nh = res->nh;
|
2021-08-29 22:16:14 +00:00
|
|
|
int max_depth;
|
2017-10-06 19:05:57 +00:00
|
|
|
int err = 0;
|
|
|
|
|
|
|
|
spin_lock_bh(&rt6_exception_lock);
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
bucket = rcu_dereference_protected(nh->rt6i_exception_bucket,
|
|
|
|
lockdep_is_held(&rt6_exception_lock));
|
2017-10-06 19:05:57 +00:00
|
|
|
if (!bucket) {
|
|
|
|
bucket = kcalloc(FIB6_EXCEPTION_BUCKET_SIZE, sizeof(*bucket),
|
|
|
|
GFP_ATOMIC);
|
|
|
|
if (!bucket) {
|
|
|
|
err = -ENOMEM;
|
|
|
|
goto out;
|
|
|
|
}
|
2019-05-23 03:27:58 +00:00
|
|
|
rcu_assign_pointer(nh->rt6i_exception_bucket, bucket);
|
|
|
|
} else if (fib6_nh_excptn_bucket_flushed(bucket)) {
|
|
|
|
err = -EINVAL;
|
|
|
|
goto out;
|
2017-10-06 19:05:57 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
2019-04-16 21:36:05 +00:00
|
|
|
/* fib6_src.plen != 0 indicates f6i is in subtree
|
2017-10-06 19:05:57 +00:00
|
|
|
* and exception table is indexed by a hash of
|
2019-04-16 21:36:05 +00:00
|
|
|
* both fib6_dst and fib6_src.
|
2017-10-06 19:05:57 +00:00
|
|
|
* Otherwise, the exception table is indexed by
|
2019-04-16 21:36:05 +00:00
|
|
|
* a hash of only fib6_dst.
|
2017-10-06 19:05:57 +00:00
|
|
|
*/
|
2019-04-16 21:36:05 +00:00
|
|
|
if (f6i->fib6_src.plen)
|
2017-10-06 19:05:57 +00:00
|
|
|
src_key = &nrt->rt6i_src.addr;
|
|
|
|
#endif
|
2019-04-16 21:36:05 +00:00
|
|
|
/* rt6_mtu_change() might lower mtu on f6i.
|
2017-10-06 19:05:59 +00:00
|
|
|
* Only insert this exception route if its mtu
|
2019-04-16 21:36:05 +00:00
|
|
|
* is less than f6i's mtu value.
|
2017-10-06 19:05:59 +00:00
|
|
|
*/
|
2019-04-16 21:36:06 +00:00
|
|
|
if (dst_metric_raw(&nrt->dst, RTAX_MTU) >= fib6_mtu(res)) {
|
2017-10-06 19:05:59 +00:00
|
|
|
err = -EINVAL;
|
|
|
|
goto out;
|
|
|
|
}
|
2017-10-06 19:05:58 +00:00
|
|
|
|
2017-10-06 19:05:57 +00:00
|
|
|
rt6_ex = __rt6_find_exception_spinlock(&bucket, &nrt->rt6i_dst.addr,
|
|
|
|
src_key);
|
|
|
|
if (rt6_ex)
|
|
|
|
rt6_remove_exception(bucket, rt6_ex);
|
|
|
|
|
|
|
|
rt6_ex = kzalloc(sizeof(*rt6_ex), GFP_ATOMIC);
|
|
|
|
if (!rt6_ex) {
|
|
|
|
err = -ENOMEM;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
rt6_ex->rt6i = nrt;
|
|
|
|
rt6_ex->stamp = jiffies;
|
|
|
|
hlist_add_head_rcu(&rt6_ex->hlist, &bucket->chain);
|
|
|
|
bucket->depth++;
|
2017-10-06 19:06:11 +00:00
|
|
|
net->ipv6.rt6_stats->fib_rt_cache++;
|
2017-10-06 19:05:57 +00:00
|
|
|
|
2021-08-29 22:16:14 +00:00
|
|
|
/* Randomize max depth to avoid some side channels attacks. */
|
|
|
|
max_depth = FIB6_MAX_DEPTH + prandom_u32_max(FIB6_MAX_DEPTH);
|
|
|
|
while (bucket->depth > max_depth)
|
2017-10-06 19:05:57 +00:00
|
|
|
rt6_exception_remove_oldest(bucket);
|
|
|
|
|
|
|
|
out:
|
|
|
|
spin_unlock_bh(&rt6_exception_lock);
|
|
|
|
|
|
|
|
/* Update fn->fn_sernum to invalidate all cached dst */
|
2017-10-19 14:07:10 +00:00
|
|
|
if (!err) {
|
2019-04-16 21:36:05 +00:00
|
|
|
spin_lock_bh(&f6i->fib6_table->tb6_lock);
|
|
|
|
fib6_update_sernum(net, f6i);
|
|
|
|
spin_unlock_bh(&f6i->fib6_table->tb6_lock);
|
2017-10-19 14:07:10 +00:00
|
|
|
fib6_force_start_gc(net);
|
|
|
|
}
|
2017-10-06 19:05:57 +00:00
|
|
|
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:27:57 +00:00
|
|
|
static void fib6_nh_flush_exceptions(struct fib6_nh *nh, struct fib6_info *from)
|
2017-10-06 19:05:57 +00:00
|
|
|
{
|
|
|
|
struct rt6_exception_bucket *bucket;
|
|
|
|
struct rt6_exception *rt6_ex;
|
|
|
|
struct hlist_node *tmp;
|
|
|
|
int i;
|
|
|
|
|
|
|
|
spin_lock_bh(&rt6_exception_lock);
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
bucket = fib6_nh_get_excptn_bucket(nh, &rt6_exception_lock);
|
2017-10-06 19:05:57 +00:00
|
|
|
if (!bucket)
|
|
|
|
goto out;
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
/* Prevent rt6_insert_exception() to recreate the bucket list */
|
|
|
|
if (!from)
|
|
|
|
fib6_nh_excptn_bucket_set_flushed(nh, &rt6_exception_lock);
|
|
|
|
|
2017-10-06 19:05:57 +00:00
|
|
|
for (i = 0; i < FIB6_EXCEPTION_BUCKET_SIZE; i++) {
|
2019-05-23 03:27:58 +00:00
|
|
|
hlist_for_each_entry_safe(rt6_ex, tmp, &bucket->chain, hlist) {
|
|
|
|
if (!from ||
|
|
|
|
rcu_access_pointer(rt6_ex->rt6i->from) == from)
|
|
|
|
rt6_remove_exception(bucket, rt6_ex);
|
|
|
|
}
|
|
|
|
WARN_ON_ONCE(!from && bucket->depth);
|
2017-10-06 19:05:57 +00:00
|
|
|
bucket++;
|
|
|
|
}
|
|
|
|
out:
|
|
|
|
spin_unlock_bh(&rt6_exception_lock);
|
|
|
|
}
|
|
|
|
|
2019-06-08 21:53:28 +00:00
|
|
|
static int rt6_nh_flush_exceptions(struct fib6_nh *nh, void *arg)
|
|
|
|
{
|
|
|
|
struct fib6_info *f6i = arg;
|
|
|
|
|
|
|
|
fib6_nh_flush_exceptions(nh, f6i);
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:27:57 +00:00
|
|
|
void rt6_flush_exceptions(struct fib6_info *f6i)
|
|
|
|
{
|
2019-06-08 21:53:28 +00:00
|
|
|
if (f6i->nh)
|
|
|
|
nexthop_for_each_fib6_nh(f6i->nh, rt6_nh_flush_exceptions,
|
|
|
|
f6i);
|
|
|
|
else
|
|
|
|
fib6_nh_flush_exceptions(f6i->fib6_nh, f6i);
|
2019-05-23 03:27:57 +00:00
|
|
|
}
|
|
|
|
|
2017-10-06 19:05:57 +00:00
|
|
|
/* Find cached rt in the hash table inside passed in rt
|
|
|
|
* Caller has to hold rcu_read_lock()
|
|
|
|
*/
|
2019-04-16 21:36:00 +00:00
|
|
|
static struct rt6_info *rt6_find_cached_rt(const struct fib6_result *res,
|
2019-05-16 20:30:54 +00:00
|
|
|
const struct in6_addr *daddr,
|
|
|
|
const struct in6_addr *saddr)
|
2017-10-06 19:05:57 +00:00
|
|
|
{
|
2019-05-16 20:30:54 +00:00
|
|
|
const struct in6_addr *src_key = NULL;
|
2017-10-06 19:05:57 +00:00
|
|
|
struct rt6_exception_bucket *bucket;
|
|
|
|
struct rt6_exception *rt6_ex;
|
2019-04-16 21:36:00 +00:00
|
|
|
struct rt6_info *ret = NULL;
|
2017-10-06 19:05:57 +00:00
|
|
|
|
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
2019-04-16 21:36:00 +00:00
|
|
|
/* fib6i_src.plen != 0 indicates f6i is in subtree
|
2017-10-06 19:05:57 +00:00
|
|
|
* and exception table is indexed by a hash of
|
2019-04-16 21:36:00 +00:00
|
|
|
* both fib6_dst and fib6_src.
|
2019-05-16 20:30:54 +00:00
|
|
|
* However, the src addr used to create the hash
|
|
|
|
* might not be exactly the passed in saddr which
|
|
|
|
* is a /128 addr from the flow.
|
|
|
|
* So we need to use f6i->fib6_src to redo lookup
|
|
|
|
* if the passed in saddr does not find anything.
|
|
|
|
* (See the logic in ip6_rt_cache_alloc() on how
|
|
|
|
* rt->rt6i_src is updated.)
|
2017-10-06 19:05:57 +00:00
|
|
|
*/
|
2019-04-16 21:36:00 +00:00
|
|
|
if (res->f6i->fib6_src.plen)
|
2017-10-06 19:05:57 +00:00
|
|
|
src_key = saddr;
|
2019-05-16 20:30:54 +00:00
|
|
|
find_ex:
|
2017-10-06 19:05:57 +00:00
|
|
|
#endif
|
2019-05-23 03:27:58 +00:00
|
|
|
bucket = fib6_nh_get_excptn_bucket(res->nh, NULL);
|
2017-10-06 19:05:57 +00:00
|
|
|
rt6_ex = __rt6_find_exception_rcu(&bucket, daddr, src_key);
|
|
|
|
|
|
|
|
if (rt6_ex && !rt6_check_expired(rt6_ex->rt6i))
|
2019-04-16 21:36:00 +00:00
|
|
|
ret = rt6_ex->rt6i;
|
2017-10-06 19:05:57 +00:00
|
|
|
|
2019-05-16 20:30:54 +00:00
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
|
|
|
/* Use fib6_src as src_key and redo lookup */
|
|
|
|
if (!ret && src_key && src_key != &res->f6i->fib6_src.addr) {
|
|
|
|
src_key = &res->f6i->fib6_src.addr;
|
|
|
|
goto find_ex;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2019-04-16 21:36:00 +00:00
|
|
|
return ret;
|
2017-10-06 19:05:57 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/* Remove the passed in cached rt from the hash table that contains it */
|
2019-05-23 03:27:58 +00:00
|
|
|
static int fib6_nh_remove_exception(const struct fib6_nh *nh, int plen,
|
2019-05-23 03:27:57 +00:00
|
|
|
const struct rt6_info *rt)
|
2017-10-06 19:05:57 +00:00
|
|
|
{
|
2019-05-23 03:27:57 +00:00
|
|
|
const struct in6_addr *src_key = NULL;
|
2017-10-06 19:05:57 +00:00
|
|
|
struct rt6_exception_bucket *bucket;
|
|
|
|
struct rt6_exception *rt6_ex;
|
|
|
|
int err;
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
if (!rcu_access_pointer(nh->rt6i_exception_bucket))
|
2017-10-06 19:05:57 +00:00
|
|
|
return -ENOENT;
|
|
|
|
|
|
|
|
spin_lock_bh(&rt6_exception_lock);
|
2019-05-23 03:27:58 +00:00
|
|
|
bucket = fib6_nh_get_excptn_bucket(nh, &rt6_exception_lock);
|
|
|
|
|
2017-10-06 19:05:57 +00:00
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
2019-05-23 03:27:58 +00:00
|
|
|
/* rt6i_src.plen != 0 indicates 'from' is in subtree
|
|
|
|
* and exception table is indexed by a hash of
|
|
|
|
* both rt6i_dst and rt6i_src.
|
2017-10-06 19:05:57 +00:00
|
|
|
* Otherwise, the exception table is indexed by
|
|
|
|
* a hash of only rt6i_dst.
|
|
|
|
*/
|
2019-05-23 03:27:57 +00:00
|
|
|
if (plen)
|
2017-10-06 19:05:57 +00:00
|
|
|
src_key = &rt->rt6i_src.addr;
|
|
|
|
#endif
|
|
|
|
rt6_ex = __rt6_find_exception_spinlock(&bucket,
|
|
|
|
&rt->rt6i_dst.addr,
|
|
|
|
src_key);
|
|
|
|
if (rt6_ex) {
|
|
|
|
rt6_remove_exception(bucket, rt6_ex);
|
|
|
|
err = 0;
|
|
|
|
} else {
|
|
|
|
err = -ENOENT;
|
|
|
|
}
|
|
|
|
|
|
|
|
spin_unlock_bh(&rt6_exception_lock);
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2019-06-08 21:53:28 +00:00
|
|
|
struct fib6_nh_excptn_arg {
|
|
|
|
struct rt6_info *rt;
|
|
|
|
int plen;
|
|
|
|
};
|
|
|
|
|
|
|
|
static int rt6_nh_remove_exception_rt(struct fib6_nh *nh, void *_arg)
|
|
|
|
{
|
|
|
|
struct fib6_nh_excptn_arg *arg = _arg;
|
|
|
|
int err;
|
|
|
|
|
|
|
|
err = fib6_nh_remove_exception(nh, arg->plen, arg->rt);
|
|
|
|
if (err == 0)
|
|
|
|
return 1;
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:27:57 +00:00
|
|
|
static int rt6_remove_exception_rt(struct rt6_info *rt)
|
|
|
|
{
|
|
|
|
struct fib6_info *from;
|
|
|
|
|
|
|
|
from = rcu_dereference(rt->from);
|
2019-05-23 03:27:58 +00:00
|
|
|
if (!from || !(rt->rt6i_flags & RTF_CACHE))
|
2019-05-23 03:27:57 +00:00
|
|
|
return -EINVAL;
|
|
|
|
|
2019-06-08 21:53:28 +00:00
|
|
|
if (from->nh) {
|
|
|
|
struct fib6_nh_excptn_arg arg = {
|
|
|
|
.rt = rt,
|
|
|
|
.plen = from->fib6_src.plen
|
|
|
|
};
|
|
|
|
int rc;
|
|
|
|
|
|
|
|
/* rc = 1 means an entry was found */
|
|
|
|
rc = nexthop_for_each_fib6_nh(from->nh,
|
|
|
|
rt6_nh_remove_exception_rt,
|
|
|
|
&arg);
|
|
|
|
return rc ? 0 : -ENOENT;
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:27:59 +00:00
|
|
|
return fib6_nh_remove_exception(from->fib6_nh,
|
2019-05-23 03:27:58 +00:00
|
|
|
from->fib6_src.plen, rt);
|
2019-05-23 03:27:57 +00:00
|
|
|
}
|
|
|
|
|
2017-10-06 19:05:57 +00:00
|
|
|
/* Find rt6_ex which contains the passed in rt cache and
|
|
|
|
* refresh its stamp
|
|
|
|
*/
|
2019-05-23 03:27:58 +00:00
|
|
|
static void fib6_nh_update_exception(const struct fib6_nh *nh, int plen,
|
2019-05-23 03:27:57 +00:00
|
|
|
const struct rt6_info *rt)
|
2017-10-06 19:05:57 +00:00
|
|
|
{
|
2019-05-23 03:27:57 +00:00
|
|
|
const struct in6_addr *src_key = NULL;
|
2017-10-06 19:05:57 +00:00
|
|
|
struct rt6_exception_bucket *bucket;
|
|
|
|
struct rt6_exception *rt6_ex;
|
2019-02-21 10:19:41 +00:00
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
bucket = fib6_nh_get_excptn_bucket(nh, NULL);
|
2017-10-06 19:05:57 +00:00
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
2019-05-23 03:27:58 +00:00
|
|
|
/* rt6i_src.plen != 0 indicates 'from' is in subtree
|
|
|
|
* and exception table is indexed by a hash of
|
|
|
|
* both rt6i_dst and rt6i_src.
|
2017-10-06 19:05:57 +00:00
|
|
|
* Otherwise, the exception table is indexed by
|
|
|
|
* a hash of only rt6i_dst.
|
|
|
|
*/
|
2019-05-23 03:27:57 +00:00
|
|
|
if (plen)
|
2017-10-06 19:05:57 +00:00
|
|
|
src_key = &rt->rt6i_src.addr;
|
|
|
|
#endif
|
2019-05-23 03:27:58 +00:00
|
|
|
rt6_ex = __rt6_find_exception_rcu(&bucket, &rt->rt6i_dst.addr, src_key);
|
2017-10-06 19:05:57 +00:00
|
|
|
if (rt6_ex)
|
|
|
|
rt6_ex->stamp = jiffies;
|
2019-05-23 03:27:57 +00:00
|
|
|
}
|
|
|
|
|
2019-06-08 21:53:28 +00:00
|
|
|
struct fib6_nh_match_arg {
|
|
|
|
const struct net_device *dev;
|
|
|
|
const struct in6_addr *gw;
|
|
|
|
struct fib6_nh *match;
|
|
|
|
};
|
|
|
|
|
|
|
|
/* determine if fib6_nh has given device and gateway */
|
|
|
|
static int fib6_nh_find_match(struct fib6_nh *nh, void *_arg)
|
|
|
|
{
|
|
|
|
struct fib6_nh_match_arg *arg = _arg;
|
|
|
|
|
|
|
|
if (arg->dev != nh->fib_nh_dev ||
|
|
|
|
(arg->gw && !nh->fib_nh_gw_family) ||
|
|
|
|
(!arg->gw && nh->fib_nh_gw_family) ||
|
|
|
|
(arg->gw && !ipv6_addr_equal(arg->gw, &nh->fib_nh_gw6)))
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
arg->match = nh;
|
|
|
|
|
|
|
|
/* found a match, break the loop */
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:27:57 +00:00
|
|
|
static void rt6_update_exception_stamp_rt(struct rt6_info *rt)
|
|
|
|
{
|
|
|
|
struct fib6_info *from;
|
2019-06-08 21:53:28 +00:00
|
|
|
struct fib6_nh *fib6_nh;
|
2017-10-06 19:05:57 +00:00
|
|
|
|
2019-05-23 03:27:57 +00:00
|
|
|
rcu_read_lock();
|
|
|
|
|
|
|
|
from = rcu_dereference(rt->from);
|
|
|
|
if (!from || !(rt->rt6i_flags & RTF_CACHE))
|
|
|
|
goto unlock;
|
|
|
|
|
2019-06-08 21:53:28 +00:00
|
|
|
if (from->nh) {
|
|
|
|
struct fib6_nh_match_arg arg = {
|
|
|
|
.dev = rt->dst.dev,
|
|
|
|
.gw = &rt->rt6i_gateway,
|
|
|
|
};
|
|
|
|
|
|
|
|
nexthop_for_each_fib6_nh(from->nh, fib6_nh_find_match, &arg);
|
|
|
|
|
|
|
|
if (!arg.match)
|
2019-08-01 21:36:35 +00:00
|
|
|
goto unlock;
|
2019-06-08 21:53:28 +00:00
|
|
|
fib6_nh = arg.match;
|
|
|
|
} else {
|
|
|
|
fib6_nh = from->fib6_nh;
|
|
|
|
}
|
|
|
|
fib6_nh_update_exception(fib6_nh, from->fib6_src.plen, rt);
|
2019-02-21 10:19:41 +00:00
|
|
|
unlock:
|
2017-10-06 19:05:57 +00:00
|
|
|
rcu_read_unlock();
|
|
|
|
}
|
|
|
|
|
ipv6: Reflect MTU changes on PMTU of exceptions for MTU-less routes
Currently, administrative MTU changes on a given netdevice are
not reflected on route exceptions for MTU-less routes, with a
set PMTU value, for that device:
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a proto kernel src 2001:db8::a metric 256 pref medium
# ping6 -c 1 -q -s10000 2001:db8::b > /dev/null
# ip netns exec a ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
# ip link set dev vti_a mtu 3000
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
# ip link set dev vti_a mtu 9000
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
The first issue is that since commit fb56be83e43d ("net-ipv6: on
device mtu change do not add mtu to mtu-less routes") we don't
call rt6_exceptions_update_pmtu() from rt6_mtu_change_route(),
which handles administrative MTU changes, if the regular route
is MTU-less.
However, PMTU exceptions should be always updated, as long as
RTAX_MTU is not locked. Keep the check for MTU-less main route,
as introduced by that commit, but, for exceptions,
call rt6_exceptions_update_pmtu() regardless of that check.
Once that is fixed, one problem remains: MTU changes are not
reflected if the new MTU is higher than the previous one,
because rt6_exceptions_update_pmtu() doesn't allow that. We
should instead allow PMTU increase if the old PMTU matches the
local MTU, as that implies that the old MTU was the lowest in the
path, and PMTU discovery might lead to different results.
The existing check in rt6_mtu_change_route() correctly took that
case into account (for regular routes only), so factor it out
and re-use it also in rt6_exceptions_update_pmtu().
While at it, fix comments style and grammar, and try to be a bit
more descriptive.
Reported-by: Xiumei Mu <xmu@redhat.com>
Fixes: fb56be83e43d ("net-ipv6: on device mtu change do not add mtu to mtu-less routes")
Fixes: f5bbe7ee79c2 ("ipv6: prepare rt6_mtu_change() for exception table")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Acked-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-03-06 10:10:19 +00:00
|
|
|
static bool rt6_mtu_change_route_allowed(struct inet6_dev *idev,
|
|
|
|
struct rt6_info *rt, int mtu)
|
|
|
|
{
|
|
|
|
/* If the new MTU is lower than the route PMTU, this new MTU will be the
|
|
|
|
* lowest MTU in the path: always allow updating the route PMTU to
|
|
|
|
* reflect PMTU decreases.
|
|
|
|
*
|
|
|
|
* If the new MTU is higher, and the route PMTU is equal to the local
|
|
|
|
* MTU, this means the old MTU is the lowest in the path, so allow
|
|
|
|
* updating it: if other nodes now have lower MTUs, PMTU discovery will
|
|
|
|
* handle this.
|
|
|
|
*/
|
|
|
|
|
|
|
|
if (dst_mtu(&rt->dst) >= mtu)
|
|
|
|
return true;
|
|
|
|
|
|
|
|
if (dst_mtu(&rt->dst) == idev->cnf.mtu6)
|
|
|
|
return true;
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
static void rt6_exceptions_update_pmtu(struct inet6_dev *idev,
|
2019-05-23 03:27:58 +00:00
|
|
|
const struct fib6_nh *nh, int mtu)
|
2017-10-06 19:05:59 +00:00
|
|
|
{
|
|
|
|
struct rt6_exception_bucket *bucket;
|
|
|
|
struct rt6_exception *rt6_ex;
|
|
|
|
int i;
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
bucket = fib6_nh_get_excptn_bucket(nh, &rt6_exception_lock);
|
ipv6: Reflect MTU changes on PMTU of exceptions for MTU-less routes
Currently, administrative MTU changes on a given netdevice are
not reflected on route exceptions for MTU-less routes, with a
set PMTU value, for that device:
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a proto kernel src 2001:db8::a metric 256 pref medium
# ping6 -c 1 -q -s10000 2001:db8::b > /dev/null
# ip netns exec a ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
# ip link set dev vti_a mtu 3000
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
# ip link set dev vti_a mtu 9000
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
The first issue is that since commit fb56be83e43d ("net-ipv6: on
device mtu change do not add mtu to mtu-less routes") we don't
call rt6_exceptions_update_pmtu() from rt6_mtu_change_route(),
which handles administrative MTU changes, if the regular route
is MTU-less.
However, PMTU exceptions should be always updated, as long as
RTAX_MTU is not locked. Keep the check for MTU-less main route,
as introduced by that commit, but, for exceptions,
call rt6_exceptions_update_pmtu() regardless of that check.
Once that is fixed, one problem remains: MTU changes are not
reflected if the new MTU is higher than the previous one,
because rt6_exceptions_update_pmtu() doesn't allow that. We
should instead allow PMTU increase if the old PMTU matches the
local MTU, as that implies that the old MTU was the lowest in the
path, and PMTU discovery might lead to different results.
The existing check in rt6_mtu_change_route() correctly took that
case into account (for regular routes only), so factor it out
and re-use it also in rt6_exceptions_update_pmtu().
While at it, fix comments style and grammar, and try to be a bit
more descriptive.
Reported-by: Xiumei Mu <xmu@redhat.com>
Fixes: fb56be83e43d ("net-ipv6: on device mtu change do not add mtu to mtu-less routes")
Fixes: f5bbe7ee79c2 ("ipv6: prepare rt6_mtu_change() for exception table")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Acked-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-03-06 10:10:19 +00:00
|
|
|
if (!bucket)
|
|
|
|
return;
|
|
|
|
|
|
|
|
for (i = 0; i < FIB6_EXCEPTION_BUCKET_SIZE; i++) {
|
|
|
|
hlist_for_each_entry(rt6_ex, &bucket->chain, hlist) {
|
|
|
|
struct rt6_info *entry = rt6_ex->rt6i;
|
|
|
|
|
|
|
|
/* For RTF_CACHE with rt6i_pmtu == 0 (i.e. a redirected
|
2018-04-18 00:33:16 +00:00
|
|
|
* route), the metrics of its rt->from have already
|
ipv6: Reflect MTU changes on PMTU of exceptions for MTU-less routes
Currently, administrative MTU changes on a given netdevice are
not reflected on route exceptions for MTU-less routes, with a
set PMTU value, for that device:
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a proto kernel src 2001:db8::a metric 256 pref medium
# ping6 -c 1 -q -s10000 2001:db8::b > /dev/null
# ip netns exec a ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
# ip link set dev vti_a mtu 3000
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
# ip link set dev vti_a mtu 9000
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
The first issue is that since commit fb56be83e43d ("net-ipv6: on
device mtu change do not add mtu to mtu-less routes") we don't
call rt6_exceptions_update_pmtu() from rt6_mtu_change_route(),
which handles administrative MTU changes, if the regular route
is MTU-less.
However, PMTU exceptions should be always updated, as long as
RTAX_MTU is not locked. Keep the check for MTU-less main route,
as introduced by that commit, but, for exceptions,
call rt6_exceptions_update_pmtu() regardless of that check.
Once that is fixed, one problem remains: MTU changes are not
reflected if the new MTU is higher than the previous one,
because rt6_exceptions_update_pmtu() doesn't allow that. We
should instead allow PMTU increase if the old PMTU matches the
local MTU, as that implies that the old MTU was the lowest in the
path, and PMTU discovery might lead to different results.
The existing check in rt6_mtu_change_route() correctly took that
case into account (for regular routes only), so factor it out
and re-use it also in rt6_exceptions_update_pmtu().
While at it, fix comments style and grammar, and try to be a bit
more descriptive.
Reported-by: Xiumei Mu <xmu@redhat.com>
Fixes: fb56be83e43d ("net-ipv6: on device mtu change do not add mtu to mtu-less routes")
Fixes: f5bbe7ee79c2 ("ipv6: prepare rt6_mtu_change() for exception table")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Acked-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-03-06 10:10:19 +00:00
|
|
|
* been updated.
|
|
|
|
*/
|
2018-04-18 00:33:16 +00:00
|
|
|
if (dst_metric_raw(&entry->dst, RTAX_MTU) &&
|
ipv6: Reflect MTU changes on PMTU of exceptions for MTU-less routes
Currently, administrative MTU changes on a given netdevice are
not reflected on route exceptions for MTU-less routes, with a
set PMTU value, for that device:
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a proto kernel src 2001:db8::a metric 256 pref medium
# ping6 -c 1 -q -s10000 2001:db8::b > /dev/null
# ip netns exec a ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
# ip link set dev vti_a mtu 3000
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
# ip link set dev vti_a mtu 9000
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
The first issue is that since commit fb56be83e43d ("net-ipv6: on
device mtu change do not add mtu to mtu-less routes") we don't
call rt6_exceptions_update_pmtu() from rt6_mtu_change_route(),
which handles administrative MTU changes, if the regular route
is MTU-less.
However, PMTU exceptions should be always updated, as long as
RTAX_MTU is not locked. Keep the check for MTU-less main route,
as introduced by that commit, but, for exceptions,
call rt6_exceptions_update_pmtu() regardless of that check.
Once that is fixed, one problem remains: MTU changes are not
reflected if the new MTU is higher than the previous one,
because rt6_exceptions_update_pmtu() doesn't allow that. We
should instead allow PMTU increase if the old PMTU matches the
local MTU, as that implies that the old MTU was the lowest in the
path, and PMTU discovery might lead to different results.
The existing check in rt6_mtu_change_route() correctly took that
case into account (for regular routes only), so factor it out
and re-use it also in rt6_exceptions_update_pmtu().
While at it, fix comments style and grammar, and try to be a bit
more descriptive.
Reported-by: Xiumei Mu <xmu@redhat.com>
Fixes: fb56be83e43d ("net-ipv6: on device mtu change do not add mtu to mtu-less routes")
Fixes: f5bbe7ee79c2 ("ipv6: prepare rt6_mtu_change() for exception table")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Acked-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-03-06 10:10:19 +00:00
|
|
|
rt6_mtu_change_route_allowed(idev, entry, mtu))
|
2018-04-18 00:33:16 +00:00
|
|
|
dst_metric_set(&entry->dst, RTAX_MTU, mtu);
|
2017-10-06 19:05:59 +00:00
|
|
|
}
|
ipv6: Reflect MTU changes on PMTU of exceptions for MTU-less routes
Currently, administrative MTU changes on a given netdevice are
not reflected on route exceptions for MTU-less routes, with a
set PMTU value, for that device:
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a proto kernel src 2001:db8::a metric 256 pref medium
# ping6 -c 1 -q -s10000 2001:db8::b > /dev/null
# ip netns exec a ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
# ip link set dev vti_a mtu 3000
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
# ip link set dev vti_a mtu 9000
# ip -6 route get 2001:db8::b
2001:db8::b from :: dev vti_a src 2001:db8::a metric 0
cache expires 571sec mtu 4926 pref medium
The first issue is that since commit fb56be83e43d ("net-ipv6: on
device mtu change do not add mtu to mtu-less routes") we don't
call rt6_exceptions_update_pmtu() from rt6_mtu_change_route(),
which handles administrative MTU changes, if the regular route
is MTU-less.
However, PMTU exceptions should be always updated, as long as
RTAX_MTU is not locked. Keep the check for MTU-less main route,
as introduced by that commit, but, for exceptions,
call rt6_exceptions_update_pmtu() regardless of that check.
Once that is fixed, one problem remains: MTU changes are not
reflected if the new MTU is higher than the previous one,
because rt6_exceptions_update_pmtu() doesn't allow that. We
should instead allow PMTU increase if the old PMTU matches the
local MTU, as that implies that the old MTU was the lowest in the
path, and PMTU discovery might lead to different results.
The existing check in rt6_mtu_change_route() correctly took that
case into account (for regular routes only), so factor it out
and re-use it also in rt6_exceptions_update_pmtu().
While at it, fix comments style and grammar, and try to be a bit
more descriptive.
Reported-by: Xiumei Mu <xmu@redhat.com>
Fixes: fb56be83e43d ("net-ipv6: on device mtu change do not add mtu to mtu-less routes")
Fixes: f5bbe7ee79c2 ("ipv6: prepare rt6_mtu_change() for exception table")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Acked-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-03-06 10:10:19 +00:00
|
|
|
bucket++;
|
2017-10-06 19:05:59 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-10-06 19:06:00 +00:00
|
|
|
#define RTF_CACHE_GATEWAY (RTF_GATEWAY | RTF_CACHE)
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
static void fib6_nh_exceptions_clean_tohost(const struct fib6_nh *nh,
|
|
|
|
const struct in6_addr *gateway)
|
2017-10-06 19:06:00 +00:00
|
|
|
{
|
|
|
|
struct rt6_exception_bucket *bucket;
|
|
|
|
struct rt6_exception *rt6_ex;
|
|
|
|
struct hlist_node *tmp;
|
|
|
|
int i;
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
if (!rcu_access_pointer(nh->rt6i_exception_bucket))
|
2017-10-06 19:06:00 +00:00
|
|
|
return;
|
|
|
|
|
|
|
|
spin_lock_bh(&rt6_exception_lock);
|
2019-05-23 03:27:58 +00:00
|
|
|
bucket = fib6_nh_get_excptn_bucket(nh, &rt6_exception_lock);
|
2017-10-06 19:06:00 +00:00
|
|
|
if (bucket) {
|
|
|
|
for (i = 0; i < FIB6_EXCEPTION_BUCKET_SIZE; i++) {
|
|
|
|
hlist_for_each_entry_safe(rt6_ex, tmp,
|
|
|
|
&bucket->chain, hlist) {
|
|
|
|
struct rt6_info *entry = rt6_ex->rt6i;
|
|
|
|
|
|
|
|
if ((entry->rt6i_flags & RTF_CACHE_GATEWAY) ==
|
|
|
|
RTF_CACHE_GATEWAY &&
|
|
|
|
ipv6_addr_equal(gateway,
|
|
|
|
&entry->rt6i_gateway)) {
|
|
|
|
rt6_remove_exception(bucket, rt6_ex);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
bucket++;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
spin_unlock_bh(&rt6_exception_lock);
|
|
|
|
}
|
|
|
|
|
2017-10-06 19:06:01 +00:00
|
|
|
static void rt6_age_examine_exception(struct rt6_exception_bucket *bucket,
|
|
|
|
struct rt6_exception *rt6_ex,
|
|
|
|
struct fib6_gc_args *gc_args,
|
|
|
|
unsigned long now)
|
|
|
|
{
|
|
|
|
struct rt6_info *rt = rt6_ex->rt6i;
|
|
|
|
|
2017-10-19 14:07:11 +00:00
|
|
|
/* we are pruning and obsoleting aged-out and non gateway exceptions
|
|
|
|
* even if others have still references to them, so that on next
|
|
|
|
* dst_check() such references can be dropped.
|
|
|
|
* EXPIRES exceptions - e.g. pmtu-generated ones are pruned when
|
|
|
|
* expired, independently from their aging, as per RFC 8201 section 4
|
|
|
|
*/
|
2018-01-26 19:40:17 +00:00
|
|
|
if (!(rt->rt6i_flags & RTF_EXPIRES)) {
|
|
|
|
if (time_after_eq(now, rt->dst.lastuse + gc_args->timeout)) {
|
|
|
|
RT6_TRACE("aging clone %p\n", rt);
|
|
|
|
rt6_remove_exception(bucket, rt6_ex);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
} else if (time_after(jiffies, rt->dst.expires)) {
|
|
|
|
RT6_TRACE("purging expired route %p\n", rt);
|
2017-10-06 19:06:01 +00:00
|
|
|
rt6_remove_exception(bucket, rt6_ex);
|
|
|
|
return;
|
2018-01-26 19:40:17 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if (rt->rt6i_flags & RTF_GATEWAY) {
|
2017-10-06 19:06:01 +00:00
|
|
|
struct neighbour *neigh;
|
|
|
|
|
2018-03-23 14:56:58 +00:00
|
|
|
neigh = __ipv6_neigh_lookup_noref(rt->dst.dev, &rt->rt6i_gateway);
|
|
|
|
|
2021-04-01 03:22:23 +00:00
|
|
|
if (!(neigh && (neigh->flags & NTF_ROUTER))) {
|
2017-10-06 19:06:01 +00:00
|
|
|
RT6_TRACE("purging route %p via non-router but gateway\n",
|
|
|
|
rt);
|
|
|
|
rt6_remove_exception(bucket, rt6_ex);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
}
|
2018-01-26 19:40:17 +00:00
|
|
|
|
2017-10-06 19:06:01 +00:00
|
|
|
gc_args->more++;
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
static void fib6_nh_age_exceptions(const struct fib6_nh *nh,
|
2019-05-23 03:27:57 +00:00
|
|
|
struct fib6_gc_args *gc_args,
|
|
|
|
unsigned long now)
|
2017-10-06 19:06:01 +00:00
|
|
|
{
|
|
|
|
struct rt6_exception_bucket *bucket;
|
|
|
|
struct rt6_exception *rt6_ex;
|
|
|
|
struct hlist_node *tmp;
|
|
|
|
int i;
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
if (!rcu_access_pointer(nh->rt6i_exception_bucket))
|
2017-10-06 19:06:01 +00:00
|
|
|
return;
|
|
|
|
|
2018-03-23 14:56:58 +00:00
|
|
|
rcu_read_lock_bh();
|
|
|
|
spin_lock(&rt6_exception_lock);
|
2019-05-23 03:27:58 +00:00
|
|
|
bucket = fib6_nh_get_excptn_bucket(nh, &rt6_exception_lock);
|
2017-10-06 19:06:01 +00:00
|
|
|
if (bucket) {
|
|
|
|
for (i = 0; i < FIB6_EXCEPTION_BUCKET_SIZE; i++) {
|
|
|
|
hlist_for_each_entry_safe(rt6_ex, tmp,
|
|
|
|
&bucket->chain, hlist) {
|
|
|
|
rt6_age_examine_exception(bucket, rt6_ex,
|
|
|
|
gc_args, now);
|
|
|
|
}
|
|
|
|
bucket++;
|
|
|
|
}
|
|
|
|
}
|
2018-03-23 14:56:58 +00:00
|
|
|
spin_unlock(&rt6_exception_lock);
|
|
|
|
rcu_read_unlock_bh();
|
2017-10-06 19:06:01 +00:00
|
|
|
}
|
|
|
|
|
2019-06-08 21:53:28 +00:00
|
|
|
struct fib6_nh_age_excptn_arg {
|
|
|
|
struct fib6_gc_args *gc_args;
|
|
|
|
unsigned long now;
|
|
|
|
};
|
|
|
|
|
|
|
|
static int rt6_nh_age_exceptions(struct fib6_nh *nh, void *_arg)
|
|
|
|
{
|
|
|
|
struct fib6_nh_age_excptn_arg *arg = _arg;
|
|
|
|
|
|
|
|
fib6_nh_age_exceptions(nh, arg->gc_args, arg->now);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
void rt6_age_exceptions(struct fib6_info *f6i,
|
2019-05-23 03:27:57 +00:00
|
|
|
struct fib6_gc_args *gc_args,
|
|
|
|
unsigned long now)
|
|
|
|
{
|
2019-06-08 21:53:28 +00:00
|
|
|
if (f6i->nh) {
|
|
|
|
struct fib6_nh_age_excptn_arg arg = {
|
|
|
|
.gc_args = gc_args,
|
|
|
|
.now = now
|
|
|
|
};
|
|
|
|
|
|
|
|
nexthop_for_each_fib6_nh(f6i->nh, rt6_nh_age_exceptions,
|
|
|
|
&arg);
|
|
|
|
} else {
|
|
|
|
fib6_nh_age_exceptions(f6i->fib6_nh, gc_args, now);
|
|
|
|
}
|
2019-05-23 03:27:57 +00:00
|
|
|
}
|
|
|
|
|
2018-05-10 03:34:21 +00:00
|
|
|
/* must be called with rcu lock held */
|
2019-04-16 21:36:10 +00:00
|
|
|
int fib6_table_lookup(struct net *net, struct fib6_table *table, int oif,
|
|
|
|
struct flowi6 *fl6, struct fib6_result *res, int strict)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2014-10-20 20:42:45 +00:00
|
|
|
struct fib6_node *fn, *saved_fn;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-05-10 03:34:19 +00:00
|
|
|
fn = fib6_node_lookup(&table->tb6_root, &fl6->daddr, &fl6->saddr);
|
2014-10-20 20:42:45 +00:00
|
|
|
saved_fn = fn;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2015-10-12 18:47:10 +00:00
|
|
|
if (fl6->flowi6_flags & FLOWI_FLAG_SKIP_NH_OIF)
|
|
|
|
oif = 0;
|
|
|
|
|
2014-10-20 20:42:43 +00:00
|
|
|
redo_rt6_select:
|
2019-04-16 21:36:10 +00:00
|
|
|
rt6_select(net, fn, oif, res, strict);
|
|
|
|
if (res->f6i == net->ipv6.fib6_null_entry) {
|
2014-10-20 20:42:43 +00:00
|
|
|
fn = fib6_backtrack(fn, &fl6->saddr);
|
|
|
|
if (fn)
|
|
|
|
goto redo_rt6_select;
|
2014-10-20 20:42:45 +00:00
|
|
|
else if (strict & RT6_LOOKUP_F_REACHABLE) {
|
|
|
|
/* also consider unreachable route */
|
|
|
|
strict &= ~RT6_LOOKUP_F_REACHABLE;
|
|
|
|
fn = saved_fn;
|
|
|
|
goto redo_rt6_select;
|
|
|
|
}
|
2014-10-20 20:42:43 +00:00
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:10 +00:00
|
|
|
trace_fib6_table_lookup(net, res, table, fl6);
|
2006-03-21 00:59:08 +00:00
|
|
|
|
2019-04-16 21:36:10 +00:00
|
|
|
return 0;
|
2018-05-10 03:34:21 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
struct rt6_info *ip6_pol_route(struct net *net, struct fib6_table *table,
|
|
|
|
int oif, struct flowi6 *fl6,
|
|
|
|
const struct sk_buff *skb, int flags)
|
|
|
|
{
|
2019-04-16 21:35:59 +00:00
|
|
|
struct fib6_result res = {};
|
2019-06-21 00:36:37 +00:00
|
|
|
struct rt6_info *rt = NULL;
|
2018-05-10 03:34:21 +00:00
|
|
|
int strict = 0;
|
|
|
|
|
2019-06-21 00:36:37 +00:00
|
|
|
WARN_ON_ONCE((flags & RT6_LOOKUP_F_DST_NOREF) &&
|
|
|
|
!rcu_read_lock_held());
|
|
|
|
|
2018-05-10 03:34:21 +00:00
|
|
|
strict |= flags & RT6_LOOKUP_F_IFACE;
|
|
|
|
strict |= flags & RT6_LOOKUP_F_IGNORE_LINKSTATE;
|
|
|
|
if (net->ipv6.devconf_all->forwarding == 0)
|
|
|
|
strict |= RT6_LOOKUP_F_REACHABLE;
|
|
|
|
|
|
|
|
rcu_read_lock();
|
|
|
|
|
2019-04-16 21:36:10 +00:00
|
|
|
fib6_table_lookup(net, table, oif, fl6, &res, strict);
|
2019-06-21 00:36:37 +00:00
|
|
|
if (res.f6i == net->ipv6.fib6_null_entry)
|
|
|
|
goto out;
|
2018-04-18 00:33:23 +00:00
|
|
|
|
2019-04-16 21:35:59 +00:00
|
|
|
fib6_select_path(net, &res, fl6, oif, false, skb, strict);
|
2019-04-09 21:41:17 +00:00
|
|
|
|
2018-04-18 00:33:23 +00:00
|
|
|
/*Search through exception table */
|
2019-04-16 21:36:00 +00:00
|
|
|
rt = rt6_find_cached_rt(&res, &fl6->daddr, &fl6->saddr);
|
2018-04-18 00:33:23 +00:00
|
|
|
if (rt) {
|
2019-06-21 00:36:37 +00:00
|
|
|
goto out;
|
2015-05-23 03:56:03 +00:00
|
|
|
} else if (unlikely((fl6->flowi6_flags & FLOWI_FLAG_KNOWN_NH) &&
|
2019-04-16 21:35:59 +00:00
|
|
|
!res.nh->fib_nh_gw_family)) {
|
2015-05-23 03:56:03 +00:00
|
|
|
/* Create a RTF_CACHE clone which will not be
|
|
|
|
* owned by the fib6 tree. It is for the special case where
|
|
|
|
* the daddr in the skb during the neighbor look-up is different
|
|
|
|
* from the fl6->daddr used to look-up route here.
|
|
|
|
*/
|
2019-06-21 00:36:37 +00:00
|
|
|
rt = ip6_rt_cache_alloc(&res, &fl6->daddr, NULL);
|
2006-08-05 06:20:06 +00:00
|
|
|
|
2019-06-21 00:36:37 +00:00
|
|
|
if (rt) {
|
|
|
|
/* 1 refcnt is taken during ip6_rt_cache_alloc().
|
|
|
|
* As rt6_uncached_list_add() does not consume refcnt,
|
|
|
|
* this refcnt is always returned to the caller even
|
|
|
|
* if caller sets RT6_LOOKUP_F_DST_NOREF flag.
|
2017-06-17 17:42:33 +00:00
|
|
|
*/
|
2019-06-21 00:36:37 +00:00
|
|
|
rt6_uncached_list_add(rt);
|
2017-10-06 19:06:11 +00:00
|
|
|
atomic_inc(&net->ipv6.rt6_stats->fib_rt_uncache);
|
2019-06-21 00:36:37 +00:00
|
|
|
rcu_read_unlock();
|
2015-11-19 20:24:22 +00:00
|
|
|
|
2019-06-21 00:36:37 +00:00
|
|
|
return rt;
|
|
|
|
}
|
2015-05-23 03:56:06 +00:00
|
|
|
} else {
|
|
|
|
/* Get a percpu copy */
|
2017-10-09 04:07:18 +00:00
|
|
|
local_bh_disable();
|
2019-06-21 00:36:37 +00:00
|
|
|
rt = rt6_get_pcpu_route(&res);
|
2015-05-23 03:56:06 +00:00
|
|
|
|
2019-06-21 00:36:37 +00:00
|
|
|
if (!rt)
|
|
|
|
rt = rt6_make_pcpu_route(net, &res);
|
2018-04-18 00:33:25 +00:00
|
|
|
|
2017-10-09 04:07:18 +00:00
|
|
|
local_bh_enable();
|
2015-05-23 03:56:06 +00:00
|
|
|
}
|
2019-06-21 00:36:37 +00:00
|
|
|
out:
|
|
|
|
if (!rt)
|
|
|
|
rt = net->ipv6.ip6_null_entry;
|
|
|
|
if (!(flags & RT6_LOOKUP_F_DST_NOREF))
|
|
|
|
ip6_hold_safe(net, &rt);
|
|
|
|
rcu_read_unlock();
|
|
|
|
|
|
|
|
return rt;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2016-06-13 20:44:19 +00:00
|
|
|
EXPORT_SYMBOL_GPL(ip6_pol_route);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2020-06-23 16:42:32 +00:00
|
|
|
INDIRECT_CALLABLE_SCOPE struct rt6_info *ip6_pol_route_input(struct net *net,
|
2018-03-02 16:32:17 +00:00
|
|
|
struct fib6_table *table,
|
|
|
|
struct flowi6 *fl6,
|
|
|
|
const struct sk_buff *skb,
|
|
|
|
int flags)
|
2007-10-15 20:02:51 +00:00
|
|
|
{
|
2018-03-02 16:32:17 +00:00
|
|
|
return ip6_pol_route(net, table, fl6->flowi6_iif, fl6, skb, flags);
|
2007-10-15 20:02:51 +00:00
|
|
|
}
|
|
|
|
|
2016-09-16 19:59:08 +00:00
|
|
|
struct dst_entry *ip6_route_input_lookup(struct net *net,
|
|
|
|
struct net_device *dev,
|
2018-03-02 16:32:17 +00:00
|
|
|
struct flowi6 *fl6,
|
|
|
|
const struct sk_buff *skb,
|
|
|
|
int flags)
|
2012-04-01 04:03:45 +00:00
|
|
|
{
|
|
|
|
if (rt6_need_strict(&fl6->daddr) && dev->type != ARPHRD_PIMREG)
|
|
|
|
flags |= RT6_LOOKUP_F_IFACE;
|
|
|
|
|
2018-03-02 16:32:17 +00:00
|
|
|
return fib6_rule_lookup(net, fl6, skb, flags, ip6_pol_route_input);
|
2012-04-01 04:03:45 +00:00
|
|
|
}
|
2016-09-16 19:59:08 +00:00
|
|
|
EXPORT_SYMBOL_GPL(ip6_route_input_lookup);
|
2012-04-01 04:03:45 +00:00
|
|
|
|
2017-08-23 07:58:29 +00:00
|
|
|
static void ip6_multipath_l3_keys(const struct sk_buff *skb,
|
2018-03-01 03:43:22 +00:00
|
|
|
struct flow_keys *keys,
|
|
|
|
struct flow_keys *flkeys)
|
2017-08-23 07:58:29 +00:00
|
|
|
{
|
|
|
|
const struct ipv6hdr *outer_iph = ipv6_hdr(skb);
|
|
|
|
const struct ipv6hdr *key_iph = outer_iph;
|
2018-03-01 03:43:22 +00:00
|
|
|
struct flow_keys *_flkeys = flkeys;
|
2017-08-23 07:58:29 +00:00
|
|
|
const struct ipv6hdr *inner_iph;
|
|
|
|
const struct icmp6hdr *icmph;
|
|
|
|
struct ipv6hdr _inner_iph;
|
2018-04-29 16:54:59 +00:00
|
|
|
struct icmp6hdr _icmph;
|
2017-08-23 07:58:29 +00:00
|
|
|
|
|
|
|
if (likely(outer_iph->nexthdr != IPPROTO_ICMPV6))
|
|
|
|
goto out;
|
|
|
|
|
2018-04-29 16:54:59 +00:00
|
|
|
icmph = skb_header_pointer(skb, skb_transport_offset(skb),
|
|
|
|
sizeof(_icmph), &_icmph);
|
|
|
|
if (!icmph)
|
|
|
|
goto out;
|
|
|
|
|
2019-11-02 00:12:04 +00:00
|
|
|
if (!icmpv6_is_err(icmph->icmp6_type))
|
2017-08-23 07:58:29 +00:00
|
|
|
goto out;
|
|
|
|
|
|
|
|
inner_iph = skb_header_pointer(skb,
|
|
|
|
skb_transport_offset(skb) + sizeof(*icmph),
|
|
|
|
sizeof(_inner_iph), &_inner_iph);
|
|
|
|
if (!inner_iph)
|
|
|
|
goto out;
|
|
|
|
|
|
|
|
key_iph = inner_iph;
|
2018-03-01 03:43:22 +00:00
|
|
|
_flkeys = NULL;
|
2017-08-23 07:58:29 +00:00
|
|
|
out:
|
2018-03-01 03:43:22 +00:00
|
|
|
if (_flkeys) {
|
|
|
|
keys->addrs.v6addrs.src = _flkeys->addrs.v6addrs.src;
|
|
|
|
keys->addrs.v6addrs.dst = _flkeys->addrs.v6addrs.dst;
|
|
|
|
keys->tags.flow_label = _flkeys->tags.flow_label;
|
|
|
|
keys->basic.ip_proto = _flkeys->basic.ip_proto;
|
|
|
|
} else {
|
|
|
|
keys->addrs.v6addrs.src = key_iph->saddr;
|
|
|
|
keys->addrs.v6addrs.dst = key_iph->daddr;
|
2018-06-04 09:36:05 +00:00
|
|
|
keys->tags.flow_label = ip6_flowlabel(key_iph);
|
2018-03-01 03:43:22 +00:00
|
|
|
keys->basic.ip_proto = key_iph->nexthdr;
|
|
|
|
}
|
2017-08-23 07:58:29 +00:00
|
|
|
}
|
|
|
|
|
2021-05-17 18:15:23 +00:00
|
|
|
static u32 rt6_multipath_custom_hash_outer(const struct net *net,
|
|
|
|
const struct sk_buff *skb,
|
|
|
|
bool *p_has_inner)
|
|
|
|
{
|
|
|
|
u32 hash_fields = ip6_multipath_hash_fields(net);
|
|
|
|
struct flow_keys keys, hash_keys;
|
|
|
|
|
|
|
|
if (!(hash_fields & FIB_MULTIPATH_HASH_FIELD_OUTER_MASK))
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
memset(&hash_keys, 0, sizeof(hash_keys));
|
|
|
|
skb_flow_dissect_flow_keys(skb, &keys, FLOW_DISSECTOR_F_STOP_AT_ENCAP);
|
|
|
|
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV6_ADDRS;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_SRC_IP)
|
|
|
|
hash_keys.addrs.v6addrs.src = keys.addrs.v6addrs.src;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_DST_IP)
|
|
|
|
hash_keys.addrs.v6addrs.dst = keys.addrs.v6addrs.dst;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_IP_PROTO)
|
|
|
|
hash_keys.basic.ip_proto = keys.basic.ip_proto;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_FLOWLABEL)
|
|
|
|
hash_keys.tags.flow_label = keys.tags.flow_label;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_SRC_PORT)
|
|
|
|
hash_keys.ports.src = keys.ports.src;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_DST_PORT)
|
|
|
|
hash_keys.ports.dst = keys.ports.dst;
|
|
|
|
|
|
|
|
*p_has_inner = !!(keys.control.flags & FLOW_DIS_ENCAPSULATION);
|
|
|
|
return flow_hash_from_keys(&hash_keys);
|
|
|
|
}
|
|
|
|
|
|
|
|
static u32 rt6_multipath_custom_hash_inner(const struct net *net,
|
|
|
|
const struct sk_buff *skb,
|
|
|
|
bool has_inner)
|
|
|
|
{
|
|
|
|
u32 hash_fields = ip6_multipath_hash_fields(net);
|
|
|
|
struct flow_keys keys, hash_keys;
|
|
|
|
|
|
|
|
/* We assume the packet carries an encapsulation, but if none was
|
|
|
|
* encountered during dissection of the outer flow, then there is no
|
|
|
|
* point in calling the flow dissector again.
|
|
|
|
*/
|
|
|
|
if (!has_inner)
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
if (!(hash_fields & FIB_MULTIPATH_HASH_FIELD_INNER_MASK))
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
memset(&hash_keys, 0, sizeof(hash_keys));
|
|
|
|
skb_flow_dissect_flow_keys(skb, &keys, 0);
|
|
|
|
|
|
|
|
if (!(keys.control.flags & FLOW_DIS_ENCAPSULATION))
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
if (keys.control.addr_type == FLOW_DISSECTOR_KEY_IPV4_ADDRS) {
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV4_ADDRS;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_INNER_SRC_IP)
|
|
|
|
hash_keys.addrs.v4addrs.src = keys.addrs.v4addrs.src;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_INNER_DST_IP)
|
|
|
|
hash_keys.addrs.v4addrs.dst = keys.addrs.v4addrs.dst;
|
|
|
|
} else if (keys.control.addr_type == FLOW_DISSECTOR_KEY_IPV6_ADDRS) {
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV6_ADDRS;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_INNER_SRC_IP)
|
|
|
|
hash_keys.addrs.v6addrs.src = keys.addrs.v6addrs.src;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_INNER_DST_IP)
|
|
|
|
hash_keys.addrs.v6addrs.dst = keys.addrs.v6addrs.dst;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_INNER_FLOWLABEL)
|
|
|
|
hash_keys.tags.flow_label = keys.tags.flow_label;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_INNER_IP_PROTO)
|
|
|
|
hash_keys.basic.ip_proto = keys.basic.ip_proto;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_INNER_SRC_PORT)
|
|
|
|
hash_keys.ports.src = keys.ports.src;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_INNER_DST_PORT)
|
|
|
|
hash_keys.ports.dst = keys.ports.dst;
|
|
|
|
|
|
|
|
return flow_hash_from_keys(&hash_keys);
|
|
|
|
}
|
|
|
|
|
|
|
|
static u32 rt6_multipath_custom_hash_skb(const struct net *net,
|
|
|
|
const struct sk_buff *skb)
|
|
|
|
{
|
|
|
|
u32 mhash, mhash_inner;
|
|
|
|
bool has_inner = true;
|
|
|
|
|
|
|
|
mhash = rt6_multipath_custom_hash_outer(net, skb, &has_inner);
|
|
|
|
mhash_inner = rt6_multipath_custom_hash_inner(net, skb, has_inner);
|
|
|
|
|
|
|
|
return jhash_2words(mhash, mhash_inner, 0);
|
|
|
|
}
|
|
|
|
|
|
|
|
static u32 rt6_multipath_custom_hash_fl6(const struct net *net,
|
|
|
|
const struct flowi6 *fl6)
|
|
|
|
{
|
|
|
|
u32 hash_fields = ip6_multipath_hash_fields(net);
|
|
|
|
struct flow_keys hash_keys;
|
|
|
|
|
|
|
|
if (!(hash_fields & FIB_MULTIPATH_HASH_FIELD_OUTER_MASK))
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
memset(&hash_keys, 0, sizeof(hash_keys));
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV6_ADDRS;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_SRC_IP)
|
|
|
|
hash_keys.addrs.v6addrs.src = fl6->saddr;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_DST_IP)
|
|
|
|
hash_keys.addrs.v6addrs.dst = fl6->daddr;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_IP_PROTO)
|
|
|
|
hash_keys.basic.ip_proto = fl6->flowi6_proto;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_FLOWLABEL)
|
|
|
|
hash_keys.tags.flow_label = (__force u32)flowi6_get_flowlabel(fl6);
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_SRC_PORT)
|
|
|
|
hash_keys.ports.src = fl6->fl6_sport;
|
|
|
|
if (hash_fields & FIB_MULTIPATH_HASH_FIELD_DST_PORT)
|
|
|
|
hash_keys.ports.dst = fl6->fl6_dport;
|
|
|
|
|
|
|
|
return flow_hash_from_keys(&hash_keys);
|
|
|
|
}
|
|
|
|
|
2017-08-23 07:58:29 +00:00
|
|
|
/* if skb is set it will be used and fl6 can be NULL */
|
2018-03-02 16:32:18 +00:00
|
|
|
u32 rt6_multipath_hash(const struct net *net, const struct flowi6 *fl6,
|
|
|
|
const struct sk_buff *skb, struct flow_keys *flkeys)
|
2017-08-23 07:58:29 +00:00
|
|
|
{
|
|
|
|
struct flow_keys hash_keys;
|
2021-05-17 18:15:21 +00:00
|
|
|
u32 mhash = 0;
|
2017-08-23 07:58:29 +00:00
|
|
|
|
2018-03-12 15:09:33 +00:00
|
|
|
switch (ip6_multipath_hash_policy(net)) {
|
2018-03-02 16:32:18 +00:00
|
|
|
case 0:
|
|
|
|
memset(&hash_keys, 0, sizeof(hash_keys));
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV6_ADDRS;
|
|
|
|
if (skb) {
|
|
|
|
ip6_multipath_l3_keys(skb, &hash_keys, flkeys);
|
|
|
|
} else {
|
|
|
|
hash_keys.addrs.v6addrs.src = fl6->saddr;
|
|
|
|
hash_keys.addrs.v6addrs.dst = fl6->daddr;
|
2018-06-04 09:36:05 +00:00
|
|
|
hash_keys.tags.flow_label = (__force u32)flowi6_get_flowlabel(fl6);
|
2018-03-02 16:32:18 +00:00
|
|
|
hash_keys.basic.ip_proto = fl6->flowi6_proto;
|
|
|
|
}
|
2021-05-17 18:15:21 +00:00
|
|
|
mhash = flow_hash_from_keys(&hash_keys);
|
2018-03-02 16:32:18 +00:00
|
|
|
break;
|
|
|
|
case 1:
|
|
|
|
if (skb) {
|
|
|
|
unsigned int flag = FLOW_DISSECTOR_F_STOP_AT_ENCAP;
|
|
|
|
struct flow_keys keys;
|
|
|
|
|
|
|
|
/* short-circuit if we already have L4 hash present */
|
|
|
|
if (skb->l4_hash)
|
|
|
|
return skb_get_hash_raw(skb) >> 1;
|
|
|
|
|
|
|
|
memset(&hash_keys, 0, sizeof(hash_keys));
|
|
|
|
|
2021-03-10 20:33:14 +00:00
|
|
|
if (!flkeys) {
|
2018-03-02 16:32:18 +00:00
|
|
|
skb_flow_dissect_flow_keys(skb, &keys, flag);
|
|
|
|
flkeys = &keys;
|
|
|
|
}
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV6_ADDRS;
|
|
|
|
hash_keys.addrs.v6addrs.src = flkeys->addrs.v6addrs.src;
|
|
|
|
hash_keys.addrs.v6addrs.dst = flkeys->addrs.v6addrs.dst;
|
|
|
|
hash_keys.ports.src = flkeys->ports.src;
|
|
|
|
hash_keys.ports.dst = flkeys->ports.dst;
|
|
|
|
hash_keys.basic.ip_proto = flkeys->basic.ip_proto;
|
|
|
|
} else {
|
|
|
|
memset(&hash_keys, 0, sizeof(hash_keys));
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV6_ADDRS;
|
|
|
|
hash_keys.addrs.v6addrs.src = fl6->saddr;
|
|
|
|
hash_keys.addrs.v6addrs.dst = fl6->daddr;
|
|
|
|
hash_keys.ports.src = fl6->fl6_sport;
|
|
|
|
hash_keys.ports.dst = fl6->fl6_dport;
|
|
|
|
hash_keys.basic.ip_proto = fl6->flowi6_proto;
|
|
|
|
}
|
2021-05-17 18:15:21 +00:00
|
|
|
mhash = flow_hash_from_keys(&hash_keys);
|
2018-03-02 16:32:18 +00:00
|
|
|
break;
|
2019-07-06 14:55:18 +00:00
|
|
|
case 2:
|
|
|
|
memset(&hash_keys, 0, sizeof(hash_keys));
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV6_ADDRS;
|
|
|
|
if (skb) {
|
|
|
|
struct flow_keys keys;
|
|
|
|
|
|
|
|
if (!flkeys) {
|
|
|
|
skb_flow_dissect_flow_keys(skb, &keys, 0);
|
|
|
|
flkeys = &keys;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Inner can be v4 or v6 */
|
|
|
|
if (flkeys->control.addr_type == FLOW_DISSECTOR_KEY_IPV4_ADDRS) {
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV4_ADDRS;
|
|
|
|
hash_keys.addrs.v4addrs.src = flkeys->addrs.v4addrs.src;
|
|
|
|
hash_keys.addrs.v4addrs.dst = flkeys->addrs.v4addrs.dst;
|
|
|
|
} else if (flkeys->control.addr_type == FLOW_DISSECTOR_KEY_IPV6_ADDRS) {
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV6_ADDRS;
|
|
|
|
hash_keys.addrs.v6addrs.src = flkeys->addrs.v6addrs.src;
|
|
|
|
hash_keys.addrs.v6addrs.dst = flkeys->addrs.v6addrs.dst;
|
|
|
|
hash_keys.tags.flow_label = flkeys->tags.flow_label;
|
|
|
|
hash_keys.basic.ip_proto = flkeys->basic.ip_proto;
|
|
|
|
} else {
|
|
|
|
/* Same as case 0 */
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV6_ADDRS;
|
|
|
|
ip6_multipath_l3_keys(skb, &hash_keys, flkeys);
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
/* Same as case 0 */
|
|
|
|
hash_keys.control.addr_type = FLOW_DISSECTOR_KEY_IPV6_ADDRS;
|
|
|
|
hash_keys.addrs.v6addrs.src = fl6->saddr;
|
|
|
|
hash_keys.addrs.v6addrs.dst = fl6->daddr;
|
|
|
|
hash_keys.tags.flow_label = (__force u32)flowi6_get_flowlabel(fl6);
|
|
|
|
hash_keys.basic.ip_proto = fl6->flowi6_proto;
|
|
|
|
}
|
2021-05-17 18:15:21 +00:00
|
|
|
mhash = flow_hash_from_keys(&hash_keys);
|
2019-07-06 14:55:18 +00:00
|
|
|
break;
|
2021-05-17 18:15:23 +00:00
|
|
|
case 3:
|
|
|
|
if (skb)
|
|
|
|
mhash = rt6_multipath_custom_hash_skb(net, skb);
|
|
|
|
else
|
|
|
|
mhash = rt6_multipath_custom_hash_fl6(net, fl6);
|
|
|
|
break;
|
2017-08-23 07:58:29 +00:00
|
|
|
}
|
|
|
|
|
2018-03-02 16:32:15 +00:00
|
|
|
return mhash >> 1;
|
2017-08-23 07:58:29 +00:00
|
|
|
}
|
|
|
|
|
2019-06-21 00:36:40 +00:00
|
|
|
/* Called with rcu held */
|
2006-08-05 06:20:06 +00:00
|
|
|
void ip6_route_input(struct sk_buff *skb)
|
|
|
|
{
|
2011-04-22 04:53:02 +00:00
|
|
|
const struct ipv6hdr *iph = ipv6_hdr(skb);
|
2008-03-25 12:47:49 +00:00
|
|
|
struct net *net = dev_net(skb->dev);
|
2019-06-21 00:36:40 +00:00
|
|
|
int flags = RT6_LOOKUP_F_HAS_SADDR | RT6_LOOKUP_F_DST_NOREF;
|
2015-08-20 11:56:31 +00:00
|
|
|
struct ip_tunnel_info *tun_info;
|
2011-03-12 21:22:43 +00:00
|
|
|
struct flowi6 fl6 = {
|
2016-09-10 19:09:57 +00:00
|
|
|
.flowi6_iif = skb->dev->ifindex,
|
2011-03-12 21:22:43 +00:00
|
|
|
.daddr = iph->daddr,
|
|
|
|
.saddr = iph->saddr,
|
2013-01-13 05:01:51 +00:00
|
|
|
.flowlabel = ip6_flowinfo(iph),
|
2011-03-12 21:22:43 +00:00
|
|
|
.flowi6_mark = skb->mark,
|
|
|
|
.flowi6_proto = iph->nexthdr,
|
2006-08-05 06:20:06 +00:00
|
|
|
};
|
2018-03-01 03:43:22 +00:00
|
|
|
struct flow_keys *flkeys = NULL, _flkeys;
|
2006-10-13 22:01:03 +00:00
|
|
|
|
2015-08-20 11:56:31 +00:00
|
|
|
tun_info = skb_tunnel_info(skb);
|
2015-08-28 18:48:19 +00:00
|
|
|
if (tun_info && !(tun_info->mode & IP_TUNNEL_INFO_TX))
|
2015-08-20 11:56:31 +00:00
|
|
|
fl6.flowi6_tun_key.tun_id = tun_info->key.tun_id;
|
2018-03-01 03:43:22 +00:00
|
|
|
|
|
|
|
if (fib6_rules_early_flow_dissect(net, skb, &fl6, &_flkeys))
|
|
|
|
flkeys = &_flkeys;
|
|
|
|
|
2017-08-23 07:58:29 +00:00
|
|
|
if (unlikely(fl6.flowi6_proto == IPPROTO_ICMPV6))
|
2018-03-02 16:32:18 +00:00
|
|
|
fl6.mp_hash = rt6_multipath_hash(net, &fl6, skb, flkeys);
|
2015-08-20 11:56:26 +00:00
|
|
|
skb_dst_drop(skb);
|
2019-06-21 00:36:40 +00:00
|
|
|
skb_dst_set_noref(skb, ip6_route_input_lookup(net, skb->dev,
|
|
|
|
&fl6, skb, flags));
|
2006-08-05 06:20:06 +00:00
|
|
|
}
|
|
|
|
|
2020-06-23 16:42:32 +00:00
|
|
|
INDIRECT_CALLABLE_SCOPE struct rt6_info *ip6_pol_route_output(struct net *net,
|
2018-03-02 16:32:17 +00:00
|
|
|
struct fib6_table *table,
|
|
|
|
struct flowi6 *fl6,
|
|
|
|
const struct sk_buff *skb,
|
|
|
|
int flags)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2018-03-02 16:32:17 +00:00
|
|
|
return ip6_pol_route(net, table, fl6->flowi6_oif, fl6, skb, flags);
|
2006-08-05 06:20:06 +00:00
|
|
|
}
|
|
|
|
|
2019-06-21 00:36:41 +00:00
|
|
|
struct dst_entry *ip6_route_output_flags_noref(struct net *net,
|
|
|
|
const struct sock *sk,
|
|
|
|
struct flowi6 *fl6, int flags)
|
2006-08-05 06:20:06 +00:00
|
|
|
{
|
2015-10-21 15:42:22 +00:00
|
|
|
bool any_src;
|
2006-08-05 06:20:06 +00:00
|
|
|
|
2018-09-19 12:56:53 +00:00
|
|
|
if (ipv6_addr_type(&fl6->daddr) &
|
|
|
|
(IPV6_ADDR_MULTICAST | IPV6_ADDR_LINKLOCAL)) {
|
2016-09-10 19:09:56 +00:00
|
|
|
struct dst_entry *dst;
|
|
|
|
|
2019-06-21 00:36:41 +00:00
|
|
|
/* This function does not take refcnt on the dst */
|
2016-09-10 19:09:56 +00:00
|
|
|
dst = l3mdev_link_scope_lookup(net, fl6);
|
|
|
|
if (dst)
|
|
|
|
return dst;
|
|
|
|
}
|
2015-10-12 18:47:10 +00:00
|
|
|
|
2012-08-08 21:53:36 +00:00
|
|
|
fl6->flowi6_iif = LOOPBACK_IFINDEX;
|
2012-06-25 15:42:26 +00:00
|
|
|
|
2019-06-21 00:36:41 +00:00
|
|
|
flags |= RT6_LOOKUP_F_DST_NOREF;
|
2015-10-21 15:42:22 +00:00
|
|
|
any_src = ipv6_addr_any(&fl6->saddr);
|
2015-09-28 17:12:13 +00:00
|
|
|
if ((sk && sk->sk_bound_dev_if) || rt6_need_strict(&fl6->daddr) ||
|
2015-10-21 15:42:22 +00:00
|
|
|
(fl6->flowi6_oif && any_src))
|
2006-08-24 00:25:05 +00:00
|
|
|
flags |= RT6_LOOKUP_F_IFACE;
|
2006-08-05 06:20:06 +00:00
|
|
|
|
2015-10-21 15:42:22 +00:00
|
|
|
if (!any_src)
|
2006-10-13 22:01:03 +00:00
|
|
|
flags |= RT6_LOOKUP_F_HAS_SADDR;
|
2010-03-07 00:14:44 +00:00
|
|
|
else if (sk)
|
|
|
|
flags |= rt6_srcprefs2flags(inet6_sk(sk)->srcprefs);
|
2006-10-13 22:01:03 +00:00
|
|
|
|
2018-03-02 16:32:17 +00:00
|
|
|
return fib6_rule_lookup(net, fl6, NULL, flags, ip6_pol_route_output);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2019-06-21 00:36:41 +00:00
|
|
|
EXPORT_SYMBOL_GPL(ip6_route_output_flags_noref);
|
|
|
|
|
|
|
|
struct dst_entry *ip6_route_output_flags(struct net *net,
|
|
|
|
const struct sock *sk,
|
|
|
|
struct flowi6 *fl6,
|
|
|
|
int flags)
|
|
|
|
{
|
2021-03-10 20:33:14 +00:00
|
|
|
struct dst_entry *dst;
|
|
|
|
struct rt6_info *rt6;
|
2019-06-21 00:36:41 +00:00
|
|
|
|
2021-03-10 20:33:14 +00:00
|
|
|
rcu_read_lock();
|
|
|
|
dst = ip6_route_output_flags_noref(net, sk, fl6, flags);
|
|
|
|
rt6 = (struct rt6_info *)dst;
|
|
|
|
/* For dst cached in uncached_list, refcnt is already taken. */
|
|
|
|
if (list_empty(&rt6->rt6i_uncached) && !dst_hold_safe(dst)) {
|
|
|
|
dst = &net->ipv6.ip6_null_entry->dst;
|
|
|
|
dst_hold(dst);
|
|
|
|
}
|
|
|
|
rcu_read_unlock();
|
2019-06-21 00:36:41 +00:00
|
|
|
|
2021-03-10 20:33:14 +00:00
|
|
|
return dst;
|
2019-06-21 00:36:41 +00:00
|
|
|
}
|
2016-01-29 11:30:19 +00:00
|
|
|
EXPORT_SYMBOL_GPL(ip6_route_output_flags);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2011-03-01 22:59:04 +00:00
|
|
|
struct dst_entry *ip6_blackhole_route(struct net *net, struct dst_entry *dst_orig)
|
2007-05-25 01:17:54 +00:00
|
|
|
{
|
2011-04-28 21:13:38 +00:00
|
|
|
struct rt6_info *rt, *ort = (struct rt6_info *) dst_orig;
|
2017-06-17 17:42:26 +00:00
|
|
|
struct net_device *loopback_dev = net->loopback_dev;
|
2007-05-25 01:17:54 +00:00
|
|
|
struct dst_entry *new = NULL;
|
|
|
|
|
2017-06-17 17:42:26 +00:00
|
|
|
rt = dst_alloc(&ip6_dst_blackhole_ops, loopback_dev, 1,
|
2017-10-09 06:39:43 +00:00
|
|
|
DST_OBSOLETE_DEAD, 0);
|
2007-05-25 01:17:54 +00:00
|
|
|
if (rt) {
|
2015-10-15 23:39:58 +00:00
|
|
|
rt6_info_init(rt);
|
2017-10-06 19:06:11 +00:00
|
|
|
atomic_inc(&net->ipv6.rt6_stats->fib_rt_alloc);
|
2012-07-05 23:37:09 +00:00
|
|
|
|
2015-10-15 23:39:58 +00:00
|
|
|
new = &rt->dst;
|
2007-05-25 01:17:54 +00:00
|
|
|
new->__use = 1;
|
2007-11-14 05:34:06 +00:00
|
|
|
new->input = dst_discard;
|
2015-10-07 21:48:47 +00:00
|
|
|
new->output = dst_discard_out;
|
2007-05-25 01:17:54 +00:00
|
|
|
|
2015-10-15 23:39:58 +00:00
|
|
|
dst_copy_metrics(new, &ort->dst);
|
2007-05-25 01:17:54 +00:00
|
|
|
|
2017-06-17 17:42:26 +00:00
|
|
|
rt->rt6i_idev = in6_dev_get(loopback_dev);
|
2011-11-21 03:39:03 +00:00
|
|
|
rt->rt6i_gateway = ort->rt6i_gateway;
|
2015-10-15 23:39:58 +00:00
|
|
|
rt->rt6i_flags = ort->rt6i_flags & ~RTF_PCPU;
|
2007-05-25 01:17:54 +00:00
|
|
|
|
|
|
|
memcpy(&rt->rt6i_dst, &ort->rt6i_dst, sizeof(struct rt6key));
|
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
|
|
|
memcpy(&rt->rt6i_src, &ort->rt6i_src, sizeof(struct rt6key));
|
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
2011-03-01 22:45:33 +00:00
|
|
|
dst_release(dst_orig);
|
|
|
|
return new ? new : ERR_PTR(-ENOMEM);
|
2007-05-25 01:17:54 +00:00
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/*
|
|
|
|
* Destination cache support functions
|
|
|
|
*/
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static bool fib6_check(struct fib6_info *f6i, u32 cookie)
|
ipv6: Stop rt6_info from using inet_peer's metrics
inet_peer is indexed by the dst address alone. However, the fib6 tree
could have multiple routing entries (rt6_info) for the same dst. For
example,
1. A /128 dst via multiple gateways.
2. A RTF_CACHE route cloned from a /128 route.
In the above cases, all of them will share the same metrics and
step on each other.
This patch will steer away from inet_peer's metrics and use
dst_cow_metrics_generic() for everything.
Change Highlights:
1. Remove rt6_cow_metrics() which currently acquires metrics from
inet_peer for DST_HOST route (i.e. /128 route).
2. Add rt6i_pmtu to take care of the pmtu update to avoid creating a
full size metrics just to override the RTAX_MTU.
3. After (2), the RTF_CACHE route can also share the metrics with its
dst.from route, by:
dst_init_metrics(&cache_rt->dst, dst_metrics_ptr(cache_rt->dst.from), true);
4. Stop creating RTF_CACHE route by cloning another RTF_CACHE route. Instead,
directly clone from rt->dst.
[ Currently, cloning from another RTF_CACHE is only possible during
rt6_do_redirect(). Also, the old clone is removed from the tree
immediately after the new clone is added. ]
In case of cloning from an older redirect RTF_CACHE, it should work as
before.
In case of cloning from an older pmtu RTF_CACHE, this patch will forget
the pmtu and re-learn it (if there is any) from the redirected route.
The _rt6i_peer and DST_METRICS_FORCE_OVERWRITE will be removed
in the next cleanup patch.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Reviewed-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Cc: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2015-04-28 20:03:06 +00:00
|
|
|
{
|
2018-04-18 00:33:25 +00:00
|
|
|
u32 rt_cookie = 0;
|
|
|
|
|
2018-04-20 22:38:03 +00:00
|
|
|
if (!fib6_get_cookie_safe(f6i, &rt_cookie) || rt_cookie != cookie)
|
2018-04-18 00:33:25 +00:00
|
|
|
return false;
|
|
|
|
|
|
|
|
if (fib6_check_expired(f6i))
|
|
|
|
return false;
|
|
|
|
|
|
|
|
return true;
|
ipv6: Stop rt6_info from using inet_peer's metrics
inet_peer is indexed by the dst address alone. However, the fib6 tree
could have multiple routing entries (rt6_info) for the same dst. For
example,
1. A /128 dst via multiple gateways.
2. A RTF_CACHE route cloned from a /128 route.
In the above cases, all of them will share the same metrics and
step on each other.
This patch will steer away from inet_peer's metrics and use
dst_cow_metrics_generic() for everything.
Change Highlights:
1. Remove rt6_cow_metrics() which currently acquires metrics from
inet_peer for DST_HOST route (i.e. /128 route).
2. Add rt6i_pmtu to take care of the pmtu update to avoid creating a
full size metrics just to override the RTAX_MTU.
3. After (2), the RTF_CACHE route can also share the metrics with its
dst.from route, by:
dst_init_metrics(&cache_rt->dst, dst_metrics_ptr(cache_rt->dst.from), true);
4. Stop creating RTF_CACHE route by cloning another RTF_CACHE route. Instead,
directly clone from rt->dst.
[ Currently, cloning from another RTF_CACHE is only possible during
rt6_do_redirect(). Also, the old clone is removed from the tree
immediately after the new clone is added. ]
In case of cloning from an older redirect RTF_CACHE, it should work as
before.
In case of cloning from an older pmtu RTF_CACHE, this patch will forget
the pmtu and re-learn it (if there is any) from the redirected route.
The _rt6i_peer and DST_METRICS_FORCE_OVERWRITE will be removed
in the next cleanup patch.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Reviewed-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Cc: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2015-04-28 20:03:06 +00:00
|
|
|
}
|
|
|
|
|
2018-04-20 22:38:02 +00:00
|
|
|
static struct dst_entry *rt6_check(struct rt6_info *rt,
|
|
|
|
struct fib6_info *from,
|
|
|
|
u32 cookie)
|
2015-05-23 03:56:03 +00:00
|
|
|
{
|
2017-08-25 07:05:42 +00:00
|
|
|
u32 rt_cookie = 0;
|
2017-08-21 16:47:10 +00:00
|
|
|
|
2019-07-17 22:08:43 +00:00
|
|
|
if (!from || !fib6_get_cookie_safe(from, &rt_cookie) ||
|
2018-04-18 00:33:25 +00:00
|
|
|
rt_cookie != cookie)
|
2015-05-23 03:56:03 +00:00
|
|
|
return NULL;
|
|
|
|
|
|
|
|
if (rt6_check_expired(rt))
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
return &rt->dst;
|
|
|
|
}
|
|
|
|
|
2018-04-20 22:38:02 +00:00
|
|
|
static struct dst_entry *rt6_dst_from_check(struct rt6_info *rt,
|
|
|
|
struct fib6_info *from,
|
|
|
|
u32 cookie)
|
2015-05-23 03:56:03 +00:00
|
|
|
{
|
2015-11-11 19:51:07 +00:00
|
|
|
if (!__rt6_check_expired(rt) &&
|
|
|
|
rt->dst.obsolete == DST_OBSOLETE_FORCE_CHK &&
|
2018-04-20 22:38:02 +00:00
|
|
|
fib6_check(from, cookie))
|
2015-05-23 03:56:03 +00:00
|
|
|
return &rt->dst;
|
|
|
|
else
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2021-02-01 17:41:32 +00:00
|
|
|
INDIRECT_CALLABLE_SCOPE struct dst_entry *ip6_dst_check(struct dst_entry *dst,
|
|
|
|
u32 cookie)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2018-04-20 22:38:00 +00:00
|
|
|
struct dst_entry *dst_ret;
|
2018-04-20 22:38:02 +00:00
|
|
|
struct fib6_info *from;
|
2005-04-16 22:20:36 +00:00
|
|
|
struct rt6_info *rt;
|
|
|
|
|
2018-04-20 22:38:00 +00:00
|
|
|
rt = container_of(dst, struct rt6_info, dst);
|
|
|
|
|
ipv6: Use global sernum for dst validation with nexthop objects
Nik reported a bug with pcpu dst cache when nexthop objects are
used illustrated by the following:
$ ip netns add foo
$ ip -netns foo li set lo up
$ ip -netns foo addr add 2001:db8:11::1/128 dev lo
$ ip netns exec foo sysctl net.ipv6.conf.all.forwarding=1
$ ip li add veth1 type veth peer name veth2
$ ip li set veth1 up
$ ip addr add 2001:db8:10::1/64 dev veth1
$ ip li set dev veth2 netns foo
$ ip -netns foo li set veth2 up
$ ip -netns foo addr add 2001:db8:10::2/64 dev veth2
$ ip -6 nexthop add id 100 via 2001:db8:10::2 dev veth1
$ ip -6 route add 2001:db8:11::1/128 nhid 100
Create a pcpu entry on cpu 0:
$ taskset -a -c 0 ip -6 route get 2001:db8:11::1
Re-add the route entry:
$ ip -6 ro del 2001:db8:11::1
$ ip -6 route add 2001:db8:11::1/128 nhid 100
Route get on cpu 0 returns the stale pcpu:
$ taskset -a -c 0 ip -6 route get 2001:db8:11::1
RTNETLINK answers: Network is unreachable
While cpu 1 works:
$ taskset -a -c 1 ip -6 route get 2001:db8:11::1
2001:db8:11::1 from :: via 2001:db8:10::2 dev veth1 src 2001:db8:10::1 metric 1024 pref medium
Conversion of FIB entries to work with external nexthop objects
missed an important difference between IPv4 and IPv6 - how dst
entries are invalidated when the FIB changes. IPv4 has a per-network
namespace generation id (rt_genid) that is bumped on changes to the FIB.
Checking if a dst_entry is still valid means comparing rt_genid in the
rtable to the current value of rt_genid for the namespace.
IPv6 also has a per network namespace counter, fib6_sernum, but the
count is saved per fib6_node. With the per-node counter only dst_entries
based on fib entries under the node are invalidated when changes are
made to the routes - limiting the scope of invalidations. IPv6 uses a
reference in the rt6_info, 'from', to track the corresponding fib entry
used to create the dst_entry. When validating a dst_entry, the 'from'
is used to backtrack to the fib6_node and check the sernum of it to the
cookie passed to the dst_check operation.
With the inline format (nexthop definition inline with the fib6_info),
dst_entries cached in the fib6_nh have a 1:1 correlation between fib
entries, nexthop data and dst_entries. With external nexthops, IPv6
looks more like IPv4 which means multiple fib entries across disparate
fib6_nodes can all reference the same fib6_nh. That means validation
of dst_entries based on external nexthops needs to use the IPv4 format
- the per-network namespace counter.
Add sernum to rt6_info and set it when creating a pcpu dst entry. Update
rt6_get_cookie to return sernum if it is set and update dst_check for
IPv6 to look for sernum set and based the check on it if so. Finally,
rt6_get_pcpu_route needs to validate the cached entry before returning
a pcpu entry (similar to the rt_cache_valid calls in __mkroute_input and
__mkroute_output for IPv4).
This problem only affects routes using the new, external nexthops.
Thanks to the kbuild test robot for catching the IS_ENABLED needed
around rt_genid_ipv6 before I sent this out.
Fixes: 5b98324ebe29 ("ipv6: Allow routes to use nexthop objects")
Reported-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Signed-off-by: David Ahern <dsahern@kernel.org>
Reviewed-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Tested-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2020-05-01 14:53:08 +00:00
|
|
|
if (rt->sernum)
|
|
|
|
return rt6_is_valid(rt) ? dst : NULL;
|
|
|
|
|
2018-04-20 22:38:00 +00:00
|
|
|
rcu_read_lock();
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2012-09-10 22:09:46 +00:00
|
|
|
/* All IPV6 dsts are created with ->obsolete set to the value
|
|
|
|
* DST_OBSOLETE_FORCE_CHK which forces validation calls down
|
|
|
|
* into this function always.
|
|
|
|
*/
|
2013-10-24 05:48:24 +00:00
|
|
|
|
2018-04-20 22:38:02 +00:00
|
|
|
from = rcu_dereference(rt->from);
|
ipv6: Stop rt6_info from using inet_peer's metrics
inet_peer is indexed by the dst address alone. However, the fib6 tree
could have multiple routing entries (rt6_info) for the same dst. For
example,
1. A /128 dst via multiple gateways.
2. A RTF_CACHE route cloned from a /128 route.
In the above cases, all of them will share the same metrics and
step on each other.
This patch will steer away from inet_peer's metrics and use
dst_cow_metrics_generic() for everything.
Change Highlights:
1. Remove rt6_cow_metrics() which currently acquires metrics from
inet_peer for DST_HOST route (i.e. /128 route).
2. Add rt6i_pmtu to take care of the pmtu update to avoid creating a
full size metrics just to override the RTAX_MTU.
3. After (2), the RTF_CACHE route can also share the metrics with its
dst.from route, by:
dst_init_metrics(&cache_rt->dst, dst_metrics_ptr(cache_rt->dst.from), true);
4. Stop creating RTF_CACHE route by cloning another RTF_CACHE route. Instead,
directly clone from rt->dst.
[ Currently, cloning from another RTF_CACHE is only possible during
rt6_do_redirect(). Also, the old clone is removed from the tree
immediately after the new clone is added. ]
In case of cloning from an older redirect RTF_CACHE, it should work as
before.
In case of cloning from an older pmtu RTF_CACHE, this patch will forget
the pmtu and re-learn it (if there is any) from the redirected route.
The _rt6i_peer and DST_METRICS_FORCE_OVERWRITE will be removed
in the next cleanup patch.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Reviewed-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Cc: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2015-04-28 20:03:06 +00:00
|
|
|
|
2018-04-20 22:38:02 +00:00
|
|
|
if (from && (rt->rt6i_flags & RTF_PCPU ||
|
|
|
|
unlikely(!list_empty(&rt->rt6i_uncached))))
|
|
|
|
dst_ret = rt6_dst_from_check(rt, from, cookie);
|
2015-05-23 03:56:03 +00:00
|
|
|
else
|
2018-04-20 22:38:02 +00:00
|
|
|
dst_ret = rt6_check(rt, from, cookie);
|
2018-04-20 22:38:00 +00:00
|
|
|
|
|
|
|
rcu_read_unlock();
|
|
|
|
|
|
|
|
return dst_ret;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2021-02-04 18:18:39 +00:00
|
|
|
EXPORT_INDIRECT_CALLABLE(ip6_dst_check);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
static struct dst_entry *ip6_negative_advice(struct dst_entry *dst)
|
|
|
|
{
|
|
|
|
struct rt6_info *rt = (struct rt6_info *) dst;
|
|
|
|
|
|
|
|
if (rt) {
|
2010-03-28 07:15:45 +00:00
|
|
|
if (rt->rt6i_flags & RTF_CACHE) {
|
2018-04-23 18:32:06 +00:00
|
|
|
rcu_read_lock();
|
2010-03-28 07:15:45 +00:00
|
|
|
if (rt6_check_expired(rt)) {
|
2018-04-18 00:33:25 +00:00
|
|
|
rt6_remove_exception_rt(rt);
|
2010-03-28 07:15:45 +00:00
|
|
|
dst = NULL;
|
|
|
|
}
|
2018-04-23 18:32:06 +00:00
|
|
|
rcu_read_unlock();
|
2010-03-28 07:15:45 +00:00
|
|
|
} else {
|
2005-04-16 22:20:36 +00:00
|
|
|
dst_release(dst);
|
2010-03-28 07:15:45 +00:00
|
|
|
dst = NULL;
|
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2010-03-28 07:15:45 +00:00
|
|
|
return dst;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static void ip6_link_failure(struct sk_buff *skb)
|
|
|
|
{
|
|
|
|
struct rt6_info *rt;
|
|
|
|
|
2010-02-18 08:25:24 +00:00
|
|
|
icmpv6_send(skb, ICMPV6_DEST_UNREACH, ICMPV6_ADDR_UNREACH, 0);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2009-06-02 05:19:30 +00:00
|
|
|
rt = (struct rt6_info *) skb_dst(skb);
|
2005-04-16 22:20:36 +00:00
|
|
|
if (rt) {
|
2018-04-23 18:32:07 +00:00
|
|
|
rcu_read_lock();
|
2013-07-10 21:00:57 +00:00
|
|
|
if (rt->rt6i_flags & RTF_CACHE) {
|
2018-11-13 16:48:28 +00:00
|
|
|
rt6_remove_exception_rt(rt);
|
2017-08-21 16:47:10 +00:00
|
|
|
} else {
|
2018-04-20 22:38:02 +00:00
|
|
|
struct fib6_info *from;
|
2017-08-21 16:47:10 +00:00
|
|
|
struct fib6_node *fn;
|
|
|
|
|
2018-04-20 22:38:02 +00:00
|
|
|
from = rcu_dereference(rt->from);
|
|
|
|
if (from) {
|
|
|
|
fn = rcu_dereference(from->fib6_node);
|
|
|
|
if (fn && (rt->rt6i_flags & RTF_DEFAULT))
|
|
|
|
fn->fn_sernum = -1;
|
|
|
|
}
|
2013-07-10 21:00:57 +00:00
|
|
|
}
|
2018-04-23 18:32:07 +00:00
|
|
|
rcu_read_unlock();
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-04-20 22:37:57 +00:00
|
|
|
static void rt6_update_expires(struct rt6_info *rt0, int timeout)
|
|
|
|
{
|
2018-04-20 22:38:02 +00:00
|
|
|
if (!(rt0->rt6i_flags & RTF_EXPIRES)) {
|
|
|
|
struct fib6_info *from;
|
|
|
|
|
|
|
|
rcu_read_lock();
|
|
|
|
from = rcu_dereference(rt0->from);
|
|
|
|
if (from)
|
|
|
|
rt0->dst.expires = from->expires;
|
|
|
|
rcu_read_unlock();
|
|
|
|
}
|
2018-04-20 22:37:57 +00:00
|
|
|
|
|
|
|
dst_set_expires(&rt0->dst, timeout);
|
|
|
|
rt0->rt6i_flags |= RTF_EXPIRES;
|
|
|
|
}
|
|
|
|
|
2015-05-23 03:56:00 +00:00
|
|
|
static void rt6_do_update_pmtu(struct rt6_info *rt, u32 mtu)
|
|
|
|
{
|
|
|
|
struct net *net = dev_net(rt->dst.dev);
|
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
dst_metric_set(&rt->dst, RTAX_MTU, mtu);
|
2015-05-23 03:56:00 +00:00
|
|
|
rt->rt6i_flags |= RTF_MODIFIED;
|
|
|
|
rt6_update_expires(rt, net->ipv6.sysctl.ip6_rt_mtu_expires);
|
|
|
|
}
|
|
|
|
|
2015-11-11 19:51:06 +00:00
|
|
|
static bool rt6_cache_allowed_for_pmtu(const struct rt6_info *rt)
|
|
|
|
{
|
|
|
|
return !(rt->rt6i_flags & RTF_CACHE) &&
|
2019-02-15 17:15:37 +00:00
|
|
|
(rt->rt6i_flags & RTF_PCPU || rcu_access_pointer(rt->from));
|
2015-11-11 19:51:06 +00:00
|
|
|
}
|
|
|
|
|
2015-05-23 03:56:00 +00:00
|
|
|
static void __ip6_rt_update_pmtu(struct dst_entry *dst, const struct sock *sk,
|
2019-12-22 02:51:09 +00:00
|
|
|
const struct ipv6hdr *iph, u32 mtu,
|
|
|
|
bool confirm_neigh)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2017-02-06 21:14:16 +00:00
|
|
|
const struct in6_addr *daddr, *saddr;
|
2014-08-24 20:53:10 +00:00
|
|
|
struct rt6_info *rt6 = (struct rt6_info *)dst;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2020-05-05 18:57:23 +00:00
|
|
|
/* Note: do *NOT* check dst_metric_locked(dst, RTAX_MTU)
|
|
|
|
* IPv6 pmtu discovery isn't optional, so 'mtu lock' cannot disable it.
|
|
|
|
* [see also comment in rt6_mtu_change_route()]
|
|
|
|
*/
|
2016-10-28 10:18:01 +00:00
|
|
|
|
2017-02-06 21:14:16 +00:00
|
|
|
if (iph) {
|
|
|
|
daddr = &iph->daddr;
|
|
|
|
saddr = &iph->saddr;
|
|
|
|
} else if (sk) {
|
|
|
|
daddr = &sk->sk_v6_daddr;
|
|
|
|
saddr = &inet6_sk(sk)->saddr;
|
|
|
|
} else {
|
|
|
|
daddr = NULL;
|
|
|
|
saddr = NULL;
|
|
|
|
}
|
2019-12-22 02:51:09 +00:00
|
|
|
|
|
|
|
if (confirm_neigh)
|
|
|
|
dst_confirm_neigh(dst, daddr);
|
|
|
|
|
2020-10-07 12:53:02 +00:00
|
|
|
if (mtu < IPV6_MIN_MTU)
|
|
|
|
return;
|
2015-05-23 03:56:00 +00:00
|
|
|
if (mtu >= dst_mtu(dst))
|
|
|
|
return;
|
2015-01-15 21:34:25 +00:00
|
|
|
|
2015-11-11 19:51:06 +00:00
|
|
|
if (!rt6_cache_allowed_for_pmtu(rt6)) {
|
2015-05-23 03:56:00 +00:00
|
|
|
rt6_do_update_pmtu(rt6, mtu);
|
2017-10-06 19:06:03 +00:00
|
|
|
/* update rt6_ex->stamp for cache */
|
|
|
|
if (rt6->rt6i_flags & RTF_CACHE)
|
|
|
|
rt6_update_exception_stamp_rt(rt6);
|
2017-02-06 21:14:16 +00:00
|
|
|
} else if (daddr) {
|
2019-04-16 21:36:01 +00:00
|
|
|
struct fib6_result res = {};
|
2015-05-23 03:56:00 +00:00
|
|
|
struct rt6_info *nrt6;
|
|
|
|
|
2018-04-20 22:37:59 +00:00
|
|
|
rcu_read_lock();
|
2019-04-16 21:36:01 +00:00
|
|
|
res.f6i = rcu_dereference(rt6->from);
|
2019-08-01 22:18:08 +00:00
|
|
|
if (!res.f6i)
|
|
|
|
goto out_unlock;
|
|
|
|
|
2019-04-16 21:36:11 +00:00
|
|
|
res.fib6_flags = res.f6i->fib6_flags;
|
|
|
|
res.fib6_type = res.f6i->fib6_type;
|
|
|
|
|
2019-06-08 21:53:31 +00:00
|
|
|
if (res.f6i->nh) {
|
|
|
|
struct fib6_nh_match_arg arg = {
|
|
|
|
.dev = dst->dev,
|
|
|
|
.gw = &rt6->rt6i_gateway,
|
|
|
|
};
|
|
|
|
|
|
|
|
nexthop_for_each_fib6_nh(res.f6i->nh,
|
|
|
|
fib6_nh_find_match, &arg);
|
|
|
|
|
|
|
|
/* fib6_info uses a nexthop that does not have fib6_nh
|
|
|
|
* using the dst->dev + gw. Should be impossible.
|
|
|
|
*/
|
2019-08-01 22:18:08 +00:00
|
|
|
if (!arg.match)
|
|
|
|
goto out_unlock;
|
2019-06-08 21:53:31 +00:00
|
|
|
|
|
|
|
res.nh = arg.match;
|
|
|
|
} else {
|
|
|
|
res.nh = res.f6i->fib6_nh;
|
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:01 +00:00
|
|
|
nrt6 = ip6_rt_cache_alloc(&res, daddr, saddr);
|
2015-05-23 03:56:00 +00:00
|
|
|
if (nrt6) {
|
|
|
|
rt6_do_update_pmtu(nrt6, mtu);
|
2019-04-16 21:36:05 +00:00
|
|
|
if (rt6_insert_exception(nrt6, &res))
|
2017-10-06 19:06:03 +00:00
|
|
|
dst_release_immediate(&nrt6->dst);
|
2015-05-23 03:56:00 +00:00
|
|
|
}
|
2019-08-01 22:18:08 +00:00
|
|
|
out_unlock:
|
2018-04-20 22:38:02 +00:00
|
|
|
rcu_read_unlock();
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2015-05-23 03:56:00 +00:00
|
|
|
static void ip6_rt_update_pmtu(struct dst_entry *dst, struct sock *sk,
|
2019-12-22 02:51:09 +00:00
|
|
|
struct sk_buff *skb, u32 mtu,
|
|
|
|
bool confirm_neigh)
|
2015-05-23 03:56:00 +00:00
|
|
|
{
|
2019-12-22 02:51:09 +00:00
|
|
|
__ip6_rt_update_pmtu(dst, sk, skb ? ipv6_hdr(skb) : NULL, mtu,
|
|
|
|
confirm_neigh);
|
2015-05-23 03:56:00 +00:00
|
|
|
}
|
|
|
|
|
2012-06-16 03:01:57 +00:00
|
|
|
void ip6_update_pmtu(struct sk_buff *skb, struct net *net, __be32 mtu,
|
2016-11-03 17:23:43 +00:00
|
|
|
int oif, u32 mark, kuid_t uid)
|
2012-06-15 21:54:11 +00:00
|
|
|
{
|
|
|
|
const struct ipv6hdr *iph = (struct ipv6hdr *) skb->data;
|
|
|
|
struct dst_entry *dst;
|
2018-09-30 06:44:51 +00:00
|
|
|
struct flowi6 fl6 = {
|
|
|
|
.flowi6_oif = oif,
|
|
|
|
.flowi6_mark = mark ? mark : IP6_REPLY_MARK(net, skb->mark),
|
|
|
|
.daddr = iph->daddr,
|
|
|
|
.saddr = iph->saddr,
|
|
|
|
.flowlabel = ip6_flowinfo(iph),
|
|
|
|
.flowi6_uid = uid,
|
|
|
|
};
|
2012-06-15 21:54:11 +00:00
|
|
|
|
|
|
|
dst = ip6_route_output(net, NULL, &fl6);
|
|
|
|
if (!dst->error)
|
2019-12-22 02:51:09 +00:00
|
|
|
__ip6_rt_update_pmtu(dst, NULL, iph, ntohl(mtu), true);
|
2012-06-15 21:54:11 +00:00
|
|
|
dst_release(dst);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(ip6_update_pmtu);
|
|
|
|
|
|
|
|
void ip6_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, __be32 mtu)
|
|
|
|
{
|
2018-11-18 18:45:30 +00:00
|
|
|
int oif = sk->sk_bound_dev_if;
|
ipv6: datagram: Update dst cache of a connected datagram sk during pmtu update
There is a case in connected UDP socket such that
getsockopt(IPV6_MTU) will return a stale MTU value. The reproducible
sequence could be the following:
1. Create a connected UDP socket
2. Send some datagrams out
3. Receive a ICMPV6_PKT_TOOBIG
4. No new outgoing datagrams to trigger the sk_dst_check()
logic to update the sk->sk_dst_cache.
5. getsockopt(IPV6_MTU) returns the mtu from the invalid
sk->sk_dst_cache instead of the newly created RTF_CACHE clone.
This patch updates the sk->sk_dst_cache for a connected datagram sk
during pmtu-update code path.
Note that the sk->sk_v6_daddr is used to do the route lookup
instead of skb->data (i.e. iph). It is because a UDP socket can become
connected after sending out some datagrams in un-connected state. or
It can be connected multiple times to different destinations. Hence,
iph may not be related to where sk is currently connected to.
It is done under '!sock_owned_by_user(sk)' condition because
the user may make another ip6_datagram_connect() (i.e changing
the sk->sk_v6_daddr) while dst lookup is happening in the pmtu-update
code path.
For the sock_owned_by_user(sk) == true case, the next patch will
introduce a release_cb() which will update the sk->sk_dst_cache.
Test:
Server (Connected UDP Socket):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Route Details:
[root@arch-fb-vm1 ~]# ip -6 r show | egrep '2fac'
2fac::/64 dev eth0 proto kernel metric 256 pref medium
2fac:face::/64 via 2fac::face dev eth0 metric 1024 pref medium
A simple python code to create a connected UDP socket:
import socket
import errno
HOST = '2fac::1'
PORT = 8080
s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
s.bind((HOST, PORT))
s.connect(('2fac:face::face', 53))
print("connected")
while True:
try:
data = s.recv(1024)
except socket.error as se:
if se.errno == errno.EMSGSIZE:
pmtu = s.getsockopt(41, 24)
print("PMTU:%d" % pmtu)
break
s.close()
Python program output after getting a ICMPV6_PKT_TOOBIG:
[root@arch-fb-vm1 ~]# python2 ~/devshare/kernel/tasks/fib6/udp-connect-53-8080.py
connected
PMTU:1300
Cache routes after recieving TOOBIG:
[root@arch-fb-vm1 ~]# ip -6 r show table cache
2fac:face::face via 2fac::face dev eth0 metric 0
cache expires 463sec mtu 1300 pref medium
Client (Send the ICMPV6_PKT_TOOBIG):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scapy is used to generate the TOOBIG message. Here is the scapy script I have
used:
>>> p=Ether(src='da:75:4d:36:ac:32', dst='52:54:00:12:34:66', type=0x86dd)/IPv6(src='2fac::face', dst='2fac::1')/ICMPv6PacketTooBig(mtu=1300)/IPv6(src='2fac::
1',dst='2fac:face::face', nh='UDP')/UDP(sport=8080,dport=53)
>>> sendp(p, iface='qemubr0')
Fixes: 45e4fd26683c ("ipv6: Only create RTF_CACHE routes after encountering pmtu exception")
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Reported-by: Wei Wang <weiwan@google.com>
Cc: Cong Wang <xiyou.wangcong@gmail.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-11 22:29:36 +00:00
|
|
|
struct dst_entry *dst;
|
|
|
|
|
2018-11-18 18:45:30 +00:00
|
|
|
if (!oif && skb->dev)
|
|
|
|
oif = l3mdev_master_ifindex(skb->dev);
|
|
|
|
|
|
|
|
ip6_update_pmtu(skb, sock_net(sk), mtu, oif, sk->sk_mark, sk->sk_uid);
|
ipv6: datagram: Update dst cache of a connected datagram sk during pmtu update
There is a case in connected UDP socket such that
getsockopt(IPV6_MTU) will return a stale MTU value. The reproducible
sequence could be the following:
1. Create a connected UDP socket
2. Send some datagrams out
3. Receive a ICMPV6_PKT_TOOBIG
4. No new outgoing datagrams to trigger the sk_dst_check()
logic to update the sk->sk_dst_cache.
5. getsockopt(IPV6_MTU) returns the mtu from the invalid
sk->sk_dst_cache instead of the newly created RTF_CACHE clone.
This patch updates the sk->sk_dst_cache for a connected datagram sk
during pmtu-update code path.
Note that the sk->sk_v6_daddr is used to do the route lookup
instead of skb->data (i.e. iph). It is because a UDP socket can become
connected after sending out some datagrams in un-connected state. or
It can be connected multiple times to different destinations. Hence,
iph may not be related to where sk is currently connected to.
It is done under '!sock_owned_by_user(sk)' condition because
the user may make another ip6_datagram_connect() (i.e changing
the sk->sk_v6_daddr) while dst lookup is happening in the pmtu-update
code path.
For the sock_owned_by_user(sk) == true case, the next patch will
introduce a release_cb() which will update the sk->sk_dst_cache.
Test:
Server (Connected UDP Socket):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Route Details:
[root@arch-fb-vm1 ~]# ip -6 r show | egrep '2fac'
2fac::/64 dev eth0 proto kernel metric 256 pref medium
2fac:face::/64 via 2fac::face dev eth0 metric 1024 pref medium
A simple python code to create a connected UDP socket:
import socket
import errno
HOST = '2fac::1'
PORT = 8080
s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
s.bind((HOST, PORT))
s.connect(('2fac:face::face', 53))
print("connected")
while True:
try:
data = s.recv(1024)
except socket.error as se:
if se.errno == errno.EMSGSIZE:
pmtu = s.getsockopt(41, 24)
print("PMTU:%d" % pmtu)
break
s.close()
Python program output after getting a ICMPV6_PKT_TOOBIG:
[root@arch-fb-vm1 ~]# python2 ~/devshare/kernel/tasks/fib6/udp-connect-53-8080.py
connected
PMTU:1300
Cache routes after recieving TOOBIG:
[root@arch-fb-vm1 ~]# ip -6 r show table cache
2fac:face::face via 2fac::face dev eth0 metric 0
cache expires 463sec mtu 1300 pref medium
Client (Send the ICMPV6_PKT_TOOBIG):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scapy is used to generate the TOOBIG message. Here is the scapy script I have
used:
>>> p=Ether(src='da:75:4d:36:ac:32', dst='52:54:00:12:34:66', type=0x86dd)/IPv6(src='2fac::face', dst='2fac::1')/ICMPv6PacketTooBig(mtu=1300)/IPv6(src='2fac::
1',dst='2fac:face::face', nh='UDP')/UDP(sport=8080,dport=53)
>>> sendp(p, iface='qemubr0')
Fixes: 45e4fd26683c ("ipv6: Only create RTF_CACHE routes after encountering pmtu exception")
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Reported-by: Wei Wang <weiwan@google.com>
Cc: Cong Wang <xiyou.wangcong@gmail.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-11 22:29:36 +00:00
|
|
|
|
|
|
|
dst = __sk_dst_get(sk);
|
|
|
|
if (!dst || !dst->obsolete ||
|
|
|
|
dst->ops->check(dst, inet6_sk(sk)->dst_cookie))
|
|
|
|
return;
|
|
|
|
|
|
|
|
bh_lock_sock(sk);
|
|
|
|
if (!sock_owned_by_user(sk) && !ipv6_addr_v4mapped(&sk->sk_v6_daddr))
|
|
|
|
ip6_datagram_dst_update(sk, false);
|
|
|
|
bh_unlock_sock(sk);
|
2012-06-15 21:54:11 +00:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(ip6_sk_update_pmtu);
|
|
|
|
|
2018-04-03 12:00:07 +00:00
|
|
|
void ip6_sk_dst_store_flow(struct sock *sk, struct dst_entry *dst,
|
|
|
|
const struct flowi6 *fl6)
|
|
|
|
{
|
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
|
|
|
struct ipv6_pinfo *np = inet6_sk(sk);
|
|
|
|
#endif
|
|
|
|
|
|
|
|
ip6_dst_store(sk, dst,
|
|
|
|
ipv6_addr_equal(&fl6->daddr, &sk->sk_v6_daddr) ?
|
|
|
|
&sk->sk_v6_daddr : NULL,
|
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
|
|
|
ipv6_addr_equal(&fl6->saddr, &np->saddr) ?
|
|
|
|
&np->saddr :
|
|
|
|
#endif
|
|
|
|
NULL);
|
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:02 +00:00
|
|
|
static bool ip6_redirect_nh_match(const struct fib6_result *res,
|
2019-04-09 21:41:19 +00:00
|
|
|
struct flowi6 *fl6,
|
|
|
|
const struct in6_addr *gw,
|
|
|
|
struct rt6_info **ret)
|
|
|
|
{
|
2019-04-16 21:36:02 +00:00
|
|
|
const struct fib6_nh *nh = res->nh;
|
|
|
|
|
2019-04-09 21:41:19 +00:00
|
|
|
if (nh->fib_nh_flags & RTNH_F_DEAD || !nh->fib_nh_gw_family ||
|
|
|
|
fl6->flowi6_oif != nh->fib_nh_dev->ifindex)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
/* rt_cache's gateway might be different from its 'parent'
|
|
|
|
* in the case of an ip redirect.
|
|
|
|
* So we keep searching in the exception table if the gateway
|
|
|
|
* is different.
|
|
|
|
*/
|
|
|
|
if (!ipv6_addr_equal(gw, &nh->fib_nh_gw6)) {
|
|
|
|
struct rt6_info *rt_cache;
|
|
|
|
|
2019-04-16 21:36:02 +00:00
|
|
|
rt_cache = rt6_find_cached_rt(res, &fl6->daddr, &fl6->saddr);
|
2019-04-09 21:41:19 +00:00
|
|
|
if (rt_cache &&
|
|
|
|
ipv6_addr_equal(gw, &rt_cache->rt6i_gateway)) {
|
|
|
|
*ret = rt_cache;
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
2019-06-08 21:53:29 +00:00
|
|
|
struct fib6_nh_rd_arg {
|
|
|
|
struct fib6_result *res;
|
|
|
|
struct flowi6 *fl6;
|
|
|
|
const struct in6_addr *gw;
|
|
|
|
struct rt6_info **ret;
|
|
|
|
};
|
|
|
|
|
|
|
|
static int fib6_nh_redirect_match(struct fib6_nh *nh, void *_arg)
|
|
|
|
{
|
|
|
|
struct fib6_nh_rd_arg *arg = _arg;
|
|
|
|
|
|
|
|
arg->res->nh = nh;
|
|
|
|
return ip6_redirect_nh_match(arg->res, arg->fl6, arg->gw, arg->ret);
|
|
|
|
}
|
|
|
|
|
2013-09-04 11:44:21 +00:00
|
|
|
/* Handle redirects */
|
|
|
|
struct ip6rd_flowi {
|
|
|
|
struct flowi6 fl6;
|
|
|
|
struct in6_addr gateway;
|
|
|
|
};
|
|
|
|
|
2020-06-23 16:42:32 +00:00
|
|
|
INDIRECT_CALLABLE_SCOPE struct rt6_info *__ip6_route_redirect(struct net *net,
|
2013-09-04 11:44:21 +00:00
|
|
|
struct fib6_table *table,
|
|
|
|
struct flowi6 *fl6,
|
2018-03-02 16:32:17 +00:00
|
|
|
const struct sk_buff *skb,
|
2013-09-04 11:44:21 +00:00
|
|
|
int flags)
|
|
|
|
{
|
|
|
|
struct ip6rd_flowi *rdfl = (struct ip6rd_flowi *)fl6;
|
2019-04-09 21:41:19 +00:00
|
|
|
struct rt6_info *ret = NULL;
|
2019-04-16 21:36:02 +00:00
|
|
|
struct fib6_result res = {};
|
2019-06-08 21:53:29 +00:00
|
|
|
struct fib6_nh_rd_arg arg = {
|
|
|
|
.res = &res,
|
|
|
|
.fl6 = fl6,
|
|
|
|
.gw = &rdfl->gateway,
|
|
|
|
.ret = &ret
|
|
|
|
};
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt;
|
2013-09-04 11:44:21 +00:00
|
|
|
struct fib6_node *fn;
|
|
|
|
|
2019-05-22 22:12:18 +00:00
|
|
|
/* l3mdev_update_flow overrides oif if the device is enslaved; in
|
|
|
|
* this case we must match on the real ingress device, so reset it
|
|
|
|
*/
|
|
|
|
if (fl6->flowi6_flags & FLOWI_FLAG_SKIP_NH_OIF)
|
|
|
|
fl6->flowi6_oif = skb->dev->ifindex;
|
|
|
|
|
2013-09-04 11:44:21 +00:00
|
|
|
/* Get the "current" route for this destination and
|
2017-01-07 22:53:00 +00:00
|
|
|
* check if the redirect has come from appropriate router.
|
2013-09-04 11:44:21 +00:00
|
|
|
*
|
|
|
|
* RFC 4861 specifies that redirects should only be
|
|
|
|
* accepted if they come from the nexthop to the target.
|
|
|
|
* Due to the way the routes are chosen, this notion
|
|
|
|
* is a bit fuzzy and one might need to check all possible
|
|
|
|
* routes.
|
|
|
|
*/
|
|
|
|
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_lock();
|
2018-05-10 03:34:19 +00:00
|
|
|
fn = fib6_node_lookup(&table->tb6_root, &fl6->daddr, &fl6->saddr);
|
2013-09-04 11:44:21 +00:00
|
|
|
restart:
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
for_each_fib6_node_rt_rcu(fn) {
|
2019-04-16 21:36:02 +00:00
|
|
|
res.f6i = rt;
|
2018-04-18 00:33:17 +00:00
|
|
|
if (fib6_check_expired(rt))
|
2013-09-04 11:44:21 +00:00
|
|
|
continue;
|
2018-04-18 22:38:59 +00:00
|
|
|
if (rt->fib6_flags & RTF_REJECT)
|
2013-09-04 11:44:21 +00:00
|
|
|
break;
|
2019-06-08 21:53:29 +00:00
|
|
|
if (unlikely(rt->nh)) {
|
|
|
|
if (nexthop_is_blackhole(rt->nh))
|
|
|
|
continue;
|
|
|
|
/* on match, res->nh is filled in and potentially ret */
|
|
|
|
if (nexthop_for_each_fib6_nh(rt->nh,
|
|
|
|
fib6_nh_redirect_match,
|
|
|
|
&arg))
|
|
|
|
goto out;
|
|
|
|
} else {
|
|
|
|
res.nh = rt->fib6_nh;
|
|
|
|
if (ip6_redirect_nh_match(&res, fl6, &rdfl->gateway,
|
|
|
|
&ret))
|
|
|
|
goto out;
|
|
|
|
}
|
2013-09-04 11:44:21 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if (!rt)
|
2018-04-18 00:33:18 +00:00
|
|
|
rt = net->ipv6.fib6_null_entry;
|
2018-04-18 22:38:59 +00:00
|
|
|
else if (rt->fib6_flags & RTF_REJECT) {
|
2018-04-18 00:33:23 +00:00
|
|
|
ret = net->ipv6.ip6_null_entry;
|
2015-01-21 03:16:02 +00:00
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:18 +00:00
|
|
|
if (rt == net->ipv6.fib6_null_entry) {
|
2014-10-20 20:42:43 +00:00
|
|
|
fn = fib6_backtrack(fn, &fl6->saddr);
|
|
|
|
if (fn)
|
|
|
|
goto restart;
|
2013-09-04 11:44:21 +00:00
|
|
|
}
|
2014-10-20 20:42:43 +00:00
|
|
|
|
2019-04-16 21:36:02 +00:00
|
|
|
res.f6i = rt;
|
2019-05-23 03:27:59 +00:00
|
|
|
res.nh = rt->fib6_nh;
|
2015-01-21 03:16:02 +00:00
|
|
|
out:
|
2019-04-16 21:36:11 +00:00
|
|
|
if (ret) {
|
2019-03-20 16:24:50 +00:00
|
|
|
ip6_hold_safe(net, &ret);
|
2019-04-16 21:36:11 +00:00
|
|
|
} else {
|
|
|
|
res.fib6_flags = res.f6i->fib6_flags;
|
|
|
|
res.fib6_type = res.f6i->fib6_type;
|
2019-04-16 21:36:02 +00:00
|
|
|
ret = ip6_create_rt_rcu(&res);
|
2019-04-16 21:36:11 +00:00
|
|
|
}
|
2013-09-04 11:44:21 +00:00
|
|
|
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_unlock();
|
2013-09-04 11:44:21 +00:00
|
|
|
|
2019-04-16 21:36:09 +00:00
|
|
|
trace_fib6_table_lookup(net, &res, table, fl6);
|
2018-04-18 00:33:23 +00:00
|
|
|
return ret;
|
2013-09-04 11:44:21 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
static struct dst_entry *ip6_route_redirect(struct net *net,
|
2018-03-02 16:32:17 +00:00
|
|
|
const struct flowi6 *fl6,
|
|
|
|
const struct sk_buff *skb,
|
|
|
|
const struct in6_addr *gateway)
|
2013-09-04 11:44:21 +00:00
|
|
|
{
|
|
|
|
int flags = RT6_LOOKUP_F_HAS_SADDR;
|
|
|
|
struct ip6rd_flowi rdfl;
|
|
|
|
|
|
|
|
rdfl.fl6 = *fl6;
|
|
|
|
rdfl.gateway = *gateway;
|
|
|
|
|
2018-03-02 16:32:17 +00:00
|
|
|
return fib6_rule_lookup(net, &rdfl.fl6, skb,
|
2013-09-04 11:44:21 +00:00
|
|
|
flags, __ip6_route_redirect);
|
|
|
|
}
|
|
|
|
|
2016-11-03 17:23:43 +00:00
|
|
|
void ip6_redirect(struct sk_buff *skb, struct net *net, int oif, u32 mark,
|
|
|
|
kuid_t uid)
|
2012-07-12 07:08:07 +00:00
|
|
|
{
|
|
|
|
const struct ipv6hdr *iph = (struct ipv6hdr *) skb->data;
|
|
|
|
struct dst_entry *dst;
|
2018-09-30 06:44:48 +00:00
|
|
|
struct flowi6 fl6 = {
|
|
|
|
.flowi6_iif = LOOPBACK_IFINDEX,
|
|
|
|
.flowi6_oif = oif,
|
|
|
|
.flowi6_mark = mark,
|
|
|
|
.daddr = iph->daddr,
|
|
|
|
.saddr = iph->saddr,
|
|
|
|
.flowlabel = ip6_flowinfo(iph),
|
|
|
|
.flowi6_uid = uid,
|
|
|
|
};
|
2012-07-12 07:08:07 +00:00
|
|
|
|
2018-03-02 16:32:17 +00:00
|
|
|
dst = ip6_route_redirect(net, &fl6, skb, &ipv6_hdr(skb)->saddr);
|
2013-09-04 11:44:21 +00:00
|
|
|
rt6_do_redirect(dst, NULL, skb);
|
2012-07-12 07:08:07 +00:00
|
|
|
dst_release(dst);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(ip6_redirect);
|
|
|
|
|
2018-09-30 06:44:50 +00:00
|
|
|
void ip6_redirect_no_header(struct sk_buff *skb, struct net *net, int oif)
|
2013-08-22 04:07:35 +00:00
|
|
|
{
|
|
|
|
const struct ipv6hdr *iph = ipv6_hdr(skb);
|
|
|
|
const struct rd_msg *msg = (struct rd_msg *)icmp6_hdr(skb);
|
|
|
|
struct dst_entry *dst;
|
2018-09-30 06:44:49 +00:00
|
|
|
struct flowi6 fl6 = {
|
|
|
|
.flowi6_iif = LOOPBACK_IFINDEX,
|
|
|
|
.flowi6_oif = oif,
|
|
|
|
.daddr = msg->dest,
|
|
|
|
.saddr = iph->daddr,
|
|
|
|
.flowi6_uid = sock_net_uid(net, NULL),
|
|
|
|
};
|
2013-08-22 04:07:35 +00:00
|
|
|
|
2018-03-02 16:32:17 +00:00
|
|
|
dst = ip6_route_redirect(net, &fl6, skb, &iph->saddr);
|
2013-09-04 11:44:21 +00:00
|
|
|
rt6_do_redirect(dst, NULL, skb);
|
2013-08-22 04:07:35 +00:00
|
|
|
dst_release(dst);
|
|
|
|
}
|
|
|
|
|
2012-07-12 07:08:07 +00:00
|
|
|
void ip6_sk_redirect(struct sk_buff *skb, struct sock *sk)
|
|
|
|
{
|
2016-11-03 17:23:43 +00:00
|
|
|
ip6_redirect(skb, sock_net(sk), sk->sk_bound_dev_if, sk->sk_mark,
|
|
|
|
sk->sk_uid);
|
2012-07-12 07:08:07 +00:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL_GPL(ip6_sk_redirect);
|
|
|
|
|
2010-12-13 20:52:14 +00:00
|
|
|
static unsigned int ip6_default_advmss(const struct dst_entry *dst)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2010-12-13 20:52:14 +00:00
|
|
|
struct net_device *dev = dst->dev;
|
|
|
|
unsigned int mtu = dst_mtu(dst);
|
|
|
|
struct net *net = dev_net(dev);
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
mtu -= sizeof(struct ipv6hdr) + sizeof(struct tcphdr);
|
|
|
|
|
2008-03-04 21:47:47 +00:00
|
|
|
if (mtu < net->ipv6.sysctl.ip6_rt_min_advmss)
|
|
|
|
mtu = net->ipv6.sysctl.ip6_rt_min_advmss;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
/*
|
2007-02-09 14:24:49 +00:00
|
|
|
* Maximal non-jumbo IPv6 payload is IPV6_MAXPLEN and
|
|
|
|
* corresponding MSS is IPV6_MAXPLEN - tcp_header_size.
|
|
|
|
* IPV6_MAXPLEN is also valid and means: "any MSS,
|
2005-04-16 22:20:36 +00:00
|
|
|
* rely only on pmtu discovery"
|
|
|
|
*/
|
|
|
|
if (mtu > IPV6_MAXPLEN - sizeof(struct tcphdr))
|
|
|
|
mtu = IPV6_MAXPLEN;
|
|
|
|
return mtu;
|
|
|
|
}
|
|
|
|
|
2021-02-01 17:41:31 +00:00
|
|
|
INDIRECT_CALLABLE_SCOPE unsigned int ip6_mtu(const struct dst_entry *dst)
|
2010-12-14 21:01:14 +00:00
|
|
|
{
|
2021-07-20 20:06:27 +00:00
|
|
|
return ip6_dst_mtu_maybe_forward(dst, false);
|
2010-12-14 21:01:14 +00:00
|
|
|
}
|
2021-02-04 18:18:39 +00:00
|
|
|
EXPORT_INDIRECT_CALLABLE(ip6_mtu);
|
2010-12-14 21:01:14 +00:00
|
|
|
|
2018-05-21 16:08:14 +00:00
|
|
|
/* MTU selection:
|
|
|
|
* 1. mtu on route is locked - use it
|
|
|
|
* 2. mtu from nexthop exception
|
|
|
|
* 3. mtu from egress device
|
|
|
|
*
|
|
|
|
* based on ip6_dst_mtu_forward and exception logic of
|
|
|
|
* rt6_find_cached_rt; called with rcu_read_lock
|
|
|
|
*/
|
2019-04-16 21:36:06 +00:00
|
|
|
u32 ip6_mtu_from_fib6(const struct fib6_result *res,
|
|
|
|
const struct in6_addr *daddr,
|
|
|
|
const struct in6_addr *saddr)
|
2018-05-21 16:08:14 +00:00
|
|
|
{
|
2019-04-16 21:36:06 +00:00
|
|
|
const struct fib6_nh *nh = res->nh;
|
|
|
|
struct fib6_info *f6i = res->f6i;
|
2018-05-21 16:08:14 +00:00
|
|
|
struct inet6_dev *idev;
|
2019-05-16 20:30:54 +00:00
|
|
|
struct rt6_info *rt;
|
2018-05-21 16:08:14 +00:00
|
|
|
u32 mtu = 0;
|
|
|
|
|
|
|
|
if (unlikely(fib6_metric_locked(f6i, RTAX_MTU))) {
|
|
|
|
mtu = f6i->fib6_pmtu;
|
|
|
|
if (mtu)
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
2019-05-16 20:30:54 +00:00
|
|
|
rt = rt6_find_cached_rt(res, daddr, saddr);
|
|
|
|
if (unlikely(rt)) {
|
|
|
|
mtu = dst_metric_raw(&rt->dst, RTAX_MTU);
|
|
|
|
} else {
|
2019-04-16 21:36:06 +00:00
|
|
|
struct net_device *dev = nh->fib_nh_dev;
|
2018-05-21 16:08:14 +00:00
|
|
|
|
|
|
|
mtu = IPV6_MIN_MTU;
|
|
|
|
idev = __in6_dev_get(dev);
|
|
|
|
if (idev && idev->cnf.mtu6 > mtu)
|
|
|
|
mtu = idev->cnf.mtu6;
|
|
|
|
}
|
|
|
|
|
|
|
|
mtu = min_t(unsigned int, mtu, IP6_MAX_MTU);
|
|
|
|
out:
|
2019-04-16 21:36:06 +00:00
|
|
|
return mtu - lwtunnel_headroom(nh->fib_nh_lws, mtu);
|
2018-05-21 16:08:14 +00:00
|
|
|
}
|
|
|
|
|
2007-12-07 00:11:48 +00:00
|
|
|
struct dst_entry *icmp6_dst_alloc(struct net_device *dev,
|
2011-12-06 22:04:13 +00:00
|
|
|
struct flowi6 *fl6)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2011-12-06 22:04:13 +00:00
|
|
|
struct dst_entry *dst;
|
2005-04-16 22:20:36 +00:00
|
|
|
struct rt6_info *rt;
|
|
|
|
struct inet6_dev *idev = in6_dev_get(dev);
|
2008-03-25 12:47:49 +00:00
|
|
|
struct net *net = dev_net(dev);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2011-12-03 23:02:47 +00:00
|
|
|
if (unlikely(!idev))
|
2012-03-14 21:13:11 +00:00
|
|
|
return ERR_PTR(-ENODEV);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2015-08-14 18:05:52 +00:00
|
|
|
rt = ip6_dst_alloc(net, dev, 0);
|
2011-12-03 23:02:47 +00:00
|
|
|
if (unlikely(!rt)) {
|
2005-04-16 22:20:36 +00:00
|
|
|
in6_dev_put(idev);
|
2011-12-06 22:04:13 +00:00
|
|
|
dst = ERR_PTR(-ENOMEM);
|
2005-04-16 22:20:36 +00:00
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
2017-12-13 11:14:57 +00:00
|
|
|
rt->dst.input = ip6_input;
|
2011-09-05 21:34:30 +00:00
|
|
|
rt->dst.output = ip6_output;
|
2013-10-20 12:43:04 +00:00
|
|
|
rt->rt6i_gateway = fl6->daddr;
|
2011-12-06 22:04:13 +00:00
|
|
|
rt->rt6i_dst.addr = fl6->daddr;
|
2011-09-05 21:34:30 +00:00
|
|
|
rt->rt6i_dst.plen = 128;
|
|
|
|
rt->rt6i_idev = idev;
|
2012-10-24 06:01:18 +00:00
|
|
|
dst_metric_set(&rt->dst, RTAX_HOPLIMIT, 0);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-01-07 10:45:04 +00:00
|
|
|
/* Add this dst into uncached_list so that rt6_disable_ip() can
|
2017-06-17 17:42:36 +00:00
|
|
|
* do proper release of the net_device
|
|
|
|
*/
|
|
|
|
rt6_uncached_list_add(rt);
|
2017-10-06 19:06:11 +00:00
|
|
|
atomic_inc(&net->ipv6.rt6_stats->fib_rt_uncache);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2011-12-06 22:04:13 +00:00
|
|
|
dst = xfrm_lookup(net, &rt->dst, flowi6_to_flowi(fl6), NULL, 0);
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
out:
|
2011-12-06 22:04:13 +00:00
|
|
|
return dst;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2008-01-18 11:56:57 +00:00
|
|
|
static int ip6_dst_gc(struct dst_ops *ops)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2009-08-29 01:34:49 +00:00
|
|
|
struct net *net = container_of(ops, struct net, ipv6.ip6_dst_ops);
|
2008-03-04 21:50:14 +00:00
|
|
|
int rt_min_interval = net->ipv6.sysctl.ip6_rt_gc_min_interval;
|
|
|
|
int rt_max_size = net->ipv6.sysctl.ip6_rt_max_size;
|
|
|
|
int rt_elasticity = net->ipv6.sysctl.ip6_rt_gc_elasticity;
|
|
|
|
int rt_gc_timeout = net->ipv6.sysctl.ip6_rt_gc_timeout;
|
|
|
|
unsigned long rt_last_gc = net->ipv6.ip6_rt_last_gc;
|
2010-10-08 06:37:34 +00:00
|
|
|
int entries;
|
2008-03-04 21:50:14 +00:00
|
|
|
|
2010-10-08 06:37:34 +00:00
|
|
|
entries = dst_entries_get_fast(ops);
|
2020-05-08 01:58:10 +00:00
|
|
|
if (entries > rt_max_size)
|
|
|
|
entries = dst_entries_get_slow(ops);
|
|
|
|
|
2013-08-01 08:04:24 +00:00
|
|
|
if (time_after(rt_last_gc + rt_min_interval, jiffies) &&
|
2010-10-08 06:37:34 +00:00
|
|
|
entries <= rt_max_size)
|
2005-04-16 22:20:36 +00:00
|
|
|
goto out;
|
|
|
|
|
2008-03-04 21:49:47 +00:00
|
|
|
net->ipv6.ip6_rt_gc_expire++;
|
2014-05-19 09:30:28 +00:00
|
|
|
fib6_run_gc(net->ipv6.ip6_rt_gc_expire, net, true);
|
2010-10-08 06:37:34 +00:00
|
|
|
entries = dst_entries_get_slow(ops);
|
|
|
|
if (entries < ops->gc_thresh)
|
2008-03-04 21:50:14 +00:00
|
|
|
net->ipv6.ip6_rt_gc_expire = rt_gc_timeout>>1;
|
2005-04-16 22:20:36 +00:00
|
|
|
out:
|
2008-03-04 21:50:14 +00:00
|
|
|
net->ipv6.ip6_rt_gc_expire -= net->ipv6.ip6_rt_gc_expire>>rt_elasticity;
|
2010-10-08 06:37:34 +00:00
|
|
|
return entries > rt_max_size;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
static int ip6_nh_lookup_table(struct net *net, struct fib6_config *cfg,
|
|
|
|
const struct in6_addr *gw_addr, u32 tbid,
|
|
|
|
int flags, struct fib6_result *res)
|
2016-04-25 04:26:04 +00:00
|
|
|
{
|
|
|
|
struct flowi6 fl6 = {
|
|
|
|
.flowi6_oif = cfg->fc_ifindex,
|
|
|
|
.daddr = *gw_addr,
|
|
|
|
.saddr = cfg->fc_prefsrc,
|
|
|
|
};
|
|
|
|
struct fib6_table *table;
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
int err;
|
2016-04-25 04:26:04 +00:00
|
|
|
|
2018-01-26 00:55:08 +00:00
|
|
|
table = fib6_get_table(net, tbid);
|
2016-04-25 04:26:04 +00:00
|
|
|
if (!table)
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
return -EINVAL;
|
2016-04-25 04:26:04 +00:00
|
|
|
|
|
|
|
if (!ipv6_addr_any(&cfg->fc_prefsrc))
|
|
|
|
flags |= RT6_LOOKUP_F_HAS_SADDR;
|
|
|
|
|
2018-01-26 00:55:08 +00:00
|
|
|
flags |= RT6_LOOKUP_F_IGNORE_LINKSTATE;
|
2016-04-25 04:26:04 +00:00
|
|
|
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
err = fib6_table_lookup(net, table, cfg->fc_ifindex, &fl6, res, flags);
|
|
|
|
if (!err && res->f6i != net->ipv6.fib6_null_entry)
|
|
|
|
fib6_select_path(net, res, &fl6, cfg->fc_ifindex,
|
|
|
|
cfg->fc_ifindex != 0, NULL, flags);
|
2016-04-25 04:26:04 +00:00
|
|
|
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
return err;
|
2016-04-25 04:26:04 +00:00
|
|
|
}
|
|
|
|
|
2018-01-26 00:55:09 +00:00
|
|
|
static int ip6_route_check_nh_onlink(struct net *net,
|
|
|
|
struct fib6_config *cfg,
|
2018-03-13 15:29:36 +00:00
|
|
|
const struct net_device *dev,
|
2018-01-26 00:55:09 +00:00
|
|
|
struct netlink_ext_ack *extack)
|
|
|
|
{
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
u32 tbid = l3mdev_fib_table_rcu(dev) ? : RT_TABLE_MAIN;
|
2018-01-26 00:55:09 +00:00
|
|
|
const struct in6_addr *gw_addr = &cfg->fc_gateway;
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
struct fib6_result res = {};
|
2018-01-26 00:55:09 +00:00
|
|
|
int err;
|
|
|
|
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
err = ip6_nh_lookup_table(net, cfg, gw_addr, tbid, 0, &res);
|
|
|
|
if (!err && !(res.fib6_flags & RTF_REJECT) &&
|
|
|
|
/* ignore match if it is the default route */
|
|
|
|
!ipv6_addr_any(&res.f6i->fib6_dst.addr) &&
|
|
|
|
(res.fib6_type != RTN_UNICAST || dev != res.nh->fib_nh_dev)) {
|
|
|
|
NL_SET_ERR_MSG(extack,
|
|
|
|
"Nexthop has invalid gateway or device mismatch");
|
|
|
|
err = -EINVAL;
|
2018-01-26 00:55:09 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2018-01-26 00:55:07 +00:00
|
|
|
static int ip6_route_check_nh(struct net *net,
|
|
|
|
struct fib6_config *cfg,
|
|
|
|
struct net_device **_dev,
|
|
|
|
struct inet6_dev **idev)
|
|
|
|
{
|
|
|
|
const struct in6_addr *gw_addr = &cfg->fc_gateway;
|
|
|
|
struct net_device *dev = _dev ? *_dev : NULL;
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
int flags = RT6_LOOKUP_F_IFACE;
|
|
|
|
struct fib6_result res = {};
|
2018-01-26 00:55:07 +00:00
|
|
|
int err = -EHOSTUNREACH;
|
|
|
|
|
|
|
|
if (cfg->fc_table) {
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
err = ip6_nh_lookup_table(net, cfg, gw_addr,
|
|
|
|
cfg->fc_table, flags, &res);
|
|
|
|
/* gw_addr can not require a gateway or resolve to a reject
|
|
|
|
* route. If a device is given, it must match the result.
|
|
|
|
*/
|
|
|
|
if (err || res.fib6_flags & RTF_REJECT ||
|
|
|
|
res.nh->fib_nh_gw_family ||
|
|
|
|
(dev && dev != res.nh->fib_nh_dev))
|
|
|
|
err = -EHOSTUNREACH;
|
2018-01-26 00:55:07 +00:00
|
|
|
}
|
|
|
|
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
if (err < 0) {
|
|
|
|
struct flowi6 fl6 = {
|
|
|
|
.flowi6_oif = cfg->fc_ifindex,
|
|
|
|
.daddr = *gw_addr,
|
|
|
|
};
|
2018-01-26 00:55:07 +00:00
|
|
|
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
err = fib6_lookup(net, cfg->fc_ifindex, &fl6, &res, flags);
|
|
|
|
if (err || res.fib6_flags & RTF_REJECT ||
|
|
|
|
res.nh->fib_nh_gw_family)
|
|
|
|
err = -EHOSTUNREACH;
|
|
|
|
|
|
|
|
if (err)
|
|
|
|
return err;
|
|
|
|
|
|
|
|
fib6_select_path(net, &res, &fl6, cfg->fc_ifindex,
|
|
|
|
cfg->fc_ifindex != 0, NULL, flags);
|
|
|
|
}
|
2018-01-26 00:55:07 +00:00
|
|
|
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
err = 0;
|
2018-01-26 00:55:07 +00:00
|
|
|
if (dev) {
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
if (dev != res.nh->fib_nh_dev)
|
|
|
|
err = -EHOSTUNREACH;
|
2018-01-26 00:55:07 +00:00
|
|
|
} else {
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
*_dev = dev = res.nh->fib_nh_dev;
|
2018-01-26 00:55:07 +00:00
|
|
|
dev_hold(dev);
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
*idev = in6_dev_get(dev);
|
2018-01-26 00:55:07 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2018-03-13 15:29:36 +00:00
|
|
|
static int ip6_validate_gw(struct net *net, struct fib6_config *cfg,
|
|
|
|
struct net_device **_dev, struct inet6_dev **idev,
|
|
|
|
struct netlink_ext_ack *extack)
|
|
|
|
{
|
|
|
|
const struct in6_addr *gw_addr = &cfg->fc_gateway;
|
|
|
|
int gwa_type = ipv6_addr_type(gw_addr);
|
2018-03-13 15:29:37 +00:00
|
|
|
bool skip_dev = gwa_type & IPV6_ADDR_LINKLOCAL ? false : true;
|
2018-03-13 15:29:36 +00:00
|
|
|
const struct net_device *dev = *_dev;
|
2018-03-13 15:29:37 +00:00
|
|
|
bool need_addr_check = !dev;
|
2018-03-13 15:29:36 +00:00
|
|
|
int err = -EINVAL;
|
|
|
|
|
|
|
|
/* if gw_addr is local we will fail to detect this in case
|
|
|
|
* address is still TENTATIVE (DAD in progress). rt6_lookup()
|
|
|
|
* will return already-added prefix route via interface that
|
|
|
|
* prefix route was assigned to, which might be non-loopback.
|
|
|
|
*/
|
2018-03-13 15:29:37 +00:00
|
|
|
if (dev &&
|
|
|
|
ipv6_chk_addr_and_flags(net, gw_addr, dev, skip_dev, 0, 0)) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Gateway can not be a local address");
|
2018-03-13 15:29:36 +00:00
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (gwa_type != (IPV6_ADDR_LINKLOCAL | IPV6_ADDR_UNICAST)) {
|
|
|
|
/* IPv6 strictly inhibits using not link-local
|
|
|
|
* addresses as nexthop address.
|
|
|
|
* Otherwise, router will not able to send redirects.
|
|
|
|
* It is very good, but in some (rare!) circumstances
|
|
|
|
* (SIT, PtP, NBMA NOARP links) it is handy to allow
|
|
|
|
* some exceptions. --ANK
|
|
|
|
* We allow IPv4-mapped nexthops to support RFC4798-type
|
|
|
|
* addressing
|
|
|
|
*/
|
|
|
|
if (!(gwa_type & (IPV6_ADDR_UNICAST | IPV6_ADDR_MAPPED))) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Invalid gateway address");
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
rcu_read_lock();
|
|
|
|
|
2018-03-13 15:29:36 +00:00
|
|
|
if (cfg->fc_flags & RTNH_F_ONLINK)
|
|
|
|
err = ip6_route_check_nh_onlink(net, cfg, dev, extack);
|
|
|
|
else
|
|
|
|
err = ip6_route_check_nh(net, cfg, _dev, idev);
|
|
|
|
|
ipv6: Convert gateway validation to use fib6_info
Gateway validation does not need a dst_entry, it only needs the fib
entry to validate the gateway resolution and egress device. So,
convert ip6_nh_lookup_table from ip6_pol_route to fib6_table_lookup
and ip6_route_check_nh to use fib6_lookup over rt6_lookup.
ip6_pol_route is a call to fib6_table_lookup and if successful a call
to fib6_select_path. From there the exception cache is searched for an
entry or a dst_entry is created to return to the caller. The exception
entry is not relevant for gateway validation, so what matters are the
calls to fib6_table_lookup and then fib6_select_path.
Similarly, rt6_lookup can be replaced with a call to fib6_lookup with
RT6_LOOKUP_F_IFACE set in flags. Again, the exception cache search is
not relevant, only the lookup with path selection. The primary difference
in the lookup paths is the use of rt6_select with fib6_lookup versus
rt6_device_match with rt6_lookup. When you remove complexities in the
rt6_select path, e.g.,
1. saddr is not set for gateway validation, so RT6_LOOKUP_F_HAS_SADDR
is not relevant
2. rt6_check_neigh is not called so that removes the RT6_NUD_FAIL_DO_RR
return and round-robin logic.
the code paths are believed to be equivalent for the given use case -
validate the gateway and optionally given the device. Furthermore, it
aligns the validation with onlink code path and the lookup path actually
used for rx and tx.
Adjust the users, ip6_route_check_nh_onlink and ip6_route_check_nh to
handle a fib6_info vs a rt6_info when performing validation checks.
Existing selftests fib-onlink-tests.sh and fib_tests.sh are used to
verify the changes.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Wei Wang <weiwan@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-24 20:44:51 +00:00
|
|
|
rcu_read_unlock();
|
|
|
|
|
2018-03-13 15:29:36 +00:00
|
|
|
if (err)
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* reload in case device was changed */
|
|
|
|
dev = *_dev;
|
|
|
|
|
|
|
|
err = -EINVAL;
|
|
|
|
if (!dev) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Egress device not specified");
|
|
|
|
goto out;
|
|
|
|
} else if (dev->flags & IFF_LOOPBACK) {
|
|
|
|
NL_SET_ERR_MSG(extack,
|
|
|
|
"Egress device can not be loopback device for this route");
|
|
|
|
goto out;
|
|
|
|
}
|
2018-03-13 15:29:37 +00:00
|
|
|
|
|
|
|
/* if we did not check gw_addr above, do so now that the
|
|
|
|
* egress device has been resolved.
|
|
|
|
*/
|
|
|
|
if (need_addr_check &&
|
|
|
|
ipv6_chk_addr_and_flags(net, gw_addr, dev, skip_dev, 0, 0)) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Gateway can not be a local address");
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
2018-03-13 15:29:36 +00:00
|
|
|
err = 0;
|
|
|
|
out:
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2019-03-28 03:53:50 +00:00
|
|
|
static bool fib6_is_reject(u32 flags, struct net_device *dev, int addr_type)
|
|
|
|
{
|
|
|
|
if ((flags & RTF_REJECT) ||
|
|
|
|
(dev && (dev->flags & IFF_LOOPBACK) &&
|
|
|
|
!(addr_type & IPV6_ADDR_LOOPBACK) &&
|
2020-07-07 13:39:24 +00:00
|
|
|
!(flags & (RTF_ANYCAST | RTF_LOCAL))))
|
2019-03-28 03:53:50 +00:00
|
|
|
return true;
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
int fib6_nh_init(struct net *net, struct fib6_nh *fib6_nh,
|
|
|
|
struct fib6_config *cfg, gfp_t gfp_flags,
|
|
|
|
struct netlink_ext_ack *extack)
|
|
|
|
{
|
|
|
|
struct net_device *dev = NULL;
|
|
|
|
struct inet6_dev *idev = NULL;
|
|
|
|
int addr_type;
|
|
|
|
int err;
|
|
|
|
|
2019-03-28 03:53:57 +00:00
|
|
|
fib6_nh->fib_nh_family = AF_INET6;
|
2019-11-07 17:26:19 +00:00
|
|
|
#ifdef CONFIG_IPV6_ROUTER_PREF
|
|
|
|
fib6_nh->last_probe = jiffies;
|
|
|
|
#endif
|
2020-05-22 05:26:13 +00:00
|
|
|
if (cfg->fc_is_fdb) {
|
|
|
|
fib6_nh->fib_nh_gw6 = cfg->fc_gateway;
|
|
|
|
fib6_nh->fib_nh_gw_family = AF_INET6;
|
|
|
|
return 0;
|
|
|
|
}
|
2019-03-28 03:53:57 +00:00
|
|
|
|
2019-03-28 03:53:50 +00:00
|
|
|
err = -ENODEV;
|
|
|
|
if (cfg->fc_ifindex) {
|
|
|
|
dev = dev_get_by_index(net, cfg->fc_ifindex);
|
|
|
|
if (!dev)
|
|
|
|
goto out;
|
|
|
|
idev = in6_dev_get(dev);
|
|
|
|
if (!idev)
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (cfg->fc_flags & RTNH_F_ONLINK) {
|
|
|
|
if (!dev) {
|
|
|
|
NL_SET_ERR_MSG(extack,
|
|
|
|
"Nexthop device required for onlink");
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (!(dev->flags & IFF_UP)) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Nexthop device is not up");
|
|
|
|
err = -ENETDOWN;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
2019-03-28 03:53:56 +00:00
|
|
|
fib6_nh->fib_nh_flags |= RTNH_F_ONLINK;
|
2019-03-28 03:53:50 +00:00
|
|
|
}
|
|
|
|
|
2019-03-28 03:53:56 +00:00
|
|
|
fib6_nh->fib_nh_weight = 1;
|
2019-03-28 03:53:50 +00:00
|
|
|
|
|
|
|
/* We cannot add true routes via loopback here,
|
|
|
|
* they would result in kernel looping; promote them to reject routes
|
|
|
|
*/
|
|
|
|
addr_type = ipv6_addr_type(&cfg->fc_dst);
|
|
|
|
if (fib6_is_reject(cfg->fc_flags, dev, addr_type)) {
|
|
|
|
/* hold loopback dev/idev if we haven't done so. */
|
|
|
|
if (dev != net->loopback_dev) {
|
|
|
|
if (dev) {
|
|
|
|
dev_put(dev);
|
|
|
|
in6_dev_put(idev);
|
|
|
|
}
|
|
|
|
dev = net->loopback_dev;
|
|
|
|
dev_hold(dev);
|
|
|
|
idev = in6_dev_get(dev);
|
|
|
|
if (!idev) {
|
|
|
|
err = -ENODEV;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
}
|
2019-06-04 01:37:03 +00:00
|
|
|
goto pcpu_alloc;
|
2019-03-28 03:53:50 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if (cfg->fc_flags & RTF_GATEWAY) {
|
|
|
|
err = ip6_validate_gw(net, cfg, &dev, &idev, extack);
|
|
|
|
if (err)
|
|
|
|
goto out;
|
|
|
|
|
2019-03-28 03:53:56 +00:00
|
|
|
fib6_nh->fib_nh_gw6 = cfg->fc_gateway;
|
2019-04-05 23:30:26 +00:00
|
|
|
fib6_nh->fib_nh_gw_family = AF_INET6;
|
2019-03-28 03:53:50 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
err = -ENODEV;
|
|
|
|
if (!dev)
|
|
|
|
goto out;
|
|
|
|
|
|
|
|
if (idev->cnf.disable_ipv6) {
|
|
|
|
NL_SET_ERR_MSG(extack, "IPv6 is disabled on nexthop device");
|
|
|
|
err = -EACCES;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (!(dev->flags & IFF_UP) && !cfg->fc_ignore_dev_down) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Nexthop device is not up");
|
|
|
|
err = -ENETDOWN;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (!(cfg->fc_flags & (RTF_LOCAL | RTF_ANYCAST)) &&
|
|
|
|
!netif_carrier_ok(dev))
|
2019-03-28 03:53:56 +00:00
|
|
|
fib6_nh->fib_nh_flags |= RTNH_F_LINKDOWN;
|
2019-03-28 03:53:50 +00:00
|
|
|
|
2020-03-27 22:00:21 +00:00
|
|
|
err = fib_nh_common_init(net, &fib6_nh->nh_common, cfg->fc_encap,
|
2019-06-04 01:37:03 +00:00
|
|
|
cfg->fc_encap_type, cfg, gfp_flags, extack);
|
|
|
|
if (err)
|
|
|
|
goto out;
|
|
|
|
|
|
|
|
pcpu_alloc:
|
2019-05-23 03:27:55 +00:00
|
|
|
fib6_nh->rt6i_pcpu = alloc_percpu_gfp(struct rt6_info *, gfp_flags);
|
|
|
|
if (!fib6_nh->rt6i_pcpu) {
|
|
|
|
err = -ENOMEM;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
2019-03-28 03:53:56 +00:00
|
|
|
fib6_nh->fib_nh_dev = dev;
|
2021-12-07 01:30:32 +00:00
|
|
|
netdev_tracker_alloc(dev, &fib6_nh->fib_nh_dev_tracker, gfp_flags);
|
|
|
|
|
2019-03-28 03:53:57 +00:00
|
|
|
fib6_nh->fib_nh_oif = dev->ifindex;
|
2019-03-28 03:53:50 +00:00
|
|
|
err = 0;
|
|
|
|
out:
|
|
|
|
if (idev)
|
|
|
|
in6_dev_put(idev);
|
|
|
|
|
|
|
|
if (err) {
|
2019-03-28 03:53:56 +00:00
|
|
|
lwtstate_put(fib6_nh->fib_nh_lws);
|
|
|
|
fib6_nh->fib_nh_lws = NULL;
|
2021-08-05 11:55:27 +00:00
|
|
|
dev_put(dev);
|
2019-03-28 03:53:50 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2019-03-28 03:53:51 +00:00
|
|
|
void fib6_nh_release(struct fib6_nh *fib6_nh)
|
|
|
|
{
|
2019-05-23 03:27:58 +00:00
|
|
|
struct rt6_exception_bucket *bucket;
|
|
|
|
|
|
|
|
rcu_read_lock();
|
|
|
|
|
|
|
|
fib6_nh_flush_exceptions(fib6_nh, NULL);
|
|
|
|
bucket = fib6_nh_get_excptn_bucket(fib6_nh, NULL);
|
|
|
|
if (bucket) {
|
|
|
|
rcu_assign_pointer(fib6_nh->rt6i_exception_bucket, NULL);
|
|
|
|
kfree(bucket);
|
|
|
|
}
|
|
|
|
|
|
|
|
rcu_read_unlock();
|
|
|
|
|
2021-11-29 15:44:11 +00:00
|
|
|
fib6_nh_release_dsts(fib6_nh);
|
|
|
|
free_percpu(fib6_nh->rt6i_pcpu);
|
2019-05-23 03:27:55 +00:00
|
|
|
|
2019-03-28 03:53:58 +00:00
|
|
|
fib_nh_common_release(&fib6_nh->nh_common);
|
2019-03-28 03:53:51 +00:00
|
|
|
}
|
|
|
|
|
2021-11-22 15:15:12 +00:00
|
|
|
void fib6_nh_release_dsts(struct fib6_nh *fib6_nh)
|
|
|
|
{
|
|
|
|
int cpu;
|
|
|
|
|
|
|
|
if (!fib6_nh->rt6i_pcpu)
|
|
|
|
return;
|
|
|
|
|
|
|
|
for_each_possible_cpu(cpu) {
|
|
|
|
struct rt6_info *pcpu_rt, **ppcpu_rt;
|
|
|
|
|
|
|
|
ppcpu_rt = per_cpu_ptr(fib6_nh->rt6i_pcpu, cpu);
|
|
|
|
pcpu_rt = xchg(ppcpu_rt, NULL);
|
|
|
|
if (pcpu_rt) {
|
|
|
|
dst_dev_put(&pcpu_rt->dst);
|
|
|
|
dst_release(&pcpu_rt->dst);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static struct fib6_info *ip6_route_info_create(struct fib6_config *cfg,
|
2018-04-18 00:33:22 +00:00
|
|
|
gfp_t gfp_flags,
|
2017-05-21 16:12:04 +00:00
|
|
|
struct netlink_ext_ack *extack)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2008-03-04 21:47:47 +00:00
|
|
|
struct net *net = cfg->fc_nlinfo.nl_net;
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt = NULL;
|
2019-06-04 03:19:52 +00:00
|
|
|
struct nexthop *nh = NULL;
|
2006-08-05 06:20:06 +00:00
|
|
|
struct fib6_table *table;
|
2019-06-04 03:19:52 +00:00
|
|
|
struct fib6_nh *fib6_nh;
|
2015-10-10 15:26:36 +00:00
|
|
|
int err = -EINVAL;
|
2019-03-28 03:53:50 +00:00
|
|
|
int addr_type;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2017-04-19 21:19:43 +00:00
|
|
|
/* RTF_PCPU is an internal flag; can not be set by userspace */
|
2017-05-21 16:12:05 +00:00
|
|
|
if (cfg->fc_flags & RTF_PCPU) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Userspace can not set RTF_PCPU");
|
2017-04-19 21:19:43 +00:00
|
|
|
goto out;
|
2017-05-21 16:12:05 +00:00
|
|
|
}
|
2017-04-19 21:19:43 +00:00
|
|
|
|
2017-10-28 00:30:12 +00:00
|
|
|
/* RTF_CACHE is an internal flag; can not be set by userspace */
|
|
|
|
if (cfg->fc_flags & RTF_CACHE) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Userspace can not set RTF_CACHE");
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:13 +00:00
|
|
|
if (cfg->fc_type > RTN_MAX) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Invalid route type");
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
2017-05-21 16:12:05 +00:00
|
|
|
if (cfg->fc_dst_len > 128) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Invalid prefix length");
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
if (cfg->fc_src_len > 128) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Invalid source address length");
|
2015-10-10 15:26:36 +00:00
|
|
|
goto out;
|
2017-05-21 16:12:05 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
#ifndef CONFIG_IPV6_SUBTREES
|
2017-05-21 16:12:05 +00:00
|
|
|
if (cfg->fc_src_len) {
|
|
|
|
NL_SET_ERR_MSG(extack,
|
|
|
|
"Specifying source address requires IPV6_SUBTREES to be enabled");
|
2015-10-10 15:26:36 +00:00
|
|
|
goto out;
|
2017-05-21 16:12:05 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
#endif
|
2019-06-08 21:53:34 +00:00
|
|
|
if (cfg->fc_nh_id) {
|
|
|
|
nh = nexthop_find_by_id(net, cfg->fc_nh_id);
|
|
|
|
if (!nh) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Nexthop id does not exist");
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
err = fib6_check_nexthop(nh, cfg, extack);
|
|
|
|
if (err)
|
|
|
|
goto out;
|
|
|
|
}
|
2018-01-26 00:55:09 +00:00
|
|
|
|
2011-11-14 00:14:49 +00:00
|
|
|
err = -ENOBUFS;
|
2011-12-03 23:02:47 +00:00
|
|
|
if (cfg->fc_nlinfo.nlh &&
|
|
|
|
!(cfg->fc_nlinfo.nlh->nlmsg_flags & NLM_F_CREATE)) {
|
2011-11-14 00:14:49 +00:00
|
|
|
table = fib6_get_table(net, cfg->fc_table);
|
2011-12-03 23:02:47 +00:00
|
|
|
if (!table) {
|
2012-05-15 14:11:53 +00:00
|
|
|
pr_warn("NLM_F_CREATE should be specified when creating new route\n");
|
2011-11-14 00:14:49 +00:00
|
|
|
table = fib6_new_table(net, cfg->fc_table);
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
table = fib6_new_table(net, cfg->fc_table);
|
|
|
|
}
|
2011-12-03 23:02:47 +00:00
|
|
|
|
|
|
|
if (!table)
|
2006-08-05 06:20:06 +00:00
|
|
|
goto out;
|
|
|
|
|
2018-04-18 00:33:25 +00:00
|
|
|
err = -ENOMEM;
|
2019-06-04 03:19:52 +00:00
|
|
|
rt = fib6_info_alloc(gfp_flags, !nh);
|
2018-04-18 00:33:25 +00:00
|
|
|
if (!rt)
|
2005-04-16 22:20:36 +00:00
|
|
|
goto out;
|
2018-04-18 00:33:25 +00:00
|
|
|
|
2018-11-06 20:51:15 +00:00
|
|
|
rt->fib6_metrics = ip_fib_metrics_init(net, cfg->fc_mx, cfg->fc_mx_len,
|
|
|
|
extack);
|
2018-10-05 03:07:51 +00:00
|
|
|
if (IS_ERR(rt->fib6_metrics)) {
|
|
|
|
err = PTR_ERR(rt->fib6_metrics);
|
2018-10-05 16:17:50 +00:00
|
|
|
/* Do not leave garbage there. */
|
|
|
|
rt->fib6_metrics = (struct dst_metrics *)&dst_default_metrics;
|
2021-07-20 13:08:40 +00:00
|
|
|
goto out_free;
|
2018-10-05 03:07:51 +00:00
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:25 +00:00
|
|
|
if (cfg->fc_flags & RTF_ADDRCONF)
|
|
|
|
rt->dst_nocount = true;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2012-04-06 00:13:10 +00:00
|
|
|
if (cfg->fc_flags & RTF_EXPIRES)
|
2018-04-18 00:33:17 +00:00
|
|
|
fib6_set_expires(rt, jiffies +
|
2012-04-06 00:13:10 +00:00
|
|
|
clock_t_to_jiffies(cfg->fc_expires));
|
|
|
|
else
|
2018-04-18 00:33:17 +00:00
|
|
|
fib6_clean_expires(rt);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2006-08-22 07:01:08 +00:00
|
|
|
if (cfg->fc_protocol == RTPROT_UNSPEC)
|
|
|
|
cfg->fc_protocol = RTPROT_BOOT;
|
2018-04-18 22:38:59 +00:00
|
|
|
rt->fib6_protocol = cfg->fc_protocol;
|
2006-08-22 07:01:08 +00:00
|
|
|
|
2019-03-28 03:53:50 +00:00
|
|
|
rt->fib6_table = table;
|
|
|
|
rt->fib6_metric = cfg->fc_metric;
|
2019-06-19 17:50:24 +00:00
|
|
|
rt->fib6_type = cfg->fc_type ? : RTN_UNICAST;
|
2019-03-28 03:53:52 +00:00
|
|
|
rt->fib6_flags = cfg->fc_flags & ~RTF_GATEWAY;
|
2015-07-21 08:43:48 +00:00
|
|
|
|
2018-04-18 22:38:59 +00:00
|
|
|
ipv6_addr_prefix(&rt->fib6_dst.addr, &cfg->fc_dst, cfg->fc_dst_len);
|
|
|
|
rt->fib6_dst.plen = cfg->fc_dst_len;
|
2014-03-27 12:04:08 +00:00
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
2018-04-18 22:38:59 +00:00
|
|
|
ipv6_addr_prefix(&rt->fib6_src.addr, &cfg->fc_src, cfg->fc_src_len);
|
|
|
|
rt->fib6_src.plen = cfg->fc_src_len;
|
2005-04-16 22:20:36 +00:00
|
|
|
#endif
|
2019-06-04 03:19:52 +00:00
|
|
|
if (nh) {
|
|
|
|
if (rt->fib6_src.plen) {
|
2019-06-06 08:40:39 +00:00
|
|
|
NL_SET_ERR_MSG(extack, "Nexthops can not be used with source routing");
|
2021-06-03 07:32:58 +00:00
|
|
|
goto out_free;
|
2019-06-04 03:19:52 +00:00
|
|
|
}
|
2020-07-25 08:02:18 +00:00
|
|
|
if (!nexthop_get(nh)) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Nexthop has been deleted");
|
2021-06-03 07:32:58 +00:00
|
|
|
goto out_free;
|
2020-07-25 08:02:18 +00:00
|
|
|
}
|
2019-06-04 03:19:52 +00:00
|
|
|
rt->nh = nh;
|
|
|
|
fib6_nh = nexthop_fib6_nh(rt->nh);
|
|
|
|
} else {
|
|
|
|
err = fib6_nh_init(net, rt->fib6_nh, cfg, gfp_flags, extack);
|
|
|
|
if (err)
|
|
|
|
goto out;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
fib6_nh = rt->fib6_nh;
|
|
|
|
|
|
|
|
/* We cannot add true routes via loopback here, they would
|
|
|
|
* result in kernel looping; promote them to reject routes
|
|
|
|
*/
|
|
|
|
addr_type = ipv6_addr_type(&cfg->fc_dst);
|
|
|
|
if (fib6_is_reject(cfg->fc_flags, rt->fib6_nh->fib_nh_dev,
|
|
|
|
addr_type))
|
|
|
|
rt->fib6_flags = RTF_REJECT | RTF_NONEXTHOP;
|
|
|
|
}
|
2018-01-25 03:45:29 +00:00
|
|
|
|
2011-04-13 21:10:57 +00:00
|
|
|
if (!ipv6_addr_any(&cfg->fc_prefsrc)) {
|
2019-06-04 03:19:52 +00:00
|
|
|
struct net_device *dev = fib6_nh->fib_nh_dev;
|
2019-03-28 03:53:50 +00:00
|
|
|
|
2011-04-13 21:10:57 +00:00
|
|
|
if (!ipv6_chk_addr(net, &cfg->fc_prefsrc, dev, 0)) {
|
2017-05-21 16:12:05 +00:00
|
|
|
NL_SET_ERR_MSG(extack, "Invalid source address");
|
2011-04-13 21:10:57 +00:00
|
|
|
err = -EINVAL;
|
|
|
|
goto out;
|
|
|
|
}
|
2018-04-18 22:38:59 +00:00
|
|
|
rt->fib6_prefsrc.addr = cfg->fc_prefsrc;
|
|
|
|
rt->fib6_prefsrc.plen = 128;
|
2011-04-13 21:10:57 +00:00
|
|
|
} else
|
2018-04-18 22:38:59 +00:00
|
|
|
rt->fib6_prefsrc.plen = 0;
|
2011-04-13 21:10:57 +00:00
|
|
|
|
2015-10-10 15:26:36 +00:00
|
|
|
return rt;
|
2015-09-08 17:53:04 +00:00
|
|
|
out:
|
2018-04-18 00:33:25 +00:00
|
|
|
fib6_info_release(rt);
|
2015-10-10 15:26:36 +00:00
|
|
|
return ERR_PTR(err);
|
2021-06-03 07:32:58 +00:00
|
|
|
out_free:
|
|
|
|
ip_fib_metrics_put(rt->fib6_metrics);
|
|
|
|
kfree(rt);
|
|
|
|
return ERR_PTR(err);
|
2015-09-08 17:53:04 +00:00
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:22 +00:00
|
|
|
int ip6_route_add(struct fib6_config *cfg, gfp_t gfp_flags,
|
2017-05-21 16:12:04 +00:00
|
|
|
struct netlink_ext_ack *extack)
|
2015-09-08 17:53:04 +00:00
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt;
|
2015-09-08 17:53:04 +00:00
|
|
|
int err;
|
|
|
|
|
2018-04-18 00:33:22 +00:00
|
|
|
rt = ip6_route_info_create(cfg, gfp_flags, extack);
|
2018-04-18 00:33:16 +00:00
|
|
|
if (IS_ERR(rt))
|
|
|
|
return PTR_ERR(rt);
|
2015-09-08 17:53:04 +00:00
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
err = __ip6_ins_rt(rt, &cfg->fc_nlinfo, extack);
|
2018-04-18 00:33:25 +00:00
|
|
|
fib6_info_release(rt);
|
2015-09-08 17:53:04 +00:00
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static int __ip6_del_rt(struct fib6_info *rt, struct nl_info *info)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2018-04-18 00:33:11 +00:00
|
|
|
struct net *net = info->nl_net;
|
2006-08-05 06:20:06 +00:00
|
|
|
struct fib6_table *table;
|
2018-04-18 00:33:11 +00:00
|
|
|
int err;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-04-18 00:33:18 +00:00
|
|
|
if (rt == net->ipv6.fib6_null_entry) {
|
2012-09-19 19:25:34 +00:00
|
|
|
err = -ENOENT;
|
|
|
|
goto out;
|
|
|
|
}
|
2006-08-07 05:22:47 +00:00
|
|
|
|
2018-04-18 22:38:59 +00:00
|
|
|
table = rt->fib6_table;
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
spin_lock_bh(&table->tb6_lock);
|
2006-08-22 07:01:08 +00:00
|
|
|
err = fib6_del(rt, info);
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
spin_unlock_bh(&table->tb6_lock);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2012-09-19 19:25:34 +00:00
|
|
|
out:
|
2018-04-18 00:33:25 +00:00
|
|
|
fib6_info_release(rt);
|
2005-04-16 22:20:36 +00:00
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2020-04-27 20:56:45 +00:00
|
|
|
int ip6_del_rt(struct net *net, struct fib6_info *rt, bool skip_notify)
|
2006-08-22 07:00:21 +00:00
|
|
|
{
|
2020-04-27 20:56:45 +00:00
|
|
|
struct nl_info info = {
|
|
|
|
.nl_net = net,
|
|
|
|
.skip_notify = skip_notify
|
|
|
|
};
|
2018-04-18 00:33:11 +00:00
|
|
|
|
2007-12-13 17:45:12 +00:00
|
|
|
return __ip6_del_rt(rt, &info);
|
2006-08-22 07:00:21 +00:00
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static int __ip6_del_rt_siblings(struct fib6_info *rt, struct fib6_config *cfg)
|
2017-02-02 20:37:08 +00:00
|
|
|
{
|
|
|
|
struct nl_info *info = &cfg->fc_nlinfo;
|
2017-02-28 00:07:43 +00:00
|
|
|
struct net *net = info->nl_net;
|
2017-02-02 20:37:11 +00:00
|
|
|
struct sk_buff *skb = NULL;
|
2017-02-02 20:37:08 +00:00
|
|
|
struct fib6_table *table;
|
2017-02-28 00:07:43 +00:00
|
|
|
int err = -ENOENT;
|
2017-02-02 20:37:08 +00:00
|
|
|
|
2018-04-18 00:33:18 +00:00
|
|
|
if (rt == net->ipv6.fib6_null_entry)
|
2017-02-28 00:07:43 +00:00
|
|
|
goto out_put;
|
2018-04-18 22:38:59 +00:00
|
|
|
table = rt->fib6_table;
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
spin_lock_bh(&table->tb6_lock);
|
2017-02-02 20:37:08 +00:00
|
|
|
|
2018-04-18 22:38:59 +00:00
|
|
|
if (rt->fib6_nsiblings && cfg->fc_delete_all_nh) {
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *sibling, *next_sibling;
|
2019-12-23 13:28:18 +00:00
|
|
|
struct fib6_node *fn;
|
2017-02-02 20:37:08 +00:00
|
|
|
|
2017-02-02 20:37:11 +00:00
|
|
|
/* prefer to send a single notification with all hops */
|
|
|
|
skb = nlmsg_new(rt6_nlmsg_size(rt), gfp_any());
|
|
|
|
if (skb) {
|
|
|
|
u32 seq = info->nlh ? info->nlh->nlmsg_seq : 0;
|
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
if (rt6_fill_node(net, skb, rt, NULL,
|
2017-02-02 20:37:11 +00:00
|
|
|
NULL, NULL, 0, RTM_DELROUTE,
|
|
|
|
info->portid, seq, 0) < 0) {
|
|
|
|
kfree_skb(skb);
|
|
|
|
skb = NULL;
|
|
|
|
} else
|
|
|
|
info->skip_notify = 1;
|
|
|
|
}
|
|
|
|
|
2019-12-23 13:28:18 +00:00
|
|
|
/* 'rt' points to the first sibling route. If it is not the
|
|
|
|
* leaf, then we do not need to send a notification. Otherwise,
|
|
|
|
* we need to check if the last sibling has a next route or not
|
|
|
|
* and emit a replace or delete notification, respectively.
|
|
|
|
*/
|
2019-06-18 15:12:49 +00:00
|
|
|
info->skip_notify_kernel = 1;
|
2019-12-23 13:28:18 +00:00
|
|
|
fn = rcu_dereference_protected(rt->fib6_node,
|
|
|
|
lockdep_is_held(&table->tb6_lock));
|
|
|
|
if (rcu_access_pointer(fn->leaf) == rt) {
|
|
|
|
struct fib6_info *last_sibling, *replace_rt;
|
|
|
|
|
|
|
|
last_sibling = list_last_entry(&rt->fib6_siblings,
|
|
|
|
struct fib6_info,
|
|
|
|
fib6_siblings);
|
|
|
|
replace_rt = rcu_dereference_protected(
|
|
|
|
last_sibling->fib6_next,
|
|
|
|
lockdep_is_held(&table->tb6_lock));
|
|
|
|
if (replace_rt)
|
|
|
|
call_fib6_entry_notifiers_replace(net,
|
|
|
|
replace_rt);
|
|
|
|
else
|
|
|
|
call_fib6_multipath_entry_notifiers(net,
|
2019-12-23 13:28:20 +00:00
|
|
|
FIB_EVENT_ENTRY_DEL,
|
2019-12-23 13:28:18 +00:00
|
|
|
rt, rt->fib6_nsiblings,
|
|
|
|
NULL);
|
|
|
|
}
|
2017-02-02 20:37:08 +00:00
|
|
|
list_for_each_entry_safe(sibling, next_sibling,
|
2018-04-18 22:38:59 +00:00
|
|
|
&rt->fib6_siblings,
|
|
|
|
fib6_siblings) {
|
2017-02-02 20:37:08 +00:00
|
|
|
err = fib6_del(sibling, info);
|
|
|
|
if (err)
|
2017-02-28 00:07:43 +00:00
|
|
|
goto out_unlock;
|
2017-02-02 20:37:08 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
err = fib6_del(rt, info);
|
2017-02-28 00:07:43 +00:00
|
|
|
out_unlock:
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
spin_unlock_bh(&table->tb6_lock);
|
2017-02-28 00:07:43 +00:00
|
|
|
out_put:
|
2018-04-18 00:33:25 +00:00
|
|
|
fib6_info_release(rt);
|
2017-02-02 20:37:11 +00:00
|
|
|
|
|
|
|
if (skb) {
|
2017-02-28 00:07:43 +00:00
|
|
|
rtnl_notify(skb, net, info->portid, RTNLGRP_IPV6_ROUTE,
|
2017-02-02 20:37:11 +00:00
|
|
|
info->nlh, gfp_any());
|
|
|
|
}
|
2017-02-02 20:37:08 +00:00
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:28:00 +00:00
|
|
|
static int __ip6_del_cached_rt(struct rt6_info *rt, struct fib6_config *cfg)
|
2018-04-18 00:33:23 +00:00
|
|
|
{
|
|
|
|
int rc = -ESRCH;
|
|
|
|
|
|
|
|
if (cfg->fc_ifindex && rt->dst.dev->ifindex != cfg->fc_ifindex)
|
|
|
|
goto out;
|
|
|
|
|
|
|
|
if (cfg->fc_flags & RTF_GATEWAY &&
|
|
|
|
!ipv6_addr_equal(&cfg->fc_gateway, &rt->rt6i_gateway))
|
|
|
|
goto out;
|
2018-11-13 16:48:28 +00:00
|
|
|
|
|
|
|
rc = rt6_remove_exception_rt(rt);
|
2018-04-18 00:33:23 +00:00
|
|
|
out:
|
|
|
|
return rc;
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:28:00 +00:00
|
|
|
static int ip6_del_cached_rt(struct fib6_config *cfg, struct fib6_info *rt,
|
|
|
|
struct fib6_nh *nh)
|
|
|
|
{
|
|
|
|
struct fib6_result res = {
|
|
|
|
.f6i = rt,
|
|
|
|
.nh = nh,
|
|
|
|
};
|
|
|
|
struct rt6_info *rt_cache;
|
|
|
|
|
|
|
|
rt_cache = rt6_find_cached_rt(&res, &cfg->fc_dst, &cfg->fc_src);
|
|
|
|
if (rt_cache)
|
|
|
|
return __ip6_del_cached_rt(rt_cache, cfg);
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2019-06-08 21:53:34 +00:00
|
|
|
struct fib6_nh_del_cached_rt_arg {
|
|
|
|
struct fib6_config *cfg;
|
|
|
|
struct fib6_info *f6i;
|
|
|
|
};
|
|
|
|
|
|
|
|
static int fib6_nh_del_cached_rt(struct fib6_nh *nh, void *_arg)
|
|
|
|
{
|
|
|
|
struct fib6_nh_del_cached_rt_arg *arg = _arg;
|
|
|
|
int rc;
|
|
|
|
|
|
|
|
rc = ip6_del_cached_rt(arg->cfg, arg->f6i, nh);
|
|
|
|
return rc != -ESRCH ? rc : 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int ip6_del_cached_rt_nh(struct fib6_config *cfg, struct fib6_info *f6i)
|
|
|
|
{
|
|
|
|
struct fib6_nh_del_cached_rt_arg arg = {
|
|
|
|
.cfg = cfg,
|
|
|
|
.f6i = f6i
|
|
|
|
};
|
|
|
|
|
|
|
|
return nexthop_for_each_fib6_nh(f6i->nh, fib6_nh_del_cached_rt, &arg);
|
|
|
|
}
|
|
|
|
|
2017-05-21 16:12:04 +00:00
|
|
|
static int ip6_route_del(struct fib6_config *cfg,
|
|
|
|
struct netlink_ext_ack *extack)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2006-08-05 06:20:06 +00:00
|
|
|
struct fib6_table *table;
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt;
|
2005-04-16 22:20:36 +00:00
|
|
|
struct fib6_node *fn;
|
|
|
|
int err = -ESRCH;
|
|
|
|
|
2008-03-04 21:47:47 +00:00
|
|
|
table = fib6_get_table(cfg->fc_nlinfo.nl_net, cfg->fc_table);
|
2017-05-21 16:12:05 +00:00
|
|
|
if (!table) {
|
|
|
|
NL_SET_ERR_MSG(extack, "FIB table does not exist");
|
2006-08-05 06:20:06 +00:00
|
|
|
return err;
|
2017-05-21 16:12:05 +00:00
|
|
|
}
|
2006-08-05 06:20:06 +00:00
|
|
|
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_lock();
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2006-08-05 06:20:06 +00:00
|
|
|
fn = fib6_locate(&table->tb6_root,
|
2006-08-22 07:01:08 +00:00
|
|
|
&cfg->fc_dst, cfg->fc_dst_len,
|
2017-10-06 19:06:02 +00:00
|
|
|
&cfg->fc_src, cfg->fc_src_len,
|
2017-10-06 19:06:03 +00:00
|
|
|
!(cfg->fc_flags & RTF_CACHE));
|
2007-02-09 14:24:49 +00:00
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
if (fn) {
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
for_each_fib6_node_rt_rcu(fn) {
|
2019-03-28 03:53:56 +00:00
|
|
|
struct fib6_nh *nh;
|
|
|
|
|
2019-06-21 15:45:25 +00:00
|
|
|
if (rt->nh && cfg->fc_nh_id &&
|
|
|
|
rt->nh->id != cfg->fc_nh_id)
|
2019-06-08 21:53:34 +00:00
|
|
|
continue;
|
2018-04-18 00:33:23 +00:00
|
|
|
|
2019-06-08 21:53:34 +00:00
|
|
|
if (cfg->fc_flags & RTF_CACHE) {
|
|
|
|
int rc = 0;
|
|
|
|
|
|
|
|
if (rt->nh) {
|
|
|
|
rc = ip6_del_cached_rt_nh(cfg, rt);
|
|
|
|
} else if (cfg->fc_nh_id) {
|
|
|
|
continue;
|
|
|
|
} else {
|
|
|
|
nh = rt->fib6_nh;
|
|
|
|
rc = ip6_del_cached_rt(cfg, rt, nh);
|
|
|
|
}
|
2019-05-23 03:28:00 +00:00
|
|
|
if (rc != -ESRCH) {
|
|
|
|
rcu_read_unlock();
|
|
|
|
return rc;
|
2018-04-18 00:33:23 +00:00
|
|
|
}
|
|
|
|
continue;
|
2017-10-06 19:06:03 +00:00
|
|
|
}
|
2019-03-28 03:53:56 +00:00
|
|
|
|
2019-06-08 21:53:34 +00:00
|
|
|
if (cfg->fc_metric && cfg->fc_metric != rt->fib6_metric)
|
|
|
|
continue;
|
|
|
|
if (cfg->fc_protocol &&
|
|
|
|
cfg->fc_protocol != rt->fib6_protocol)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
if (rt->nh) {
|
|
|
|
if (!fib6_info_hold_safe(rt))
|
|
|
|
continue;
|
|
|
|
rcu_read_unlock();
|
|
|
|
|
|
|
|
return __ip6_del_rt(rt, &cfg->fc_nlinfo);
|
|
|
|
}
|
|
|
|
if (cfg->fc_nh_id)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
nh = rt->fib6_nh;
|
2006-08-22 07:01:08 +00:00
|
|
|
if (cfg->fc_ifindex &&
|
2019-03-28 03:53:56 +00:00
|
|
|
(!nh->fib_nh_dev ||
|
|
|
|
nh->fib_nh_dev->ifindex != cfg->fc_ifindex))
|
2005-04-16 22:20:36 +00:00
|
|
|
continue;
|
2006-08-22 07:01:08 +00:00
|
|
|
if (cfg->fc_flags & RTF_GATEWAY &&
|
2019-03-28 03:53:56 +00:00
|
|
|
!ipv6_addr_equal(&cfg->fc_gateway, &nh->fib_nh_gw6))
|
2005-04-16 22:20:36 +00:00
|
|
|
continue;
|
2018-07-22 03:56:32 +00:00
|
|
|
if (!fib6_info_hold_safe(rt))
|
|
|
|
continue;
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_unlock();
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2017-02-02 20:37:08 +00:00
|
|
|
/* if gateway was specified only delete the one hop */
|
|
|
|
if (cfg->fc_flags & RTF_GATEWAY)
|
|
|
|
return __ip6_del_rt(rt, &cfg->fc_nlinfo);
|
|
|
|
|
|
|
|
return __ip6_del_rt_siblings(rt, cfg);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
}
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_unlock();
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2012-07-17 10:29:28 +00:00
|
|
|
static void rt6_do_redirect(struct dst_entry *dst, struct sock *sk, struct sk_buff *skb)
|
2006-08-24 00:18:26 +00:00
|
|
|
{
|
|
|
|
struct netevent_redirect netevent;
|
2012-07-12 06:43:53 +00:00
|
|
|
struct rt6_info *rt, *nrt = NULL;
|
2019-04-16 21:36:01 +00:00
|
|
|
struct fib6_result res = {};
|
2012-07-12 06:43:53 +00:00
|
|
|
struct ndisc_options ndopts;
|
|
|
|
struct inet6_dev *in6_dev;
|
|
|
|
struct neighbour *neigh;
|
2013-01-05 16:34:51 +00:00
|
|
|
struct rd_msg *msg;
|
2012-07-12 07:05:02 +00:00
|
|
|
int optlen, on_link;
|
|
|
|
u8 *lladdr;
|
2012-07-12 06:43:53 +00:00
|
|
|
|
2013-05-28 20:34:26 +00:00
|
|
|
optlen = skb_tail_pointer(skb) - skb_transport_header(skb);
|
2013-01-05 16:34:51 +00:00
|
|
|
optlen -= sizeof(*msg);
|
2012-07-12 06:43:53 +00:00
|
|
|
|
|
|
|
if (optlen < 0) {
|
2012-07-12 07:05:02 +00:00
|
|
|
net_dbg_ratelimited("rt6_do_redirect: packet too short\n");
|
2012-07-12 06:43:53 +00:00
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2013-01-05 16:34:51 +00:00
|
|
|
msg = (struct rd_msg *)icmp6_hdr(skb);
|
2012-07-12 06:43:53 +00:00
|
|
|
|
2013-01-05 16:34:51 +00:00
|
|
|
if (ipv6_addr_is_multicast(&msg->dest)) {
|
2012-07-12 07:05:02 +00:00
|
|
|
net_dbg_ratelimited("rt6_do_redirect: destination address is multicast\n");
|
2012-07-12 06:43:53 +00:00
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2012-07-12 07:05:02 +00:00
|
|
|
on_link = 0;
|
2013-01-05 16:34:51 +00:00
|
|
|
if (ipv6_addr_equal(&msg->dest, &msg->target)) {
|
2012-07-12 06:43:53 +00:00
|
|
|
on_link = 1;
|
2013-01-05 16:34:51 +00:00
|
|
|
} else if (ipv6_addr_type(&msg->target) !=
|
2012-07-12 06:43:53 +00:00
|
|
|
(IPV6_ADDR_UNICAST|IPV6_ADDR_LINKLOCAL)) {
|
2012-07-12 07:05:02 +00:00
|
|
|
net_dbg_ratelimited("rt6_do_redirect: target address is not link-local unicast\n");
|
2012-07-12 06:43:53 +00:00
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
in6_dev = __in6_dev_get(skb->dev);
|
|
|
|
if (!in6_dev)
|
|
|
|
return;
|
|
|
|
if (in6_dev->cnf.forwarding || !in6_dev->cnf.accept_redirects)
|
|
|
|
return;
|
|
|
|
|
|
|
|
/* RFC2461 8.1:
|
|
|
|
* The IP source address of the Redirect MUST be the same as the current
|
|
|
|
* first-hop router for the specified ICMP Destination Address.
|
|
|
|
*/
|
|
|
|
|
2016-06-15 19:20:23 +00:00
|
|
|
if (!ndisc_parse_options(skb->dev, msg->opt, optlen, &ndopts)) {
|
2012-07-12 06:43:53 +00:00
|
|
|
net_dbg_ratelimited("rt6_redirect: invalid ND options\n");
|
|
|
|
return;
|
|
|
|
}
|
2012-07-12 07:05:02 +00:00
|
|
|
|
|
|
|
lladdr = NULL;
|
2012-07-12 06:43:53 +00:00
|
|
|
if (ndopts.nd_opts_tgt_lladdr) {
|
|
|
|
lladdr = ndisc_opt_addr_data(ndopts.nd_opts_tgt_lladdr,
|
|
|
|
skb->dev);
|
|
|
|
if (!lladdr) {
|
|
|
|
net_dbg_ratelimited("rt6_redirect: invalid link-layer address length\n");
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2012-07-12 07:05:02 +00:00
|
|
|
rt = (struct rt6_info *) dst;
|
2015-11-02 00:24:38 +00:00
|
|
|
if (rt->rt6i_flags & RTF_REJECT) {
|
2012-07-12 07:05:02 +00:00
|
|
|
net_dbg_ratelimited("rt6_redirect: source isn't a valid nexthop for redirect target\n");
|
2012-07-12 06:43:53 +00:00
|
|
|
return;
|
2012-07-12 07:05:02 +00:00
|
|
|
}
|
2012-07-12 06:43:53 +00:00
|
|
|
|
2012-07-12 07:05:02 +00:00
|
|
|
/* Redirect received -> path was valid.
|
|
|
|
* Look, redirects are sent only in response to data packets,
|
|
|
|
* so that this nexthop apparently is reachable. --ANK
|
|
|
|
*/
|
2017-02-06 21:14:16 +00:00
|
|
|
dst_confirm_neigh(&rt->dst, &ipv6_hdr(skb)->saddr);
|
2006-08-24 00:18:26 +00:00
|
|
|
|
2013-01-05 16:34:51 +00:00
|
|
|
neigh = __neigh_lookup(&nd_tbl, &msg->target, skb->dev, 1);
|
2012-07-12 07:05:02 +00:00
|
|
|
if (!neigh)
|
|
|
|
return;
|
2006-08-24 00:18:26 +00:00
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/*
|
|
|
|
* We have finally decided to accept it.
|
|
|
|
*/
|
|
|
|
|
2016-06-15 19:20:23 +00:00
|
|
|
ndisc_update(skb->dev, neigh, lladdr, NUD_STALE,
|
2005-04-16 22:20:36 +00:00
|
|
|
NEIGH_UPDATE_F_WEAK_OVERRIDE|
|
|
|
|
NEIGH_UPDATE_F_OVERRIDE|
|
|
|
|
(on_link ? 0 : (NEIGH_UPDATE_F_OVERRIDE_ISROUTER|
|
2016-06-15 19:20:23 +00:00
|
|
|
NEIGH_UPDATE_F_ISROUTER)),
|
|
|
|
NDISC_REDIRECT, &ndopts);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-04-20 22:37:59 +00:00
|
|
|
rcu_read_lock();
|
2019-04-16 21:36:01 +00:00
|
|
|
res.f6i = rcu_dereference(rt->from);
|
2019-05-03 02:14:21 +00:00
|
|
|
if (!res.f6i)
|
2019-04-30 17:45:12 +00:00
|
|
|
goto out;
|
2018-04-23 18:32:07 +00:00
|
|
|
|
2019-06-08 21:53:30 +00:00
|
|
|
if (res.f6i->nh) {
|
|
|
|
struct fib6_nh_match_arg arg = {
|
|
|
|
.dev = dst->dev,
|
|
|
|
.gw = &rt->rt6i_gateway,
|
|
|
|
};
|
|
|
|
|
|
|
|
nexthop_for_each_fib6_nh(res.f6i->nh,
|
|
|
|
fib6_nh_find_match, &arg);
|
|
|
|
|
|
|
|
/* fib6_info uses a nexthop that does not have fib6_nh
|
|
|
|
* using the dst->dev. Should be impossible
|
|
|
|
*/
|
|
|
|
if (!arg.match)
|
|
|
|
goto out;
|
|
|
|
res.nh = arg.match;
|
|
|
|
} else {
|
|
|
|
res.nh = res.f6i->fib6_nh;
|
|
|
|
}
|
|
|
|
|
2019-04-16 21:36:11 +00:00
|
|
|
res.fib6_flags = res.f6i->fib6_flags;
|
|
|
|
res.fib6_type = res.f6i->fib6_type;
|
2019-04-16 21:36:01 +00:00
|
|
|
nrt = ip6_rt_cache_alloc(&res, &msg->dest, NULL);
|
2011-12-03 23:02:47 +00:00
|
|
|
if (!nrt)
|
2005-04-16 22:20:36 +00:00
|
|
|
goto out;
|
|
|
|
|
|
|
|
nrt->rt6i_flags = RTF_GATEWAY|RTF_UP|RTF_DYNAMIC|RTF_CACHE;
|
|
|
|
if (on_link)
|
|
|
|
nrt->rt6i_flags &= ~RTF_GATEWAY;
|
|
|
|
|
2011-11-21 03:39:03 +00:00
|
|
|
nrt->rt6i_gateway = *(struct in6_addr *)neigh->primary_key;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-04-30 17:45:12 +00:00
|
|
|
/* rt6_insert_exception() will take care of duplicated exceptions */
|
2019-04-16 21:36:05 +00:00
|
|
|
if (rt6_insert_exception(nrt, &res)) {
|
2017-10-06 19:06:03 +00:00
|
|
|
dst_release_immediate(&nrt->dst);
|
|
|
|
goto out;
|
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2010-06-11 06:31:35 +00:00
|
|
|
netevent.old = &rt->dst;
|
|
|
|
netevent.new = &nrt->dst;
|
2013-01-05 16:34:51 +00:00
|
|
|
netevent.daddr = &msg->dest;
|
2013-01-14 09:28:27 +00:00
|
|
|
netevent.neigh = neigh;
|
2006-07-31 03:43:36 +00:00
|
|
|
call_netevent_notifiers(NETEVENT_REDIRECT, &netevent);
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
out:
|
2019-04-30 17:45:12 +00:00
|
|
|
rcu_read_unlock();
|
2012-07-12 06:43:53 +00:00
|
|
|
neigh_release(neigh);
|
2012-07-12 07:05:02 +00:00
|
|
|
}
|
|
|
|
|
2006-03-21 01:06:24 +00:00
|
|
|
#ifdef CONFIG_IPV6_ROUTE_INFO
|
2018-04-18 00:33:26 +00:00
|
|
|
static struct fib6_info *rt6_get_route_info(struct net *net,
|
2011-04-22 04:53:02 +00:00
|
|
|
const struct in6_addr *prefix, int prefixlen,
|
2016-10-24 17:52:35 +00:00
|
|
|
const struct in6_addr *gwaddr,
|
|
|
|
struct net_device *dev)
|
2006-03-21 01:06:24 +00:00
|
|
|
{
|
2016-10-24 17:52:35 +00:00
|
|
|
u32 tb_id = l3mdev_fib_table(dev) ? : RT6_TABLE_INFO;
|
|
|
|
int ifindex = dev->ifindex;
|
2006-03-21 01:06:24 +00:00
|
|
|
struct fib6_node *fn;
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt = NULL;
|
2006-08-05 06:20:06 +00:00
|
|
|
struct fib6_table *table;
|
|
|
|
|
2016-10-24 17:52:35 +00:00
|
|
|
table = fib6_get_table(net, tb_id);
|
2011-12-03 23:02:47 +00:00
|
|
|
if (!table)
|
2006-08-05 06:20:06 +00:00
|
|
|
return NULL;
|
2006-03-21 01:06:24 +00:00
|
|
|
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_lock();
|
2017-10-06 19:06:02 +00:00
|
|
|
fn = fib6_locate(&table->tb6_root, prefix, prefixlen, NULL, 0, true);
|
2006-03-21 01:06:24 +00:00
|
|
|
if (!fn)
|
|
|
|
goto out;
|
|
|
|
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
for_each_fib6_node_rt_rcu(fn) {
|
2019-06-04 03:19:52 +00:00
|
|
|
/* these routes do not use nexthops */
|
|
|
|
if (rt->nh)
|
|
|
|
continue;
|
2019-05-23 03:27:59 +00:00
|
|
|
if (rt->fib6_nh->fib_nh_dev->ifindex != ifindex)
|
2006-03-21 01:06:24 +00:00
|
|
|
continue;
|
2019-03-28 03:53:52 +00:00
|
|
|
if (!(rt->fib6_flags & RTF_ROUTEINFO) ||
|
2019-05-23 03:27:59 +00:00
|
|
|
!rt->fib6_nh->fib_nh_gw_family)
|
2006-03-21 01:06:24 +00:00
|
|
|
continue;
|
2019-05-23 03:27:59 +00:00
|
|
|
if (!ipv6_addr_equal(&rt->fib6_nh->fib_nh_gw6, gwaddr))
|
2006-03-21 01:06:24 +00:00
|
|
|
continue;
|
2018-07-22 03:56:32 +00:00
|
|
|
if (!fib6_info_hold_safe(rt))
|
|
|
|
continue;
|
2006-03-21 01:06:24 +00:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
out:
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_unlock();
|
2006-03-21 01:06:24 +00:00
|
|
|
return rt;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static struct fib6_info *rt6_add_route_info(struct net *net,
|
2011-04-22 04:53:02 +00:00
|
|
|
const struct in6_addr *prefix, int prefixlen,
|
2016-10-24 17:52:35 +00:00
|
|
|
const struct in6_addr *gwaddr,
|
|
|
|
struct net_device *dev,
|
2012-04-15 05:58:06 +00:00
|
|
|
unsigned int pref)
|
2006-03-21 01:06:24 +00:00
|
|
|
{
|
2006-08-22 07:01:08 +00:00
|
|
|
struct fib6_config cfg = {
|
2008-02-10 07:43:11 +00:00
|
|
|
.fc_metric = IP6_RT_PRIO_USER,
|
2016-10-24 17:52:35 +00:00
|
|
|
.fc_ifindex = dev->ifindex,
|
2006-08-22 07:01:08 +00:00
|
|
|
.fc_dst_len = prefixlen,
|
|
|
|
.fc_flags = RTF_GATEWAY | RTF_ADDRCONF | RTF_ROUTEINFO |
|
|
|
|
RTF_UP | RTF_PREF(pref),
|
2017-08-03 06:13:46 +00:00
|
|
|
.fc_protocol = RTPROT_RA,
|
2018-04-18 00:33:13 +00:00
|
|
|
.fc_type = RTN_UNICAST,
|
2012-09-07 20:12:54 +00:00
|
|
|
.fc_nlinfo.portid = 0,
|
2008-03-04 21:46:48 +00:00
|
|
|
.fc_nlinfo.nlh = NULL,
|
|
|
|
.fc_nlinfo.nl_net = net,
|
2006-08-22 07:01:08 +00:00
|
|
|
};
|
|
|
|
|
2020-09-21 06:38:56 +00:00
|
|
|
cfg.fc_table = l3mdev_fib_table(dev) ? : RT6_TABLE_INFO;
|
2011-11-21 03:39:03 +00:00
|
|
|
cfg.fc_dst = *prefix;
|
|
|
|
cfg.fc_gateway = *gwaddr;
|
2006-03-21 01:06:24 +00:00
|
|
|
|
2006-03-21 01:06:42 +00:00
|
|
|
/* We should treat it as a default route if prefix length is 0. */
|
|
|
|
if (!prefixlen)
|
2006-08-22 07:01:08 +00:00
|
|
|
cfg.fc_flags |= RTF_DEFAULT;
|
2006-03-21 01:06:24 +00:00
|
|
|
|
2018-04-18 00:33:22 +00:00
|
|
|
ip6_route_add(&cfg, GFP_ATOMIC, NULL);
|
2006-03-21 01:06:24 +00:00
|
|
|
|
2016-10-24 17:52:35 +00:00
|
|
|
return rt6_get_route_info(net, prefix, prefixlen, gwaddr, dev);
|
2006-03-21 01:06:24 +00:00
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt6_get_dflt_router(struct net *net,
|
2018-04-18 00:33:11 +00:00
|
|
|
const struct in6_addr *addr,
|
|
|
|
struct net_device *dev)
|
2007-02-09 14:24:49 +00:00
|
|
|
{
|
2016-10-24 17:52:35 +00:00
|
|
|
u32 tb_id = l3mdev_fib_table(dev) ? : RT6_TABLE_DFLT;
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt;
|
2006-08-05 06:20:06 +00:00
|
|
|
struct fib6_table *table;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-04-18 00:33:11 +00:00
|
|
|
table = fib6_get_table(net, tb_id);
|
2011-12-03 23:02:47 +00:00
|
|
|
if (!table)
|
2006-08-05 06:20:06 +00:00
|
|
|
return NULL;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_lock();
|
|
|
|
for_each_fib6_node_rt_rcu(&table->tb6_root) {
|
2019-06-04 03:19:52 +00:00
|
|
|
struct fib6_nh *nh;
|
2019-03-28 03:53:56 +00:00
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
/* RA routes do not use nexthops */
|
|
|
|
if (rt->nh)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
nh = rt->fib6_nh;
|
2019-03-28 03:53:56 +00:00
|
|
|
if (dev == nh->fib_nh_dev &&
|
2018-04-18 22:38:59 +00:00
|
|
|
((rt->fib6_flags & (RTF_ADDRCONF | RTF_DEFAULT)) == (RTF_ADDRCONF | RTF_DEFAULT)) &&
|
2019-03-28 03:53:56 +00:00
|
|
|
ipv6_addr_equal(&nh->fib_nh_gw6, addr))
|
2005-04-16 22:20:36 +00:00
|
|
|
break;
|
|
|
|
}
|
2018-07-22 03:56:32 +00:00
|
|
|
if (rt && !fib6_info_hold_safe(rt))
|
|
|
|
rt = NULL;
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_unlock();
|
2005-04-16 22:20:36 +00:00
|
|
|
return rt;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt6_add_dflt_router(struct net *net,
|
2018-04-18 00:33:11 +00:00
|
|
|
const struct in6_addr *gwaddr,
|
2006-03-21 01:04:53 +00:00
|
|
|
struct net_device *dev,
|
net: allow user to set metric on default route learned via Router Advertisement
For IPv4, default route is learned via DHCPv4 and user is allowed to change
metric using config etc/network/interfaces. But for IPv6, default route can
be learned via RA, for which, currently a fixed metric value 1024 is used.
Ideally, user should be able to configure metric on default route for IPv6
similar to IPv4. This patch adds sysctl for the same.
Logs:
For IPv4:
Config in etc/network/interfaces:
auto eth0
iface eth0 inet dhcp
metric 4261413864
IPv4 Kernel Route Table:
$ ip route list
default via 172.21.47.1 dev eth0 metric 4261413864
FRR Table, if a static route is configured:
[In real scenario, it is useful to prefer BGP learned default route over DHCPv4 default route.]
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
> - selected route, * - FIB route
S>* 0.0.0.0/0 [20/0] is directly connected, eth0, 00:00:03
K 0.0.0.0/0 [254/1000] via 172.21.47.1, eth0, 6d08h51m
i.e. User can prefer Default Router learned via Routing Protocol in IPv4.
Similar behavior is not possible for IPv6, without this fix.
After fix [for IPv6]:
sudo sysctl -w net.ipv6.conf.eth0.net.ipv6.conf.eth0.ra_defrtr_metric=1996489705
IP monitor: [When IPv6 RA is received]
default via fe80::xx16:xxxx:feb3:ce8e dev eth0 proto ra metric 1996489705 pref high
Kernel IPv6 routing table
$ ip -6 route list
default via fe80::be16:65ff:feb3:ce8e dev eth0 proto ra metric 1996489705 expires 21sec hoplimit 64 pref high
FRR Table, if a static route is configured:
[In real scenario, it is useful to prefer BGP learned default route over IPv6 RA default route.]
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
> - selected route, * - FIB route
S>* ::/0 [20/0] is directly connected, eth0, 00:00:06
K ::/0 [119/1001] via fe80::xx16:xxxx:feb3:ce8e, eth0, 6d07h43m
If the metric is changed later, the effect will be seen only when next IPv6
RA is received, because the default route must be fully controlled by RA msg.
Below metric is changed from 1996489705 to 1996489704.
$ sudo sysctl -w net.ipv6.conf.eth0.ra_defrtr_metric=1996489704
net.ipv6.conf.eth0.ra_defrtr_metric = 1996489704
IP monitor:
[On next IPv6 RA msg, Kernel deletes prev route and installs new route with updated metric]
Deleted default via fe80::xx16:xxxx:feb3:ce8e dev eth0 proto ra metric 1996489705 expires 3sec hoplimit 64 pref high
default via fe80::xx16:xxxx:feb3:ce8e dev eth0 proto ra metric 1996489704 pref high
Signed-off-by: Praveen Chaudhary <pchaudhary@linkedin.com>
Signed-off-by: Zhenggen Xu <zxu@linkedin.com>
Reviewed-by: David Ahern <dsahern@kernel.org>
Link: https://lore.kernel.org/r/20210125214430.24079-1-pchaudhary@linkedin.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2021-01-25 21:44:30 +00:00
|
|
|
unsigned int pref,
|
|
|
|
u32 defrtr_usr_metric)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2006-08-22 07:01:08 +00:00
|
|
|
struct fib6_config cfg = {
|
2015-10-12 18:47:10 +00:00
|
|
|
.fc_table = l3mdev_fib_table(dev) ? : RT6_TABLE_DFLT,
|
net: allow user to set metric on default route learned via Router Advertisement
For IPv4, default route is learned via DHCPv4 and user is allowed to change
metric using config etc/network/interfaces. But for IPv6, default route can
be learned via RA, for which, currently a fixed metric value 1024 is used.
Ideally, user should be able to configure metric on default route for IPv6
similar to IPv4. This patch adds sysctl for the same.
Logs:
For IPv4:
Config in etc/network/interfaces:
auto eth0
iface eth0 inet dhcp
metric 4261413864
IPv4 Kernel Route Table:
$ ip route list
default via 172.21.47.1 dev eth0 metric 4261413864
FRR Table, if a static route is configured:
[In real scenario, it is useful to prefer BGP learned default route over DHCPv4 default route.]
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
> - selected route, * - FIB route
S>* 0.0.0.0/0 [20/0] is directly connected, eth0, 00:00:03
K 0.0.0.0/0 [254/1000] via 172.21.47.1, eth0, 6d08h51m
i.e. User can prefer Default Router learned via Routing Protocol in IPv4.
Similar behavior is not possible for IPv6, without this fix.
After fix [for IPv6]:
sudo sysctl -w net.ipv6.conf.eth0.net.ipv6.conf.eth0.ra_defrtr_metric=1996489705
IP monitor: [When IPv6 RA is received]
default via fe80::xx16:xxxx:feb3:ce8e dev eth0 proto ra metric 1996489705 pref high
Kernel IPv6 routing table
$ ip -6 route list
default via fe80::be16:65ff:feb3:ce8e dev eth0 proto ra metric 1996489705 expires 21sec hoplimit 64 pref high
FRR Table, if a static route is configured:
[In real scenario, it is useful to prefer BGP learned default route over IPv6 RA default route.]
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
> - selected route, * - FIB route
S>* ::/0 [20/0] is directly connected, eth0, 00:00:06
K ::/0 [119/1001] via fe80::xx16:xxxx:feb3:ce8e, eth0, 6d07h43m
If the metric is changed later, the effect will be seen only when next IPv6
RA is received, because the default route must be fully controlled by RA msg.
Below metric is changed from 1996489705 to 1996489704.
$ sudo sysctl -w net.ipv6.conf.eth0.ra_defrtr_metric=1996489704
net.ipv6.conf.eth0.ra_defrtr_metric = 1996489704
IP monitor:
[On next IPv6 RA msg, Kernel deletes prev route and installs new route with updated metric]
Deleted default via fe80::xx16:xxxx:feb3:ce8e dev eth0 proto ra metric 1996489705 expires 3sec hoplimit 64 pref high
default via fe80::xx16:xxxx:feb3:ce8e dev eth0 proto ra metric 1996489704 pref high
Signed-off-by: Praveen Chaudhary <pchaudhary@linkedin.com>
Signed-off-by: Zhenggen Xu <zxu@linkedin.com>
Reviewed-by: David Ahern <dsahern@kernel.org>
Link: https://lore.kernel.org/r/20210125214430.24079-1-pchaudhary@linkedin.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2021-01-25 21:44:30 +00:00
|
|
|
.fc_metric = defrtr_usr_metric,
|
2006-08-22 07:01:08 +00:00
|
|
|
.fc_ifindex = dev->ifindex,
|
|
|
|
.fc_flags = RTF_GATEWAY | RTF_ADDRCONF | RTF_DEFAULT |
|
|
|
|
RTF_UP | RTF_EXPIRES | RTF_PREF(pref),
|
2017-08-03 06:13:46 +00:00
|
|
|
.fc_protocol = RTPROT_RA,
|
2018-04-18 00:33:13 +00:00
|
|
|
.fc_type = RTN_UNICAST,
|
2012-09-07 20:12:54 +00:00
|
|
|
.fc_nlinfo.portid = 0,
|
2008-03-04 21:47:47 +00:00
|
|
|
.fc_nlinfo.nlh = NULL,
|
2018-04-18 00:33:11 +00:00
|
|
|
.fc_nlinfo.nl_net = net,
|
2006-08-22 07:01:08 +00:00
|
|
|
};
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2011-11-21 03:39:03 +00:00
|
|
|
cfg.fc_gateway = *gwaddr;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-04-18 00:33:22 +00:00
|
|
|
if (!ip6_route_add(&cfg, GFP_ATOMIC, NULL)) {
|
2016-10-24 17:52:35 +00:00
|
|
|
struct fib6_table *table;
|
|
|
|
|
|
|
|
table = fib6_get_table(dev_net(dev), cfg.fc_table);
|
|
|
|
if (table)
|
|
|
|
table->flags |= RT6_TABLE_HAS_DFLT_ROUTER;
|
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-04-18 00:33:11 +00:00
|
|
|
return rt6_get_dflt_router(net, gwaddr, dev);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:11 +00:00
|
|
|
static void __rt6_purge_dflt_routers(struct net *net,
|
|
|
|
struct fib6_table *table)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
restart:
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_lock();
|
|
|
|
for_each_fib6_node_rt_rcu(&table->tb6_root) {
|
2018-04-18 22:39:05 +00:00
|
|
|
struct net_device *dev = fib6_info_nh_dev(rt);
|
|
|
|
struct inet6_dev *idev = dev ? __in6_dev_get(dev) : NULL;
|
|
|
|
|
2018-04-18 22:38:59 +00:00
|
|
|
if (rt->fib6_flags & (RTF_DEFAULT | RTF_ADDRCONF) &&
|
2018-07-22 03:56:32 +00:00
|
|
|
(!idev || idev->cnf.accept_ra != 2) &&
|
|
|
|
fib6_info_hold_safe(rt)) {
|
2018-04-18 00:33:25 +00:00
|
|
|
rcu_read_unlock();
|
2020-04-27 20:56:45 +00:00
|
|
|
ip6_del_rt(net, rt, false);
|
2005-04-16 22:20:36 +00:00
|
|
|
goto restart;
|
|
|
|
}
|
|
|
|
}
|
ipv6: replace rwlock with rcu and spinlock in fib6_table
With all the preparation work before, we are now ready to replace rwlock
with rcu and spinlock in fib6_table.
That means now all fib6_node in fib6_table are protected by rcu. And
when freeing fib6_node, call_rcu() is used to wait for the rcu grace
period before releasing the memory.
When accessing fib6_node, corresponding rcu APIs need to be used.
And all previous sessions protected by the write lock will now be
protected by the spin lock per table.
All previous sessions protected by read lock will now be protected by
rcu_read_lock().
A couple of things to note here:
1. As part of the work of replacing rwlock with rcu, the linked list of
fn->leaf now has to be rcu protected as well. So both fn->leaf and
rt->dst.rt6_next are now __rcu tagged and corresponding rcu APIs are
used when manipulating them.
2. For fn->rr_ptr, first of all, it also needs to be rcu protected now
and is tagged with __rcu and rcu APIs are used in corresponding places.
Secondly, fn->rr_ptr is changed in rt6_select() which is a reader
thread. This makes the issue a bit complicated. We think a valid
solution for it is to let rt6_select() grab the tb6_lock if it decides
to change it. As it is not in the normal operation and only happens when
there is no valid neighbor cache for the route, we think the performance
impact should be low.
3. fib6_walk_continue() has to be called with tb6_lock held even in the
route dumping related functions, e.g. inet6_dump_fib(),
fib6_tables_dump() and ipv6_route_seq_ops. It is because
fib6_walk_continue() makes modifications to the walker structure, and so
are fib6_repair_tree() and fib6_del_route(). In order to do proper
syncing between them, we need to let fib6_walk_continue() hold the lock.
We may be able to do further improvement on the way we do the tree walk
to get rid of the need for holding the spin lock. But not for now.
4. When fib6_del_route() removes a route from the tree, we no longer
mark rt->dst.rt6_next to NULL to make simultaneous reader be able to
further traverse the list with rcu. However, rt->dst.rt6_next is only
valid within this same rcu period. No one should access it later.
5. All the operation of atomic_inc(rt->rt6i_ref) is changed to be
performed before we publish this route (either by linking it to fn->leaf
or insert it in the list pointed by fn->leaf) just to be safe because as
soon as we publish the route, some read thread will be able to access it.
Signed-off-by: Wei Wang <weiwan@google.com>
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-10-06 19:06:10 +00:00
|
|
|
rcu_read_unlock();
|
2016-10-24 17:52:35 +00:00
|
|
|
|
|
|
|
table->flags &= ~RT6_TABLE_HAS_DFLT_ROUTER;
|
|
|
|
}
|
|
|
|
|
|
|
|
void rt6_purge_dflt_routers(struct net *net)
|
|
|
|
{
|
|
|
|
struct fib6_table *table;
|
|
|
|
struct hlist_head *head;
|
|
|
|
unsigned int h;
|
|
|
|
|
|
|
|
rcu_read_lock();
|
|
|
|
|
|
|
|
for (h = 0; h < FIB6_TABLE_HASHSZ; h++) {
|
|
|
|
head = &net->ipv6.fib_table_hash[h];
|
|
|
|
hlist_for_each_entry_rcu(table, head, tb6_hlist) {
|
|
|
|
if (table->flags & RT6_TABLE_HAS_DFLT_ROUTER)
|
2018-04-18 00:33:11 +00:00
|
|
|
__rt6_purge_dflt_routers(net, table);
|
2016-10-24 17:52:35 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
rcu_read_unlock();
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2008-03-04 21:47:47 +00:00
|
|
|
static void rtmsg_to_fib6_config(struct net *net,
|
|
|
|
struct in6_rtmsg *rtmsg,
|
2006-08-22 07:01:08 +00:00
|
|
|
struct fib6_config *cfg)
|
|
|
|
{
|
2018-09-30 06:44:52 +00:00
|
|
|
*cfg = (struct fib6_config){
|
|
|
|
.fc_table = l3mdev_fib_table_by_index(net, rtmsg->rtmsg_ifindex) ?
|
|
|
|
: RT6_TABLE_MAIN,
|
|
|
|
.fc_ifindex = rtmsg->rtmsg_ifindex,
|
2019-03-21 12:21:34 +00:00
|
|
|
.fc_metric = rtmsg->rtmsg_metric ? : IP6_RT_PRIO_USER,
|
2018-09-30 06:44:52 +00:00
|
|
|
.fc_expires = rtmsg->rtmsg_info,
|
|
|
|
.fc_dst_len = rtmsg->rtmsg_dst_len,
|
|
|
|
.fc_src_len = rtmsg->rtmsg_src_len,
|
|
|
|
.fc_flags = rtmsg->rtmsg_flags,
|
|
|
|
.fc_type = rtmsg->rtmsg_type,
|
|
|
|
|
|
|
|
.fc_nlinfo.nl_net = net,
|
|
|
|
|
|
|
|
.fc_dst = rtmsg->rtmsg_dst,
|
|
|
|
.fc_src = rtmsg->rtmsg_src,
|
|
|
|
.fc_gateway = rtmsg->rtmsg_gateway,
|
|
|
|
};
|
2006-08-22 07:01:08 +00:00
|
|
|
}
|
|
|
|
|
2020-05-18 06:28:05 +00:00
|
|
|
int ipv6_route_ioctl(struct net *net, unsigned int cmd, struct in6_rtmsg *rtmsg)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2006-08-22 07:01:08 +00:00
|
|
|
struct fib6_config cfg;
|
2005-04-16 22:20:36 +00:00
|
|
|
int err;
|
|
|
|
|
2020-05-18 06:28:05 +00:00
|
|
|
if (cmd != SIOCADDRT && cmd != SIOCDELRT)
|
|
|
|
return -EINVAL;
|
|
|
|
if (!ns_capable(net->user_ns, CAP_NET_ADMIN))
|
|
|
|
return -EPERM;
|
2006-08-22 07:01:08 +00:00
|
|
|
|
2020-05-18 06:28:05 +00:00
|
|
|
rtmsg_to_fib6_config(net, rtmsg, &cfg);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2020-05-18 06:28:05 +00:00
|
|
|
rtnl_lock();
|
|
|
|
switch (cmd) {
|
|
|
|
case SIOCADDRT:
|
|
|
|
err = ip6_route_add(&cfg, GFP_KERNEL, NULL);
|
|
|
|
break;
|
|
|
|
case SIOCDELRT:
|
|
|
|
err = ip6_route_del(&cfg, NULL);
|
|
|
|
break;
|
2007-04-21 00:09:22 +00:00
|
|
|
}
|
2020-05-18 06:28:05 +00:00
|
|
|
rtnl_unlock();
|
|
|
|
return err;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Drop the packet on the floor
|
|
|
|
*/
|
|
|
|
|
2009-06-23 11:31:07 +00:00
|
|
|
static int ip6_pkt_drop(struct sk_buff *skb, u8 code, int ipstats_mib_noroutes)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2009-06-02 05:19:30 +00:00
|
|
|
struct dst_entry *dst = skb_dst(skb);
|
2019-04-27 13:14:33 +00:00
|
|
|
struct net *net = dev_net(dst->dev);
|
|
|
|
struct inet6_dev *idev;
|
|
|
|
int type;
|
|
|
|
|
|
|
|
if (netif_is_l3_master(skb->dev) &&
|
|
|
|
dst->dev == net->loopback_dev)
|
|
|
|
idev = __in6_dev_get_safely(dev_get_by_index_rcu(net, IP6CB(skb)->iif));
|
|
|
|
else
|
|
|
|
idev = ip6_dst_idev(dst);
|
|
|
|
|
2007-04-13 23:18:02 +00:00
|
|
|
switch (ipstats_mib_noroutes) {
|
|
|
|
case IPSTATS_MIB_INNOROUTES:
|
2007-04-26 00:54:47 +00:00
|
|
|
type = ipv6_addr_type(&ipv6_hdr(skb)->daddr);
|
2010-02-25 23:28:58 +00:00
|
|
|
if (type == IPV6_ADDR_ANY) {
|
2019-04-27 13:14:33 +00:00
|
|
|
IP6_INC_STATS(net, idev, IPSTATS_MIB_INADDRERRORS);
|
2007-04-13 23:18:02 +00:00
|
|
|
break;
|
|
|
|
}
|
2020-03-12 22:50:22 +00:00
|
|
|
fallthrough;
|
2007-04-13 23:18:02 +00:00
|
|
|
case IPSTATS_MIB_OUTNOROUTES:
|
2019-04-27 13:14:33 +00:00
|
|
|
IP6_INC_STATS(net, idev, ipstats_mib_noroutes);
|
2007-04-13 23:18:02 +00:00
|
|
|
break;
|
|
|
|
}
|
2019-04-27 13:14:33 +00:00
|
|
|
|
|
|
|
/* Start over by dropping the dst for l3mdev case */
|
|
|
|
if (netif_is_l3_master(skb->dev))
|
|
|
|
skb_dst_drop(skb);
|
|
|
|
|
2010-02-18 08:25:24 +00:00
|
|
|
icmpv6_send(skb, ICMPV6_DEST_UNREACH, code, 0);
|
2005-04-16 22:20:36 +00:00
|
|
|
kfree_skb(skb);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2006-10-19 03:46:54 +00:00
|
|
|
static int ip6_pkt_discard(struct sk_buff *skb)
|
|
|
|
{
|
2007-04-13 23:18:02 +00:00
|
|
|
return ip6_pkt_drop(skb, ICMPV6_NOROUTE, IPSTATS_MIB_INNOROUTES);
|
2006-10-19 03:46:54 +00:00
|
|
|
}
|
|
|
|
|
2015-10-07 21:48:47 +00:00
|
|
|
static int ip6_pkt_discard_out(struct net *net, struct sock *sk, struct sk_buff *skb)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2009-06-02 05:19:30 +00:00
|
|
|
skb->dev = skb_dst(skb)->dev;
|
2007-04-13 23:18:02 +00:00
|
|
|
return ip6_pkt_drop(skb, ICMPV6_NOROUTE, IPSTATS_MIB_OUTNOROUTES);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2006-10-19 03:46:54 +00:00
|
|
|
static int ip6_pkt_prohibit(struct sk_buff *skb)
|
|
|
|
{
|
2007-04-13 23:18:02 +00:00
|
|
|
return ip6_pkt_drop(skb, ICMPV6_ADM_PROHIBITED, IPSTATS_MIB_INNOROUTES);
|
2006-10-19 03:46:54 +00:00
|
|
|
}
|
|
|
|
|
2015-10-07 21:48:47 +00:00
|
|
|
static int ip6_pkt_prohibit_out(struct net *net, struct sock *sk, struct sk_buff *skb)
|
2006-10-19 03:46:54 +00:00
|
|
|
{
|
2009-06-02 05:19:30 +00:00
|
|
|
skb->dev = skb_dst(skb)->dev;
|
2007-04-13 23:18:02 +00:00
|
|
|
return ip6_pkt_drop(skb, ICMPV6_ADM_PROHIBITED, IPSTATS_MIB_OUTNOROUTES);
|
2006-10-19 03:46:54 +00:00
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/*
|
|
|
|
* Allocate a dst for local (unicast / anycast) address.
|
|
|
|
*/
|
|
|
|
|
2018-04-18 22:39:00 +00:00
|
|
|
struct fib6_info *addrconf_f6i_alloc(struct net *net,
|
|
|
|
struct inet6_dev *idev,
|
|
|
|
const struct in6_addr *addr,
|
|
|
|
bool anycast, gfp_t gfp_flags)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2019-03-21 12:21:35 +00:00
|
|
|
struct fib6_config cfg = {
|
|
|
|
.fc_table = l3mdev_fib_table(idev->dev) ? : RT6_TABLE_LOCAL,
|
|
|
|
.fc_ifindex = idev->dev->ifindex,
|
2019-09-02 16:23:36 +00:00
|
|
|
.fc_flags = RTF_UP | RTF_NONEXTHOP,
|
2019-03-21 12:21:35 +00:00
|
|
|
.fc_dst = *addr,
|
|
|
|
.fc_dst_len = 128,
|
|
|
|
.fc_protocol = RTPROT_KERNEL,
|
|
|
|
.fc_nlinfo.nl_net = net,
|
|
|
|
.fc_ignore_dev_down = true,
|
|
|
|
};
|
2019-09-02 16:23:36 +00:00
|
|
|
struct fib6_info *f6i;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-04-18 00:33:13 +00:00
|
|
|
if (anycast) {
|
2019-03-21 12:21:35 +00:00
|
|
|
cfg.fc_type = RTN_ANYCAST;
|
|
|
|
cfg.fc_flags |= RTF_ANYCAST;
|
2018-04-18 00:33:13 +00:00
|
|
|
} else {
|
2019-03-21 12:21:35 +00:00
|
|
|
cfg.fc_type = RTN_LOCAL;
|
|
|
|
cfg.fc_flags |= RTF_LOCAL;
|
2018-04-18 00:33:13 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-09-02 16:23:36 +00:00
|
|
|
f6i = ip6_route_info_create(&cfg, gfp_flags, NULL);
|
2019-09-06 03:56:37 +00:00
|
|
|
if (!IS_ERR(f6i))
|
2019-09-02 16:23:36 +00:00
|
|
|
f6i->dst_nocount = true;
|
|
|
|
return f6i;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2011-04-13 21:10:57 +00:00
|
|
|
/* remove deleted ip from prefsrc entries */
|
|
|
|
struct arg_dev_net_ip {
|
|
|
|
struct net_device *dev;
|
|
|
|
struct net *net;
|
|
|
|
struct in6_addr *addr;
|
|
|
|
};
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static int fib6_remove_prefsrc(struct fib6_info *rt, void *arg)
|
2011-04-13 21:10:57 +00:00
|
|
|
{
|
|
|
|
struct net_device *dev = ((struct arg_dev_net_ip *)arg)->dev;
|
|
|
|
struct net *net = ((struct arg_dev_net_ip *)arg)->net;
|
|
|
|
struct in6_addr *addr = ((struct arg_dev_net_ip *)arg)->addr;
|
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
if (!rt->nh &&
|
|
|
|
((void *)rt->fib6_nh->fib_nh_dev == dev || !dev) &&
|
2018-04-18 00:33:18 +00:00
|
|
|
rt != net->ipv6.fib6_null_entry &&
|
2018-04-18 22:38:59 +00:00
|
|
|
ipv6_addr_equal(addr, &rt->fib6_prefsrc.addr)) {
|
2017-10-06 19:05:58 +00:00
|
|
|
spin_lock_bh(&rt6_exception_lock);
|
2011-04-13 21:10:57 +00:00
|
|
|
/* remove prefsrc entry */
|
2018-04-18 22:38:59 +00:00
|
|
|
rt->fib6_prefsrc.plen = 0;
|
2017-10-06 19:05:58 +00:00
|
|
|
spin_unlock_bh(&rt6_exception_lock);
|
2011-04-13 21:10:57 +00:00
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
void rt6_remove_prefsrc(struct inet6_ifaddr *ifp)
|
|
|
|
{
|
|
|
|
struct net *net = dev_net(ifp->idev->dev);
|
|
|
|
struct arg_dev_net_ip adni = {
|
|
|
|
.dev = ifp->idev->dev,
|
|
|
|
.net = net,
|
|
|
|
.addr = &ifp->addr,
|
|
|
|
};
|
2013-12-27 08:32:38 +00:00
|
|
|
fib6_clean_all(net, fib6_remove_prefsrc, &adni);
|
2011-04-13 21:10:57 +00:00
|
|
|
}
|
|
|
|
|
2019-03-28 03:53:52 +00:00
|
|
|
#define RTF_RA_ROUTER (RTF_ADDRCONF | RTF_DEFAULT)
|
2014-05-15 07:56:14 +00:00
|
|
|
|
|
|
|
/* Remove routers and update dst entries when gateway turn into host. */
|
2018-04-18 00:33:26 +00:00
|
|
|
static int fib6_clean_tohost(struct fib6_info *rt, void *arg)
|
2014-05-15 07:56:14 +00:00
|
|
|
{
|
|
|
|
struct in6_addr *gateway = (struct in6_addr *)arg;
|
2019-06-04 03:19:52 +00:00
|
|
|
struct fib6_nh *nh;
|
|
|
|
|
|
|
|
/* RA routes do not use nexthops */
|
|
|
|
if (rt->nh)
|
|
|
|
return 0;
|
2014-05-15 07:56:14 +00:00
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
nh = rt->fib6_nh;
|
2018-04-18 22:38:59 +00:00
|
|
|
if (((rt->fib6_flags & RTF_RA_ROUTER) == RTF_RA_ROUTER) &&
|
2019-05-23 03:27:58 +00:00
|
|
|
nh->fib_nh_gw_family && ipv6_addr_equal(gateway, &nh->fib_nh_gw6))
|
2014-05-15 07:56:14 +00:00
|
|
|
return -1;
|
2017-10-06 19:06:00 +00:00
|
|
|
|
|
|
|
/* Further clean up cached routes in exception table.
|
|
|
|
* This is needed because cached route may have a different
|
|
|
|
* gateway than its 'parent' in the case of an ip redirect.
|
|
|
|
*/
|
2019-05-23 03:27:58 +00:00
|
|
|
fib6_nh_exceptions_clean_tohost(nh, gateway);
|
2017-10-06 19:06:00 +00:00
|
|
|
|
2014-05-15 07:56:14 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
void rt6_clean_tohost(struct net *net, struct in6_addr *gateway)
|
|
|
|
{
|
|
|
|
fib6_clean_all(net, fib6_clean_tohost, gateway);
|
|
|
|
}
|
|
|
|
|
2018-01-07 10:45:03 +00:00
|
|
|
struct arg_netdev_event {
|
|
|
|
const struct net_device *dev;
|
2018-01-07 10:45:04 +00:00
|
|
|
union {
|
2019-04-23 15:48:09 +00:00
|
|
|
unsigned char nh_flags;
|
2018-01-07 10:45:04 +00:00
|
|
|
unsigned long event;
|
|
|
|
};
|
2018-01-07 10:45:03 +00:00
|
|
|
};
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static struct fib6_info *rt6_multipath_first_sibling(const struct fib6_info *rt)
|
2018-01-09 14:40:25 +00:00
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *iter;
|
2018-01-09 14:40:25 +00:00
|
|
|
struct fib6_node *fn;
|
|
|
|
|
2018-04-18 22:38:59 +00:00
|
|
|
fn = rcu_dereference_protected(rt->fib6_node,
|
|
|
|
lockdep_is_held(&rt->fib6_table->tb6_lock));
|
2018-01-09 14:40:25 +00:00
|
|
|
iter = rcu_dereference_protected(fn->leaf,
|
2018-04-18 22:38:59 +00:00
|
|
|
lockdep_is_held(&rt->fib6_table->tb6_lock));
|
2018-01-09 14:40:25 +00:00
|
|
|
while (iter) {
|
2018-04-18 22:38:59 +00:00
|
|
|
if (iter->fib6_metric == rt->fib6_metric &&
|
2018-07-03 21:36:21 +00:00
|
|
|
rt6_qualify_for_ecmp(iter))
|
2018-01-09 14:40:25 +00:00
|
|
|
return iter;
|
2018-05-04 20:54:24 +00:00
|
|
|
iter = rcu_dereference_protected(iter->fib6_next,
|
2018-04-18 22:38:59 +00:00
|
|
|
lockdep_is_held(&rt->fib6_table->tb6_lock));
|
2018-01-09 14:40:25 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
/* only called for fib entries with builtin fib6_nh */
|
2018-04-18 00:33:26 +00:00
|
|
|
static bool rt6_is_dead(const struct fib6_info *rt)
|
2018-01-09 14:40:25 +00:00
|
|
|
{
|
2019-05-23 03:27:59 +00:00
|
|
|
if (rt->fib6_nh->fib_nh_flags & RTNH_F_DEAD ||
|
|
|
|
(rt->fib6_nh->fib_nh_flags & RTNH_F_LINKDOWN &&
|
|
|
|
ip6_ignore_linkdown(rt->fib6_nh->fib_nh_dev)))
|
2018-01-09 14:40:25 +00:00
|
|
|
return true;
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static int rt6_multipath_total_weight(const struct fib6_info *rt)
|
2018-01-09 14:40:25 +00:00
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *iter;
|
2018-01-09 14:40:25 +00:00
|
|
|
int total = 0;
|
|
|
|
|
|
|
|
if (!rt6_is_dead(rt))
|
2019-05-23 03:27:59 +00:00
|
|
|
total += rt->fib6_nh->fib_nh_weight;
|
2018-01-09 14:40:25 +00:00
|
|
|
|
2018-04-18 22:38:59 +00:00
|
|
|
list_for_each_entry(iter, &rt->fib6_siblings, fib6_siblings) {
|
2018-01-09 14:40:25 +00:00
|
|
|
if (!rt6_is_dead(iter))
|
2019-05-23 03:27:59 +00:00
|
|
|
total += iter->fib6_nh->fib_nh_weight;
|
2018-01-09 14:40:25 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
return total;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static void rt6_upper_bound_set(struct fib6_info *rt, int *weight, int total)
|
2018-01-09 14:40:25 +00:00
|
|
|
{
|
|
|
|
int upper_bound = -1;
|
|
|
|
|
|
|
|
if (!rt6_is_dead(rt)) {
|
2019-05-23 03:27:59 +00:00
|
|
|
*weight += rt->fib6_nh->fib_nh_weight;
|
2018-01-09 14:40:25 +00:00
|
|
|
upper_bound = DIV_ROUND_CLOSEST_ULL((u64) (*weight) << 31,
|
|
|
|
total) - 1;
|
|
|
|
}
|
2019-05-23 03:27:59 +00:00
|
|
|
atomic_set(&rt->fib6_nh->fib_nh_upper_bound, upper_bound);
|
2018-01-09 14:40:25 +00:00
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static void rt6_multipath_upper_bound_set(struct fib6_info *rt, int total)
|
2018-01-09 14:40:25 +00:00
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *iter;
|
2018-01-09 14:40:25 +00:00
|
|
|
int weight = 0;
|
|
|
|
|
|
|
|
rt6_upper_bound_set(rt, &weight, total);
|
|
|
|
|
2018-04-18 22:38:59 +00:00
|
|
|
list_for_each_entry(iter, &rt->fib6_siblings, fib6_siblings)
|
2018-01-09 14:40:25 +00:00
|
|
|
rt6_upper_bound_set(iter, &weight, total);
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
void rt6_multipath_rebalance(struct fib6_info *rt)
|
2018-01-09 14:40:25 +00:00
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *first;
|
2018-01-09 14:40:25 +00:00
|
|
|
int total;
|
|
|
|
|
|
|
|
/* In case the entire multipath route was marked for flushing,
|
|
|
|
* then there is no need to rebalance upon the removal of every
|
|
|
|
* sibling route.
|
|
|
|
*/
|
2018-04-18 22:38:59 +00:00
|
|
|
if (!rt->fib6_nsiblings || rt->should_flush)
|
2018-01-09 14:40:25 +00:00
|
|
|
return;
|
|
|
|
|
|
|
|
/* During lookup routes are evaluated in order, so we need to
|
|
|
|
* make sure upper bounds are assigned from the first sibling
|
|
|
|
* onwards.
|
|
|
|
*/
|
|
|
|
first = rt6_multipath_first_sibling(rt);
|
|
|
|
if (WARN_ON_ONCE(!first))
|
|
|
|
return;
|
|
|
|
|
|
|
|
total = rt6_multipath_total_weight(first);
|
|
|
|
rt6_multipath_upper_bound_set(first, total);
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static int fib6_ifup(struct fib6_info *rt, void *p_arg)
|
2018-01-07 10:45:03 +00:00
|
|
|
{
|
|
|
|
const struct arg_netdev_event *arg = p_arg;
|
2018-04-18 00:33:10 +00:00
|
|
|
struct net *net = dev_net(arg->dev);
|
2018-01-07 10:45:03 +00:00
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
if (rt != net->ipv6.fib6_null_entry && !rt->nh &&
|
2019-05-23 03:27:59 +00:00
|
|
|
rt->fib6_nh->fib_nh_dev == arg->dev) {
|
|
|
|
rt->fib6_nh->fib_nh_flags &= ~arg->nh_flags;
|
2018-04-18 00:33:10 +00:00
|
|
|
fib6_update_sernum_upto_root(net, rt);
|
2018-01-09 14:40:25 +00:00
|
|
|
rt6_multipath_rebalance(rt);
|
2018-01-07 10:45:15 +00:00
|
|
|
}
|
2018-01-07 10:45:03 +00:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2019-04-23 15:48:09 +00:00
|
|
|
void rt6_sync_up(struct net_device *dev, unsigned char nh_flags)
|
2018-01-07 10:45:03 +00:00
|
|
|
{
|
|
|
|
struct arg_netdev_event arg = {
|
|
|
|
.dev = dev,
|
2018-01-12 20:07:36 +00:00
|
|
|
{
|
|
|
|
.nh_flags = nh_flags,
|
|
|
|
},
|
2018-01-07 10:45:03 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
if (nh_flags & RTNH_F_DEAD && netif_carrier_ok(dev))
|
|
|
|
arg.nh_flags |= RTNH_F_LINKDOWN;
|
|
|
|
|
|
|
|
fib6_clean_all(dev_net(dev), fib6_ifup, &arg);
|
|
|
|
}
|
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
/* only called for fib entries with inline fib6_nh */
|
2018-04-18 00:33:26 +00:00
|
|
|
static bool rt6_multipath_uses_dev(const struct fib6_info *rt,
|
2018-01-07 10:45:15 +00:00
|
|
|
const struct net_device *dev)
|
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *iter;
|
2018-01-07 10:45:15 +00:00
|
|
|
|
2019-05-23 03:27:59 +00:00
|
|
|
if (rt->fib6_nh->fib_nh_dev == dev)
|
2018-01-07 10:45:15 +00:00
|
|
|
return true;
|
2018-04-18 22:38:59 +00:00
|
|
|
list_for_each_entry(iter, &rt->fib6_siblings, fib6_siblings)
|
2019-05-23 03:27:59 +00:00
|
|
|
if (iter->fib6_nh->fib_nh_dev == dev)
|
2018-01-07 10:45:15 +00:00
|
|
|
return true;
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static void rt6_multipath_flush(struct fib6_info *rt)
|
2018-01-07 10:45:15 +00:00
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *iter;
|
2018-01-07 10:45:15 +00:00
|
|
|
|
|
|
|
rt->should_flush = 1;
|
2018-04-18 22:38:59 +00:00
|
|
|
list_for_each_entry(iter, &rt->fib6_siblings, fib6_siblings)
|
2018-01-07 10:45:15 +00:00
|
|
|
iter->should_flush = 1;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static unsigned int rt6_multipath_dead_count(const struct fib6_info *rt,
|
2018-01-07 10:45:15 +00:00
|
|
|
const struct net_device *down_dev)
|
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *iter;
|
2018-01-07 10:45:15 +00:00
|
|
|
unsigned int dead = 0;
|
|
|
|
|
2019-05-23 03:27:59 +00:00
|
|
|
if (rt->fib6_nh->fib_nh_dev == down_dev ||
|
|
|
|
rt->fib6_nh->fib_nh_flags & RTNH_F_DEAD)
|
2018-01-07 10:45:15 +00:00
|
|
|
dead++;
|
2018-04-18 22:38:59 +00:00
|
|
|
list_for_each_entry(iter, &rt->fib6_siblings, fib6_siblings)
|
2019-05-23 03:27:59 +00:00
|
|
|
if (iter->fib6_nh->fib_nh_dev == down_dev ||
|
|
|
|
iter->fib6_nh->fib_nh_flags & RTNH_F_DEAD)
|
2018-01-07 10:45:15 +00:00
|
|
|
dead++;
|
|
|
|
|
|
|
|
return dead;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static void rt6_multipath_nh_flags_set(struct fib6_info *rt,
|
2018-01-07 10:45:15 +00:00
|
|
|
const struct net_device *dev,
|
2019-04-23 15:48:09 +00:00
|
|
|
unsigned char nh_flags)
|
2018-01-07 10:45:15 +00:00
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *iter;
|
2018-01-07 10:45:15 +00:00
|
|
|
|
2019-05-23 03:27:59 +00:00
|
|
|
if (rt->fib6_nh->fib_nh_dev == dev)
|
|
|
|
rt->fib6_nh->fib_nh_flags |= nh_flags;
|
2018-04-18 22:38:59 +00:00
|
|
|
list_for_each_entry(iter, &rt->fib6_siblings, fib6_siblings)
|
2019-05-23 03:27:59 +00:00
|
|
|
if (iter->fib6_nh->fib_nh_dev == dev)
|
|
|
|
iter->fib6_nh->fib_nh_flags |= nh_flags;
|
2018-01-07 10:45:15 +00:00
|
|
|
}
|
|
|
|
|
2017-01-18 15:40:36 +00:00
|
|
|
/* called with write lock held for table with rt */
|
2018-04-18 00:33:26 +00:00
|
|
|
static int fib6_ifdown(struct fib6_info *rt, void *p_arg)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2018-01-07 10:45:04 +00:00
|
|
|
const struct arg_netdev_event *arg = p_arg;
|
|
|
|
const struct net_device *dev = arg->dev;
|
2018-04-18 00:33:10 +00:00
|
|
|
struct net *net = dev_net(dev);
|
2008-03-04 21:48:30 +00:00
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
if (rt == net->ipv6.fib6_null_entry || rt->nh)
|
2018-01-07 10:45:05 +00:00
|
|
|
return 0;
|
|
|
|
|
|
|
|
switch (arg->event) {
|
|
|
|
case NETDEV_UNREGISTER:
|
2019-05-23 03:27:59 +00:00
|
|
|
return rt->fib6_nh->fib_nh_dev == dev ? -1 : 0;
|
2018-01-07 10:45:05 +00:00
|
|
|
case NETDEV_DOWN:
|
2018-01-07 10:45:15 +00:00
|
|
|
if (rt->should_flush)
|
2018-01-07 10:45:05 +00:00
|
|
|
return -1;
|
2018-04-18 22:38:59 +00:00
|
|
|
if (!rt->fib6_nsiblings)
|
2019-05-23 03:27:59 +00:00
|
|
|
return rt->fib6_nh->fib_nh_dev == dev ? -1 : 0;
|
2018-01-07 10:45:15 +00:00
|
|
|
if (rt6_multipath_uses_dev(rt, dev)) {
|
|
|
|
unsigned int count;
|
|
|
|
|
|
|
|
count = rt6_multipath_dead_count(rt, dev);
|
2018-04-18 22:38:59 +00:00
|
|
|
if (rt->fib6_nsiblings + 1 == count) {
|
2018-01-07 10:45:15 +00:00
|
|
|
rt6_multipath_flush(rt);
|
|
|
|
return -1;
|
|
|
|
}
|
|
|
|
rt6_multipath_nh_flags_set(rt, dev, RTNH_F_DEAD |
|
|
|
|
RTNH_F_LINKDOWN);
|
2018-04-18 00:33:10 +00:00
|
|
|
fib6_update_sernum(net, rt);
|
2018-01-09 14:40:25 +00:00
|
|
|
rt6_multipath_rebalance(rt);
|
2018-01-07 10:45:15 +00:00
|
|
|
}
|
|
|
|
return -2;
|
2018-01-07 10:45:05 +00:00
|
|
|
case NETDEV_CHANGE:
|
2019-05-23 03:27:59 +00:00
|
|
|
if (rt->fib6_nh->fib_nh_dev != dev ||
|
2018-04-18 22:38:59 +00:00
|
|
|
rt->fib6_flags & (RTF_LOCAL | RTF_ANYCAST))
|
2018-01-07 10:45:05 +00:00
|
|
|
break;
|
2019-05-23 03:27:59 +00:00
|
|
|
rt->fib6_nh->fib_nh_flags |= RTNH_F_LINKDOWN;
|
2018-01-09 14:40:25 +00:00
|
|
|
rt6_multipath_rebalance(rt);
|
2018-01-07 10:45:05 +00:00
|
|
|
break;
|
2018-01-07 10:45:02 +00:00
|
|
|
}
|
2011-12-26 20:24:36 +00:00
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2018-01-07 10:45:05 +00:00
|
|
|
void rt6_sync_down_dev(struct net_device *dev, unsigned long event)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2018-01-07 10:45:04 +00:00
|
|
|
struct arg_netdev_event arg = {
|
2008-03-04 21:48:30 +00:00
|
|
|
.dev = dev,
|
2018-01-12 20:07:36 +00:00
|
|
|
{
|
|
|
|
.event = event,
|
|
|
|
},
|
2008-03-04 21:48:30 +00:00
|
|
|
};
|
2018-10-12 03:17:21 +00:00
|
|
|
struct net *net = dev_net(dev);
|
2008-03-04 21:48:30 +00:00
|
|
|
|
2018-10-12 03:17:21 +00:00
|
|
|
if (net->ipv6.sysctl.skip_notify_on_dev_down)
|
|
|
|
fib6_clean_all_skip_notify(net, fib6_ifdown, &arg);
|
|
|
|
else
|
|
|
|
fib6_clean_all(net, fib6_ifdown, &arg);
|
2018-01-07 10:45:04 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
void rt6_disable_ip(struct net_device *dev, unsigned long event)
|
|
|
|
{
|
|
|
|
rt6_sync_down_dev(dev, event);
|
|
|
|
rt6_uncached_list_flush_dev(dev_net(dev), dev);
|
|
|
|
neigh_ifdown(&nd_tbl, dev);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2012-04-15 05:58:06 +00:00
|
|
|
struct rt6_mtu_change_arg {
|
2005-04-16 22:20:36 +00:00
|
|
|
struct net_device *dev;
|
2012-04-15 05:58:06 +00:00
|
|
|
unsigned int mtu;
|
2019-05-23 03:27:57 +00:00
|
|
|
struct fib6_info *f6i;
|
2005-04-16 22:20:36 +00:00
|
|
|
};
|
|
|
|
|
2019-05-23 03:27:58 +00:00
|
|
|
static int fib6_nh_mtu_change(struct fib6_nh *nh, void *_arg)
|
2019-05-23 03:27:57 +00:00
|
|
|
{
|
|
|
|
struct rt6_mtu_change_arg *arg = (struct rt6_mtu_change_arg *)_arg;
|
2019-05-23 03:27:58 +00:00
|
|
|
struct fib6_info *f6i = arg->f6i;
|
2019-05-23 03:27:57 +00:00
|
|
|
|
|
|
|
/* For administrative MTU increase, there is no way to discover
|
|
|
|
* IPv6 PMTU increase, so PMTU increase should be updated here.
|
|
|
|
* Since RFC 1981 doesn't include administrative MTU increase
|
|
|
|
* update PMTU increase is a MUST. (i.e. jumbo frame)
|
|
|
|
*/
|
|
|
|
if (nh->fib_nh_dev == arg->dev) {
|
|
|
|
struct inet6_dev *idev = __in6_dev_get(arg->dev);
|
|
|
|
u32 mtu = f6i->fib6_pmtu;
|
|
|
|
|
|
|
|
if (mtu >= arg->mtu ||
|
|
|
|
(mtu < arg->mtu && mtu == idev->cnf.mtu6))
|
|
|
|
fib6_metric_set(f6i, RTAX_MTU, arg->mtu);
|
|
|
|
|
|
|
|
spin_lock_bh(&rt6_exception_lock);
|
2019-05-23 03:27:58 +00:00
|
|
|
rt6_exceptions_update_pmtu(idev, nh, arg->mtu);
|
2019-05-23 03:27:57 +00:00
|
|
|
spin_unlock_bh(&rt6_exception_lock);
|
|
|
|
}
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int rt6_mtu_change_route(struct fib6_info *f6i, void *p_arg)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
|
|
|
struct rt6_mtu_change_arg *arg = (struct rt6_mtu_change_arg *) p_arg;
|
|
|
|
struct inet6_dev *idev;
|
|
|
|
|
|
|
|
/* In IPv6 pmtu discovery is not optional,
|
|
|
|
so that RTAX_MTU lock cannot disable it.
|
|
|
|
We still use this lock to block changes
|
|
|
|
caused by addrconf/ndisc.
|
|
|
|
*/
|
|
|
|
|
|
|
|
idev = __in6_dev_get(arg->dev);
|
2011-12-03 23:02:47 +00:00
|
|
|
if (!idev)
|
2005-04-16 22:20:36 +00:00
|
|
|
return 0;
|
|
|
|
|
2019-05-23 03:27:57 +00:00
|
|
|
if (fib6_metric_locked(f6i, RTAX_MTU))
|
|
|
|
return 0;
|
2018-04-18 00:33:16 +00:00
|
|
|
|
2019-05-23 03:27:57 +00:00
|
|
|
arg->f6i = f6i;
|
2019-06-08 21:53:31 +00:00
|
|
|
if (f6i->nh) {
|
|
|
|
/* fib6_nh_mtu_change only returns 0, so this is safe */
|
|
|
|
return nexthop_for_each_fib6_nh(f6i->nh, fib6_nh_mtu_change,
|
|
|
|
arg);
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:27:59 +00:00
|
|
|
return fib6_nh_mtu_change(f6i->fib6_nh, arg);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2012-04-15 05:58:06 +00:00
|
|
|
void rt6_mtu_change(struct net_device *dev, unsigned int mtu)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2006-08-05 06:20:06 +00:00
|
|
|
struct rt6_mtu_change_arg arg = {
|
|
|
|
.dev = dev,
|
|
|
|
.mtu = mtu,
|
|
|
|
};
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2013-12-27 08:32:38 +00:00
|
|
|
fib6_clean_all(dev_net(dev), rt6_mtu_change_route, &arg);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2007-06-05 19:38:30 +00:00
|
|
|
static const struct nla_policy rtm_ipv6_policy[RTA_MAX+1] = {
|
2019-05-22 19:07:43 +00:00
|
|
|
[RTA_UNSPEC] = { .strict_start_type = RTA_DPORT + 1 },
|
2006-08-27 03:13:18 +00:00
|
|
|
[RTA_GATEWAY] = { .len = sizeof(struct in6_addr) },
|
2018-04-23 01:29:23 +00:00
|
|
|
[RTA_PREFSRC] = { .len = sizeof(struct in6_addr) },
|
2006-08-22 07:01:08 +00:00
|
|
|
[RTA_OIF] = { .type = NLA_U32 },
|
2006-08-22 07:01:47 +00:00
|
|
|
[RTA_IIF] = { .type = NLA_U32 },
|
2006-08-22 07:01:08 +00:00
|
|
|
[RTA_PRIORITY] = { .type = NLA_U32 },
|
|
|
|
[RTA_METRICS] = { .type = NLA_NESTED },
|
2012-10-22 03:42:09 +00:00
|
|
|
[RTA_MULTIPATH] = { .len = sizeof(struct rtnexthop) },
|
2015-03-11 14:39:21 +00:00
|
|
|
[RTA_PREF] = { .type = NLA_U8 },
|
2015-07-21 08:43:48 +00:00
|
|
|
[RTA_ENCAP_TYPE] = { .type = NLA_U16 },
|
|
|
|
[RTA_ENCAP] = { .type = NLA_NESTED },
|
2015-12-16 09:50:11 +00:00
|
|
|
[RTA_EXPIRES] = { .type = NLA_U32 },
|
2016-11-03 17:23:42 +00:00
|
|
|
[RTA_UID] = { .type = NLA_U32 },
|
2017-02-27 12:59:39 +00:00
|
|
|
[RTA_MARK] = { .type = NLA_U32 },
|
2018-04-23 01:29:23 +00:00
|
|
|
[RTA_TABLE] = { .type = NLA_U32 },
|
2018-05-22 21:03:28 +00:00
|
|
|
[RTA_IP_PROTO] = { .type = NLA_U8 },
|
|
|
|
[RTA_SPORT] = { .type = NLA_U16 },
|
|
|
|
[RTA_DPORT] = { .type = NLA_U16 },
|
2019-06-08 21:53:34 +00:00
|
|
|
[RTA_NH_ID] = { .type = NLA_U32 },
|
2006-08-22 07:01:08 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
static int rtm_to_fib6_config(struct sk_buff *skb, struct nlmsghdr *nlh,
|
2017-05-21 16:12:04 +00:00
|
|
|
struct fib6_config *cfg,
|
|
|
|
struct netlink_ext_ack *extack)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2006-08-22 07:01:08 +00:00
|
|
|
struct rtmsg *rtm;
|
|
|
|
struct nlattr *tb[RTA_MAX+1];
|
2015-03-11 14:39:21 +00:00
|
|
|
unsigned int pref;
|
2006-08-22 07:01:08 +00:00
|
|
|
int err;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
netlink: make validation more configurable for future strictness
We currently have two levels of strict validation:
1) liberal (default)
- undefined (type >= max) & NLA_UNSPEC attributes accepted
- attribute length >= expected accepted
- garbage at end of message accepted
2) strict (opt-in)
- NLA_UNSPEC attributes accepted
- attribute length >= expected accepted
Split out parsing strictness into four different options:
* TRAILING - check that there's no trailing data after parsing
attributes (in message or nested)
* MAXTYPE - reject attrs > max known type
* UNSPEC - reject attributes with NLA_UNSPEC policy entries
* STRICT_ATTRS - strictly validate attribute size
The default for future things should be *everything*.
The current *_strict() is a combination of TRAILING and MAXTYPE,
and is renamed to _deprecated_strict().
The current regular parsing has none of this, and is renamed to
*_parse_deprecated().
Additionally it allows us to selectively set one of the new flags
even on old policies. Notably, the UNSPEC flag could be useful in
this case, since it can be arranged (by filling in the policy) to
not be an incompatible userspace ABI change, but would then going
forward prevent forgetting attribute entries. Similar can apply
to the POLICY flag.
We end up with the following renames:
* nla_parse -> nla_parse_deprecated
* nla_parse_strict -> nla_parse_deprecated_strict
* nlmsg_parse -> nlmsg_parse_deprecated
* nlmsg_parse_strict -> nlmsg_parse_deprecated_strict
* nla_parse_nested -> nla_parse_nested_deprecated
* nla_validate_nested -> nla_validate_nested_deprecated
Using spatch, of course:
@@
expression TB, MAX, HEAD, LEN, POL, EXT;
@@
-nla_parse(TB, MAX, HEAD, LEN, POL, EXT)
+nla_parse_deprecated(TB, MAX, HEAD, LEN, POL, EXT)
@@
expression NLH, HDRLEN, TB, MAX, POL, EXT;
@@
-nlmsg_parse(NLH, HDRLEN, TB, MAX, POL, EXT)
+nlmsg_parse_deprecated(NLH, HDRLEN, TB, MAX, POL, EXT)
@@
expression NLH, HDRLEN, TB, MAX, POL, EXT;
@@
-nlmsg_parse_strict(NLH, HDRLEN, TB, MAX, POL, EXT)
+nlmsg_parse_deprecated_strict(NLH, HDRLEN, TB, MAX, POL, EXT)
@@
expression TB, MAX, NLA, POL, EXT;
@@
-nla_parse_nested(TB, MAX, NLA, POL, EXT)
+nla_parse_nested_deprecated(TB, MAX, NLA, POL, EXT)
@@
expression START, MAX, POL, EXT;
@@
-nla_validate_nested(START, MAX, POL, EXT)
+nla_validate_nested_deprecated(START, MAX, POL, EXT)
@@
expression NLH, HDRLEN, MAX, POL, EXT;
@@
-nlmsg_validate(NLH, HDRLEN, MAX, POL, EXT)
+nlmsg_validate_deprecated(NLH, HDRLEN, MAX, POL, EXT)
For this patch, don't actually add the strict, non-renamed versions
yet so that it breaks compile if I get it wrong.
Also, while at it, make nla_validate and nla_parse go down to a
common __nla_validate_parse() function to avoid code duplication.
Ultimately, this allows us to have very strict validation for every
new caller of nla_parse()/nlmsg_parse() etc as re-introduced in the
next patch, while existing things will continue to work as is.
In effect then, this adds fully strict validation for any new command.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-26 12:07:28 +00:00
|
|
|
err = nlmsg_parse_deprecated(nlh, sizeof(*rtm), tb, RTA_MAX,
|
|
|
|
rtm_ipv6_policy, extack);
|
2006-08-22 07:01:08 +00:00
|
|
|
if (err < 0)
|
|
|
|
goto errout;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2006-08-22 07:01:08 +00:00
|
|
|
err = -EINVAL;
|
|
|
|
rtm = nlmsg_data(nlh);
|
|
|
|
|
2018-09-30 06:44:53 +00:00
|
|
|
*cfg = (struct fib6_config){
|
|
|
|
.fc_table = rtm->rtm_table,
|
|
|
|
.fc_dst_len = rtm->rtm_dst_len,
|
|
|
|
.fc_src_len = rtm->rtm_src_len,
|
|
|
|
.fc_flags = RTF_UP,
|
|
|
|
.fc_protocol = rtm->rtm_protocol,
|
|
|
|
.fc_type = rtm->rtm_type,
|
|
|
|
|
|
|
|
.fc_nlinfo.portid = NETLINK_CB(skb).portid,
|
|
|
|
.fc_nlinfo.nlh = nlh,
|
|
|
|
.fc_nlinfo.nl_net = sock_net(skb->sk),
|
|
|
|
};
|
2006-08-22 07:01:08 +00:00
|
|
|
|
2012-09-05 02:12:42 +00:00
|
|
|
if (rtm->rtm_type == RTN_UNREACHABLE ||
|
|
|
|
rtm->rtm_type == RTN_BLACKHOLE ||
|
2012-09-06 05:53:35 +00:00
|
|
|
rtm->rtm_type == RTN_PROHIBIT ||
|
|
|
|
rtm->rtm_type == RTN_THROW)
|
2006-08-22 07:01:08 +00:00
|
|
|
cfg->fc_flags |= RTF_REJECT;
|
|
|
|
|
2010-09-27 00:07:02 +00:00
|
|
|
if (rtm->rtm_type == RTN_LOCAL)
|
|
|
|
cfg->fc_flags |= RTF_LOCAL;
|
|
|
|
|
2015-04-28 20:03:03 +00:00
|
|
|
if (rtm->rtm_flags & RTM_F_CLONED)
|
|
|
|
cfg->fc_flags |= RTF_CACHE;
|
|
|
|
|
2018-01-26 00:55:09 +00:00
|
|
|
cfg->fc_flags |= (rtm->rtm_flags & RTNH_F_ONLINK);
|
|
|
|
|
2019-06-08 21:53:34 +00:00
|
|
|
if (tb[RTA_NH_ID]) {
|
|
|
|
if (tb[RTA_GATEWAY] || tb[RTA_OIF] ||
|
|
|
|
tb[RTA_MULTIPATH] || tb[RTA_ENCAP]) {
|
|
|
|
NL_SET_ERR_MSG(extack,
|
|
|
|
"Nexthop specification and nexthop id are mutually exclusive");
|
|
|
|
goto errout;
|
|
|
|
}
|
|
|
|
cfg->fc_nh_id = nla_get_u32(tb[RTA_NH_ID]);
|
|
|
|
}
|
|
|
|
|
2006-08-22 07:01:08 +00:00
|
|
|
if (tb[RTA_GATEWAY]) {
|
2015-03-29 14:59:26 +00:00
|
|
|
cfg->fc_gateway = nla_get_in6_addr(tb[RTA_GATEWAY]);
|
2006-08-22 07:01:08 +00:00
|
|
|
cfg->fc_flags |= RTF_GATEWAY;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2019-02-26 17:00:03 +00:00
|
|
|
if (tb[RTA_VIA]) {
|
|
|
|
NL_SET_ERR_MSG(extack, "IPv6 does not support RTA_VIA attribute");
|
|
|
|
goto errout;
|
|
|
|
}
|
2006-08-22 07:01:08 +00:00
|
|
|
|
|
|
|
if (tb[RTA_DST]) {
|
|
|
|
int plen = (rtm->rtm_dst_len + 7) >> 3;
|
|
|
|
|
|
|
|
if (nla_len(tb[RTA_DST]) < plen)
|
|
|
|
goto errout;
|
|
|
|
|
|
|
|
nla_memcpy(&cfg->fc_dst, tb[RTA_DST], plen);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2006-08-22 07:01:08 +00:00
|
|
|
|
|
|
|
if (tb[RTA_SRC]) {
|
|
|
|
int plen = (rtm->rtm_src_len + 7) >> 3;
|
|
|
|
|
|
|
|
if (nla_len(tb[RTA_SRC]) < plen)
|
|
|
|
goto errout;
|
|
|
|
|
|
|
|
nla_memcpy(&cfg->fc_src, tb[RTA_SRC], plen);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2006-08-22 07:01:08 +00:00
|
|
|
|
2011-04-13 21:10:57 +00:00
|
|
|
if (tb[RTA_PREFSRC])
|
2015-03-29 14:59:26 +00:00
|
|
|
cfg->fc_prefsrc = nla_get_in6_addr(tb[RTA_PREFSRC]);
|
2011-04-13 21:10:57 +00:00
|
|
|
|
2006-08-22 07:01:08 +00:00
|
|
|
if (tb[RTA_OIF])
|
|
|
|
cfg->fc_ifindex = nla_get_u32(tb[RTA_OIF]);
|
|
|
|
|
|
|
|
if (tb[RTA_PRIORITY])
|
|
|
|
cfg->fc_metric = nla_get_u32(tb[RTA_PRIORITY]);
|
|
|
|
|
|
|
|
if (tb[RTA_METRICS]) {
|
|
|
|
cfg->fc_mx = nla_data(tb[RTA_METRICS]);
|
|
|
|
cfg->fc_mx_len = nla_len(tb[RTA_METRICS]);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2006-08-22 07:01:08 +00:00
|
|
|
|
|
|
|
if (tb[RTA_TABLE])
|
|
|
|
cfg->fc_table = nla_get_u32(tb[RTA_TABLE]);
|
|
|
|
|
2012-10-22 03:42:09 +00:00
|
|
|
if (tb[RTA_MULTIPATH]) {
|
|
|
|
cfg->fc_mp = nla_data(tb[RTA_MULTIPATH]);
|
|
|
|
cfg->fc_mp_len = nla_len(tb[RTA_MULTIPATH]);
|
2017-01-17 22:57:36 +00:00
|
|
|
|
|
|
|
err = lwtunnel_valid_encap_type_attr(cfg->fc_mp,
|
2017-05-27 22:19:27 +00:00
|
|
|
cfg->fc_mp_len, extack);
|
2017-01-17 22:57:36 +00:00
|
|
|
if (err < 0)
|
|
|
|
goto errout;
|
2012-10-22 03:42:09 +00:00
|
|
|
}
|
|
|
|
|
2015-03-11 14:39:21 +00:00
|
|
|
if (tb[RTA_PREF]) {
|
|
|
|
pref = nla_get_u8(tb[RTA_PREF]);
|
|
|
|
if (pref != ICMPV6_ROUTER_PREF_LOW &&
|
|
|
|
pref != ICMPV6_ROUTER_PREF_HIGH)
|
|
|
|
pref = ICMPV6_ROUTER_PREF_MEDIUM;
|
|
|
|
cfg->fc_flags |= RTF_PREF(pref);
|
|
|
|
}
|
|
|
|
|
2015-07-21 08:43:48 +00:00
|
|
|
if (tb[RTA_ENCAP])
|
|
|
|
cfg->fc_encap = tb[RTA_ENCAP];
|
|
|
|
|
2017-01-17 22:57:36 +00:00
|
|
|
if (tb[RTA_ENCAP_TYPE]) {
|
2015-07-21 08:43:48 +00:00
|
|
|
cfg->fc_encap_type = nla_get_u16(tb[RTA_ENCAP_TYPE]);
|
|
|
|
|
2017-05-27 22:19:27 +00:00
|
|
|
err = lwtunnel_valid_encap_type(cfg->fc_encap_type, extack);
|
2017-01-17 22:57:36 +00:00
|
|
|
if (err < 0)
|
|
|
|
goto errout;
|
|
|
|
}
|
|
|
|
|
2015-12-16 09:50:11 +00:00
|
|
|
if (tb[RTA_EXPIRES]) {
|
|
|
|
unsigned long timeout = addrconf_timeout_fixup(nla_get_u32(tb[RTA_EXPIRES]), HZ);
|
|
|
|
|
|
|
|
if (addrconf_finite_timeout(timeout)) {
|
|
|
|
cfg->fc_expires = jiffies_to_clock_t(timeout * HZ);
|
|
|
|
cfg->fc_flags |= RTF_EXPIRES;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2006-08-22 07:01:08 +00:00
|
|
|
err = 0;
|
|
|
|
errout:
|
|
|
|
return err;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2015-09-08 17:53:04 +00:00
|
|
|
struct rt6_nh {
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *fib6_info;
|
2015-09-08 17:53:04 +00:00
|
|
|
struct fib6_config r_cfg;
|
|
|
|
struct list_head next;
|
|
|
|
};
|
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
static int ip6_route_info_append(struct net *net,
|
|
|
|
struct list_head *rt6_nh_list,
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt,
|
|
|
|
struct fib6_config *r_cfg)
|
2015-09-08 17:53:04 +00:00
|
|
|
{
|
|
|
|
struct rt6_nh *nh;
|
|
|
|
int err = -EEXIST;
|
|
|
|
|
|
|
|
list_for_each_entry(nh, rt6_nh_list, next) {
|
2018-04-18 00:33:26 +00:00
|
|
|
/* check if fib6_info already exists */
|
|
|
|
if (rt6_duplicate_nexthop(nh->fib6_info, rt))
|
2015-09-08 17:53:04 +00:00
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
|
|
|
nh = kzalloc(sizeof(*nh), GFP_KERNEL);
|
|
|
|
if (!nh)
|
|
|
|
return -ENOMEM;
|
2018-04-18 00:33:26 +00:00
|
|
|
nh->fib6_info = rt;
|
2015-09-08 17:53:04 +00:00
|
|
|
memcpy(&nh->r_cfg, r_cfg, sizeof(*r_cfg));
|
|
|
|
list_add_tail(&nh->next, rt6_nh_list);
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
static void ip6_route_mpath_notify(struct fib6_info *rt,
|
|
|
|
struct fib6_info *rt_last,
|
2017-02-02 20:37:10 +00:00
|
|
|
struct nl_info *info,
|
|
|
|
__u16 nlflags)
|
|
|
|
{
|
|
|
|
/* if this is an APPEND route, then rt points to the first route
|
|
|
|
* inserted and rt_last points to last route inserted. Userspace
|
|
|
|
* wants a consistent dump of the route which starts at the first
|
|
|
|
* nexthop. Since sibling routes are always added at the end of
|
|
|
|
* the list, find the first sibling of the last route appended
|
|
|
|
*/
|
2018-04-18 22:38:59 +00:00
|
|
|
if ((nlflags & NLM_F_APPEND) && rt_last && rt_last->fib6_nsiblings) {
|
|
|
|
rt = list_first_entry(&rt_last->fib6_siblings,
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info,
|
2018-04-18 22:38:59 +00:00
|
|
|
fib6_siblings);
|
2017-02-02 20:37:10 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if (rt)
|
|
|
|
inet6_rt_notify(RTM_NEWROUTE, rt, info, nlflags);
|
|
|
|
}
|
|
|
|
|
2019-12-23 13:28:15 +00:00
|
|
|
static bool ip6_route_mpath_should_notify(const struct fib6_info *rt)
|
|
|
|
{
|
|
|
|
bool rt_can_ecmp = rt6_qualify_for_ecmp(rt);
|
|
|
|
bool should_notify = false;
|
|
|
|
struct fib6_info *leaf;
|
|
|
|
struct fib6_node *fn;
|
|
|
|
|
|
|
|
rcu_read_lock();
|
|
|
|
fn = rcu_dereference(rt->fib6_node);
|
|
|
|
if (!fn)
|
|
|
|
goto out;
|
|
|
|
|
|
|
|
leaf = rcu_dereference(fn->leaf);
|
|
|
|
if (!leaf)
|
|
|
|
goto out;
|
|
|
|
|
|
|
|
if (rt == leaf ||
|
|
|
|
(rt_can_ecmp && rt->fib6_metric == leaf->fib6_metric &&
|
|
|
|
rt6_qualify_for_ecmp(leaf)))
|
|
|
|
should_notify = true;
|
|
|
|
out:
|
|
|
|
rcu_read_unlock();
|
|
|
|
|
|
|
|
return should_notify;
|
|
|
|
}
|
|
|
|
|
2021-12-31 00:36:33 +00:00
|
|
|
static int fib6_gw_from_attr(struct in6_addr *gw, struct nlattr *nla,
|
|
|
|
struct netlink_ext_ack *extack)
|
|
|
|
{
|
|
|
|
if (nla_len(nla) < sizeof(*gw)) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Invalid IPv6 address in RTA_GATEWAY");
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
|
|
|
*gw = nla_get_in6_addr(nla);
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2017-05-21 16:12:04 +00:00
|
|
|
static int ip6_route_multipath_add(struct fib6_config *cfg,
|
|
|
|
struct netlink_ext_ack *extack)
|
2012-10-22 03:42:09 +00:00
|
|
|
{
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt_notif = NULL, *rt_last = NULL;
|
2017-02-02 20:37:10 +00:00
|
|
|
struct nl_info *info = &cfg->fc_nlinfo;
|
2012-10-22 03:42:09 +00:00
|
|
|
struct fib6_config r_cfg;
|
|
|
|
struct rtnexthop *rtnh;
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt;
|
2015-09-08 17:53:04 +00:00
|
|
|
struct rt6_nh *err_nh;
|
|
|
|
struct rt6_nh *nh, *nh_safe;
|
2017-02-02 20:37:10 +00:00
|
|
|
__u16 nlflags;
|
2012-10-22 03:42:09 +00:00
|
|
|
int remaining;
|
|
|
|
int attrlen;
|
2015-09-08 17:53:04 +00:00
|
|
|
int err = 1;
|
|
|
|
int nhn = 0;
|
|
|
|
int replace = (cfg->fc_nlinfo.nlh &&
|
|
|
|
(cfg->fc_nlinfo.nlh->nlmsg_flags & NLM_F_REPLACE));
|
|
|
|
LIST_HEAD(rt6_nh_list);
|
2012-10-22 03:42:09 +00:00
|
|
|
|
2017-02-02 20:37:10 +00:00
|
|
|
nlflags = replace ? NLM_F_REPLACE : NLM_F_CREATE;
|
|
|
|
if (info->nlh && info->nlh->nlmsg_flags & NLM_F_APPEND)
|
|
|
|
nlflags |= NLM_F_APPEND;
|
|
|
|
|
2015-05-18 18:53:55 +00:00
|
|
|
remaining = cfg->fc_mp_len;
|
2012-10-22 03:42:09 +00:00
|
|
|
rtnh = (struct rtnexthop *)cfg->fc_mp;
|
|
|
|
|
2015-09-08 17:53:04 +00:00
|
|
|
/* Parse a Multipath Entry and build a list (rt6_nh_list) of
|
2018-04-18 00:33:26 +00:00
|
|
|
* fib6_info structs per nexthop
|
2015-09-08 17:53:04 +00:00
|
|
|
*/
|
2012-10-22 03:42:09 +00:00
|
|
|
while (rtnh_ok(rtnh, remaining)) {
|
|
|
|
memcpy(&r_cfg, cfg, sizeof(*cfg));
|
|
|
|
if (rtnh->rtnh_ifindex)
|
|
|
|
r_cfg.fc_ifindex = rtnh->rtnh_ifindex;
|
|
|
|
|
|
|
|
attrlen = rtnh_attrlen(rtnh);
|
|
|
|
if (attrlen > 0) {
|
|
|
|
struct nlattr *nla, *attrs = rtnh_attrs(rtnh);
|
|
|
|
|
|
|
|
nla = nla_find(attrs, attrlen, RTA_GATEWAY);
|
|
|
|
if (nla) {
|
2022-01-03 17:05:55 +00:00
|
|
|
err = fib6_gw_from_attr(&r_cfg.fc_gateway, nla,
|
2021-12-31 00:36:33 +00:00
|
|
|
extack);
|
2022-01-03 17:05:55 +00:00
|
|
|
if (err)
|
|
|
|
goto cleanup;
|
2021-12-31 00:36:33 +00:00
|
|
|
|
2012-10-22 03:42:09 +00:00
|
|
|
r_cfg.fc_flags |= RTF_GATEWAY;
|
|
|
|
}
|
2015-07-21 08:43:48 +00:00
|
|
|
r_cfg.fc_encap = nla_find(attrs, attrlen, RTA_ENCAP);
|
2021-12-31 00:36:35 +00:00
|
|
|
|
|
|
|
/* RTA_ENCAP_TYPE length checked in
|
|
|
|
* lwtunnel_valid_encap_type_attr
|
|
|
|
*/
|
2015-07-21 08:43:48 +00:00
|
|
|
nla = nla_find(attrs, attrlen, RTA_ENCAP_TYPE);
|
|
|
|
if (nla)
|
|
|
|
r_cfg.fc_encap_type = nla_get_u16(nla);
|
2012-10-22 03:42:09 +00:00
|
|
|
}
|
2015-09-08 17:53:04 +00:00
|
|
|
|
2018-03-20 17:06:59 +00:00
|
|
|
r_cfg.fc_flags |= (rtnh->rtnh_flags & RTNH_F_ONLINK);
|
2018-04-18 00:33:22 +00:00
|
|
|
rt = ip6_route_info_create(&r_cfg, GFP_KERNEL, extack);
|
2015-10-10 15:26:36 +00:00
|
|
|
if (IS_ERR(rt)) {
|
|
|
|
err = PTR_ERR(rt);
|
|
|
|
rt = NULL;
|
2015-09-08 17:53:04 +00:00
|
|
|
goto cleanup;
|
2015-10-10 15:26:36 +00:00
|
|
|
}
|
2018-07-15 16:35:19 +00:00
|
|
|
if (!rt6_qualify_for_ecmp(rt)) {
|
|
|
|
err = -EINVAL;
|
|
|
|
NL_SET_ERR_MSG(extack,
|
|
|
|
"Device only routes can not be added for IPv6 using the multipath API.");
|
|
|
|
fib6_info_release(rt);
|
|
|
|
goto cleanup;
|
|
|
|
}
|
2015-09-08 17:53:04 +00:00
|
|
|
|
2019-05-23 03:27:59 +00:00
|
|
|
rt->fib6_nh->fib_nh_weight = rtnh->rtnh_hops + 1;
|
2018-01-09 14:40:28 +00:00
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
err = ip6_route_info_append(info->nl_net, &rt6_nh_list,
|
|
|
|
rt, &r_cfg);
|
2012-10-22 03:42:09 +00:00
|
|
|
if (err) {
|
2018-04-18 00:33:25 +00:00
|
|
|
fib6_info_release(rt);
|
2015-09-08 17:53:04 +00:00
|
|
|
goto cleanup;
|
|
|
|
}
|
|
|
|
|
|
|
|
rtnh = rtnh_next(rtnh, &remaining);
|
|
|
|
}
|
|
|
|
|
2019-06-20 09:10:21 +00:00
|
|
|
if (list_empty(&rt6_nh_list)) {
|
|
|
|
NL_SET_ERR_MSG(extack,
|
|
|
|
"Invalid nexthop configuration - no valid nexthops");
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
2017-02-02 20:37:10 +00:00
|
|
|
/* for add and replace send one notification with all nexthops.
|
|
|
|
* Skip the notification in fib6_add_rt2node and send one with
|
|
|
|
* the full route when done
|
|
|
|
*/
|
|
|
|
info->skip_notify = 1;
|
|
|
|
|
2019-06-18 15:12:48 +00:00
|
|
|
/* For add and replace, send one notification with all nexthops. For
|
|
|
|
* append, send one notification with all appended nexthops.
|
|
|
|
*/
|
|
|
|
info->skip_notify_kernel = 1;
|
|
|
|
|
2015-09-08 17:53:04 +00:00
|
|
|
err_nh = NULL;
|
|
|
|
list_for_each_entry(nh, &rt6_nh_list, next) {
|
2018-04-18 00:33:26 +00:00
|
|
|
err = __ip6_ins_rt(nh->fib6_info, info, extack);
|
|
|
|
fib6_info_release(nh->fib6_info);
|
2018-04-18 00:33:25 +00:00
|
|
|
|
2018-06-04 20:41:42 +00:00
|
|
|
if (!err) {
|
|
|
|
/* save reference to last route successfully inserted */
|
|
|
|
rt_last = nh->fib6_info;
|
|
|
|
|
|
|
|
/* save reference to first route for notification */
|
|
|
|
if (!rt_notif)
|
|
|
|
rt_notif = nh->fib6_info;
|
|
|
|
}
|
2017-02-02 20:37:10 +00:00
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
/* nh->fib6_info is used or freed at this point, reset to NULL*/
|
|
|
|
nh->fib6_info = NULL;
|
2015-09-08 17:53:04 +00:00
|
|
|
if (err) {
|
|
|
|
if (replace && nhn)
|
2019-01-14 18:52:45 +00:00
|
|
|
NL_SET_ERR_MSG_MOD(extack,
|
|
|
|
"multipath route replace failed (check consistency of installed routes)");
|
2015-09-08 17:53:04 +00:00
|
|
|
err_nh = nh;
|
|
|
|
goto add_errout;
|
2012-10-22 03:42:09 +00:00
|
|
|
}
|
2015-09-08 17:53:04 +00:00
|
|
|
|
2012-11-01 22:58:22 +00:00
|
|
|
/* Because each route is added like a single route we remove
|
2015-05-18 18:54:00 +00:00
|
|
|
* these flags after the first nexthop: if there is a collision,
|
|
|
|
* we have already failed to add the first nexthop:
|
|
|
|
* fib6_add_rt2node() has rejected it; when replacing, old
|
|
|
|
* nexthops have been replaced by first new, the rest should
|
|
|
|
* be added to it.
|
2012-11-01 22:58:22 +00:00
|
|
|
*/
|
2021-04-08 22:01:29 +00:00
|
|
|
if (cfg->fc_nlinfo.nlh) {
|
|
|
|
cfg->fc_nlinfo.nlh->nlmsg_flags &= ~(NLM_F_EXCL |
|
|
|
|
NLM_F_REPLACE);
|
|
|
|
cfg->fc_nlinfo.nlh->nlmsg_flags |= NLM_F_CREATE;
|
|
|
|
}
|
2015-09-08 17:53:04 +00:00
|
|
|
nhn++;
|
|
|
|
}
|
|
|
|
|
2019-12-23 13:28:15 +00:00
|
|
|
/* An in-kernel notification should only be sent in case the new
|
|
|
|
* multipath route is added as the first route in the node, or if
|
|
|
|
* it was appended to it. We pass 'rt_notif' since it is the first
|
|
|
|
* sibling and might allow us to skip some checks in the replace case.
|
|
|
|
*/
|
|
|
|
if (ip6_route_mpath_should_notify(rt_notif)) {
|
|
|
|
enum fib_event_type fib_event;
|
|
|
|
|
|
|
|
if (rt_notif->fib6_nsiblings != nhn - 1)
|
|
|
|
fib_event = FIB_EVENT_ENTRY_APPEND;
|
|
|
|
else
|
2019-12-23 13:28:20 +00:00
|
|
|
fib_event = FIB_EVENT_ENTRY_REPLACE;
|
2019-12-23 13:28:15 +00:00
|
|
|
|
|
|
|
err = call_fib6_multipath_entry_notifiers(info->nl_net,
|
|
|
|
fib_event, rt_notif,
|
|
|
|
nhn - 1, extack);
|
|
|
|
if (err) {
|
|
|
|
/* Delete all the siblings that were just added */
|
|
|
|
err_nh = NULL;
|
|
|
|
goto add_errout;
|
|
|
|
}
|
|
|
|
}
|
2019-06-18 15:12:48 +00:00
|
|
|
|
2017-02-02 20:37:10 +00:00
|
|
|
/* success ... tell user about new route */
|
|
|
|
ip6_route_mpath_notify(rt_notif, rt_last, info, nlflags);
|
2015-09-08 17:53:04 +00:00
|
|
|
goto cleanup;
|
|
|
|
|
|
|
|
add_errout:
|
2017-02-02 20:37:10 +00:00
|
|
|
/* send notification for routes that were added so that
|
|
|
|
* the delete notifications sent by ip6_route_del are
|
|
|
|
* coherent
|
|
|
|
*/
|
|
|
|
if (rt_notif)
|
|
|
|
ip6_route_mpath_notify(rt_notif, rt_last, info, nlflags);
|
|
|
|
|
2015-09-08 17:53:04 +00:00
|
|
|
/* Delete routes that were already added */
|
|
|
|
list_for_each_entry(nh, &rt6_nh_list, next) {
|
|
|
|
if (err_nh == nh)
|
|
|
|
break;
|
2017-05-21 16:12:04 +00:00
|
|
|
ip6_route_del(&nh->r_cfg, extack);
|
2015-09-08 17:53:04 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
cleanup:
|
|
|
|
list_for_each_entry_safe(nh, nh_safe, &rt6_nh_list, next) {
|
2018-04-18 00:33:26 +00:00
|
|
|
if (nh->fib6_info)
|
|
|
|
fib6_info_release(nh->fib6_info);
|
2015-09-08 17:53:04 +00:00
|
|
|
list_del(&nh->next);
|
|
|
|
kfree(nh);
|
|
|
|
}
|
|
|
|
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2017-05-21 16:12:04 +00:00
|
|
|
static int ip6_route_multipath_del(struct fib6_config *cfg,
|
|
|
|
struct netlink_ext_ack *extack)
|
2015-09-08 17:53:04 +00:00
|
|
|
{
|
|
|
|
struct fib6_config r_cfg;
|
|
|
|
struct rtnexthop *rtnh;
|
2020-09-11 10:35:09 +00:00
|
|
|
int last_err = 0;
|
2015-09-08 17:53:04 +00:00
|
|
|
int remaining;
|
|
|
|
int attrlen;
|
2020-09-11 10:35:09 +00:00
|
|
|
int err;
|
2015-09-08 17:53:04 +00:00
|
|
|
|
|
|
|
remaining = cfg->fc_mp_len;
|
|
|
|
rtnh = (struct rtnexthop *)cfg->fc_mp;
|
|
|
|
|
|
|
|
/* Parse a Multipath Entry */
|
|
|
|
while (rtnh_ok(rtnh, remaining)) {
|
|
|
|
memcpy(&r_cfg, cfg, sizeof(*cfg));
|
|
|
|
if (rtnh->rtnh_ifindex)
|
|
|
|
r_cfg.fc_ifindex = rtnh->rtnh_ifindex;
|
|
|
|
|
|
|
|
attrlen = rtnh_attrlen(rtnh);
|
|
|
|
if (attrlen > 0) {
|
|
|
|
struct nlattr *nla, *attrs = rtnh_attrs(rtnh);
|
|
|
|
|
|
|
|
nla = nla_find(attrs, attrlen, RTA_GATEWAY);
|
|
|
|
if (nla) {
|
2021-12-31 00:36:34 +00:00
|
|
|
err = fib6_gw_from_attr(&r_cfg.fc_gateway, nla,
|
|
|
|
extack);
|
2022-01-03 17:19:11 +00:00
|
|
|
if (err) {
|
|
|
|
last_err = err;
|
|
|
|
goto next_rtnh;
|
|
|
|
}
|
2021-12-31 00:36:34 +00:00
|
|
|
|
2015-09-08 17:53:04 +00:00
|
|
|
r_cfg.fc_flags |= RTF_GATEWAY;
|
|
|
|
}
|
|
|
|
}
|
2017-05-21 16:12:04 +00:00
|
|
|
err = ip6_route_del(&r_cfg, extack);
|
2015-09-08 17:53:04 +00:00
|
|
|
if (err)
|
|
|
|
last_err = err;
|
|
|
|
|
2022-01-03 17:19:11 +00:00
|
|
|
next_rtnh:
|
2012-10-22 03:42:09 +00:00
|
|
|
rtnh = rtnh_next(rtnh, &remaining);
|
|
|
|
}
|
|
|
|
|
|
|
|
return last_err;
|
|
|
|
}
|
|
|
|
|
2017-04-16 16:48:24 +00:00
|
|
|
static int inet6_rtm_delroute(struct sk_buff *skb, struct nlmsghdr *nlh,
|
|
|
|
struct netlink_ext_ack *extack)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2006-08-22 07:01:08 +00:00
|
|
|
struct fib6_config cfg;
|
|
|
|
int err;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2017-05-21 16:12:04 +00:00
|
|
|
err = rtm_to_fib6_config(skb, nlh, &cfg, extack);
|
2006-08-22 07:01:08 +00:00
|
|
|
if (err < 0)
|
|
|
|
return err;
|
|
|
|
|
2019-06-08 21:53:34 +00:00
|
|
|
if (cfg.fc_nh_id &&
|
|
|
|
!nexthop_find_by_id(sock_net(skb->sk), cfg.fc_nh_id)) {
|
|
|
|
NL_SET_ERR_MSG(extack, "Nexthop id does not exist");
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
2012-10-22 03:42:09 +00:00
|
|
|
if (cfg.fc_mp)
|
2017-05-21 16:12:04 +00:00
|
|
|
return ip6_route_multipath_del(&cfg, extack);
|
2017-02-02 20:37:08 +00:00
|
|
|
else {
|
|
|
|
cfg.fc_delete_all_nh = 1;
|
2017-05-21 16:12:04 +00:00
|
|
|
return ip6_route_del(&cfg, extack);
|
2017-02-02 20:37:08 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2017-04-16 16:48:24 +00:00
|
|
|
static int inet6_rtm_newroute(struct sk_buff *skb, struct nlmsghdr *nlh,
|
|
|
|
struct netlink_ext_ack *extack)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2006-08-22 07:01:08 +00:00
|
|
|
struct fib6_config cfg;
|
|
|
|
int err;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2017-05-21 16:12:04 +00:00
|
|
|
err = rtm_to_fib6_config(skb, nlh, &cfg, extack);
|
2006-08-22 07:01:08 +00:00
|
|
|
if (err < 0)
|
|
|
|
return err;
|
|
|
|
|
2019-03-21 12:21:34 +00:00
|
|
|
if (cfg.fc_metric == 0)
|
|
|
|
cfg.fc_metric = IP6_RT_PRIO_USER;
|
|
|
|
|
2012-10-22 03:42:09 +00:00
|
|
|
if (cfg.fc_mp)
|
2017-05-21 16:12:04 +00:00
|
|
|
return ip6_route_multipath_add(&cfg, extack);
|
2012-10-22 03:42:09 +00:00
|
|
|
else
|
2018-04-18 00:33:22 +00:00
|
|
|
return ip6_route_add(&cfg, GFP_KERNEL, extack);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2019-06-08 21:53:26 +00:00
|
|
|
/* add the overhead of this fib6_nh to nexthop_len */
|
|
|
|
static int rt6_nh_nlmsg_size(struct fib6_nh *nh, void *arg)
|
2006-11-10 22:10:15 +00:00
|
|
|
{
|
2019-06-08 21:53:26 +00:00
|
|
|
int *nexthop_len = arg;
|
2017-02-02 20:37:09 +00:00
|
|
|
|
2019-06-08 21:53:26 +00:00
|
|
|
*nexthop_len += nla_total_size(0) /* RTA_MULTIPATH */
|
|
|
|
+ NLA_ALIGN(sizeof(struct rtnexthop))
|
|
|
|
+ nla_total_size(16); /* RTA_GATEWAY */
|
|
|
|
|
|
|
|
if (nh->fib_nh_lws) {
|
|
|
|
/* RTA_ENCAP_TYPE */
|
|
|
|
*nexthop_len += lwtunnel_get_encap_size(nh->fib_nh_lws);
|
|
|
|
/* RTA_ENCAP */
|
|
|
|
*nexthop_len += nla_total_size(2);
|
|
|
|
}
|
2019-06-04 03:19:52 +00:00
|
|
|
|
2019-06-08 21:53:26 +00:00
|
|
|
return 0;
|
|
|
|
}
|
2017-02-02 20:37:09 +00:00
|
|
|
|
2019-06-08 21:53:26 +00:00
|
|
|
static size_t rt6_nlmsg_size(struct fib6_info *f6i)
|
|
|
|
{
|
|
|
|
int nexthop_len;
|
|
|
|
|
|
|
|
if (f6i->nh) {
|
|
|
|
nexthop_len = nla_total_size(4); /* RTA_NH_ID */
|
|
|
|
nexthop_for_each_fib6_nh(f6i->nh, rt6_nh_nlmsg_size,
|
|
|
|
&nexthop_len);
|
|
|
|
} else {
|
|
|
|
struct fib6_nh *nh = f6i->fib6_nh;
|
|
|
|
|
|
|
|
nexthop_len = 0;
|
|
|
|
if (f6i->fib6_nsiblings) {
|
|
|
|
nexthop_len = nla_total_size(0) /* RTA_MULTIPATH */
|
|
|
|
+ NLA_ALIGN(sizeof(struct rtnexthop))
|
|
|
|
+ nla_total_size(16) /* RTA_GATEWAY */
|
|
|
|
+ lwtunnel_get_encap_size(nh->fib_nh_lws);
|
|
|
|
|
|
|
|
nexthop_len *= f6i->fib6_nsiblings;
|
|
|
|
}
|
|
|
|
nexthop_len += lwtunnel_get_encap_size(nh->fib_nh_lws);
|
2017-02-02 20:37:09 +00:00
|
|
|
}
|
|
|
|
|
2006-11-10 22:10:15 +00:00
|
|
|
return NLMSG_ALIGN(sizeof(struct rtmsg))
|
|
|
|
+ nla_total_size(16) /* RTA_SRC */
|
|
|
|
+ nla_total_size(16) /* RTA_DST */
|
|
|
|
+ nla_total_size(16) /* RTA_GATEWAY */
|
|
|
|
+ nla_total_size(16) /* RTA_PREFSRC */
|
|
|
|
+ nla_total_size(4) /* RTA_TABLE */
|
|
|
|
+ nla_total_size(4) /* RTA_IIF */
|
|
|
|
+ nla_total_size(4) /* RTA_OIF */
|
|
|
|
+ nla_total_size(4) /* RTA_PRIORITY */
|
2007-01-24 06:09:41 +00:00
|
|
|
+ RTAX_MAX * nla_total_size(4) /* RTA_METRICS */
|
2015-01-05 22:57:47 +00:00
|
|
|
+ nla_total_size(sizeof(struct rta_cacheinfo))
|
2015-03-11 14:39:21 +00:00
|
|
|
+ nla_total_size(TCP_CA_NAME_MAX) /* RTAX_CC_ALGO */
|
2015-07-21 08:43:48 +00:00
|
|
|
+ nla_total_size(1) /* RTA_PREF */
|
2017-02-02 20:37:09 +00:00
|
|
|
+ nexthop_len;
|
|
|
|
}
|
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
static int rt6_fill_node_nexthop(struct sk_buff *skb, struct nexthop *nh,
|
|
|
|
unsigned char *flags)
|
|
|
|
{
|
|
|
|
if (nexthop_is_multipath(nh)) {
|
|
|
|
struct nlattr *mp;
|
|
|
|
|
2019-09-03 22:22:12 +00:00
|
|
|
mp = nla_nest_start_noflag(skb, RTA_MULTIPATH);
|
2019-06-04 03:19:52 +00:00
|
|
|
if (!mp)
|
|
|
|
goto nla_put_failure;
|
|
|
|
|
2019-09-04 14:11:58 +00:00
|
|
|
if (nexthop_mpath_fill_node(skb, nh, AF_INET6))
|
2019-06-04 03:19:52 +00:00
|
|
|
goto nla_put_failure;
|
|
|
|
|
|
|
|
nla_nest_end(skb, mp);
|
|
|
|
} else {
|
|
|
|
struct fib6_nh *fib6_nh;
|
|
|
|
|
|
|
|
fib6_nh = nexthop_fib6_nh(nh);
|
2019-09-04 14:11:58 +00:00
|
|
|
if (fib_nexthop_info(skb, &fib6_nh->nh_common, AF_INET6,
|
2019-06-04 03:19:52 +00:00
|
|
|
flags, false) < 0)
|
|
|
|
goto nla_put_failure;
|
|
|
|
}
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
nla_put_failure:
|
|
|
|
return -EMSGSIZE;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
static int rt6_fill_node(struct net *net, struct sk_buff *skb,
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *rt, struct dst_entry *dst,
|
2018-04-18 00:33:16 +00:00
|
|
|
struct in6_addr *dest, struct in6_addr *src,
|
2012-09-07 20:12:54 +00:00
|
|
|
int iif, int type, u32 portid, u32 seq,
|
2017-01-17 23:51:08 +00:00
|
|
|
unsigned int flags)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2018-09-11 06:33:58 +00:00
|
|
|
struct rt6_info *rt6 = (struct rt6_info *)dst;
|
|
|
|
struct rt6key *rt6_dst, *rt6_src;
|
|
|
|
u32 *pmetrics, table, rt6_flags;
|
2019-06-04 03:19:52 +00:00
|
|
|
unsigned char nh_flags = 0;
|
2006-08-22 07:01:27 +00:00
|
|
|
struct nlmsghdr *nlh;
|
2018-09-11 06:33:58 +00:00
|
|
|
struct rtmsg *rtm;
|
2018-04-18 00:33:16 +00:00
|
|
|
long expires = 0;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2012-09-07 20:12:54 +00:00
|
|
|
nlh = nlmsg_put(skb, portid, seq, type, sizeof(*rtm), flags);
|
2011-12-03 23:02:47 +00:00
|
|
|
if (!nlh)
|
2007-02-01 07:16:40 +00:00
|
|
|
return -EMSGSIZE;
|
2006-08-22 07:01:27 +00:00
|
|
|
|
2018-09-11 06:33:58 +00:00
|
|
|
if (rt6) {
|
|
|
|
rt6_dst = &rt6->rt6i_dst;
|
|
|
|
rt6_src = &rt6->rt6i_src;
|
|
|
|
rt6_flags = rt6->rt6i_flags;
|
|
|
|
} else {
|
|
|
|
rt6_dst = &rt->fib6_dst;
|
|
|
|
rt6_src = &rt->fib6_src;
|
|
|
|
rt6_flags = rt->fib6_flags;
|
|
|
|
}
|
|
|
|
|
2006-08-22 07:01:27 +00:00
|
|
|
rtm = nlmsg_data(nlh);
|
2005-04-16 22:20:36 +00:00
|
|
|
rtm->rtm_family = AF_INET6;
|
2018-09-11 06:33:58 +00:00
|
|
|
rtm->rtm_dst_len = rt6_dst->plen;
|
|
|
|
rtm->rtm_src_len = rt6_src->plen;
|
2005-04-16 22:20:36 +00:00
|
|
|
rtm->rtm_tos = 0;
|
2018-04-18 22:38:59 +00:00
|
|
|
if (rt->fib6_table)
|
|
|
|
table = rt->fib6_table->tb6_id;
|
2006-08-05 06:20:06 +00:00
|
|
|
else
|
2006-08-11 06:09:48 +00:00
|
|
|
table = RT6_TABLE_UNSPEC;
|
2019-02-21 00:23:04 +00:00
|
|
|
rtm->rtm_table = table < 256 ? table : RT_TABLE_COMPAT;
|
2012-04-02 00:27:33 +00:00
|
|
|
if (nla_put_u32(skb, RTA_TABLE, table))
|
|
|
|
goto nla_put_failure;
|
2018-04-18 00:33:13 +00:00
|
|
|
|
|
|
|
rtm->rtm_type = rt->fib6_type;
|
2005-04-16 22:20:36 +00:00
|
|
|
rtm->rtm_flags = 0;
|
|
|
|
rtm->rtm_scope = RT_SCOPE_UNIVERSE;
|
2018-04-18 22:38:59 +00:00
|
|
|
rtm->rtm_protocol = rt->fib6_protocol;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-09-11 06:33:58 +00:00
|
|
|
if (rt6_flags & RTF_CACHE)
|
2005-04-16 22:20:36 +00:00
|
|
|
rtm->rtm_flags |= RTM_F_CLONED;
|
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
if (dest) {
|
|
|
|
if (nla_put_in6_addr(skb, RTA_DST, dest))
|
2012-04-02 00:27:33 +00:00
|
|
|
goto nla_put_failure;
|
2007-02-09 14:24:49 +00:00
|
|
|
rtm->rtm_dst_len = 128;
|
2005-04-16 22:20:36 +00:00
|
|
|
} else if (rtm->rtm_dst_len)
|
2018-09-11 06:33:58 +00:00
|
|
|
if (nla_put_in6_addr(skb, RTA_DST, &rt6_dst->addr))
|
2012-04-02 00:27:33 +00:00
|
|
|
goto nla_put_failure;
|
2005-04-16 22:20:36 +00:00
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
|
|
|
if (src) {
|
2015-03-29 14:59:25 +00:00
|
|
|
if (nla_put_in6_addr(skb, RTA_SRC, src))
|
2012-04-02 00:27:33 +00:00
|
|
|
goto nla_put_failure;
|
2007-02-09 14:24:49 +00:00
|
|
|
rtm->rtm_src_len = 128;
|
2012-04-02 00:27:33 +00:00
|
|
|
} else if (rtm->rtm_src_len &&
|
2018-09-11 06:33:58 +00:00
|
|
|
nla_put_in6_addr(skb, RTA_SRC, &rt6_src->addr))
|
2012-04-02 00:27:33 +00:00
|
|
|
goto nla_put_failure;
|
2005-04-16 22:20:36 +00:00
|
|
|
#endif
|
2008-04-03 00:22:53 +00:00
|
|
|
if (iif) {
|
|
|
|
#ifdef CONFIG_IPV6_MROUTE
|
2018-09-11 06:33:58 +00:00
|
|
|
if (ipv6_addr_is_multicast(&rt6_dst->addr)) {
|
2017-01-17 23:51:07 +00:00
|
|
|
int err = ip6mr_get_route(net, skb, rtm, portid);
|
|
|
|
|
|
|
|
if (err == 0)
|
|
|
|
return 0;
|
|
|
|
if (err < 0)
|
|
|
|
goto nla_put_failure;
|
2008-04-03 00:22:53 +00:00
|
|
|
} else
|
|
|
|
#endif
|
2012-04-02 00:27:33 +00:00
|
|
|
if (nla_put_u32(skb, RTA_IIF, iif))
|
|
|
|
goto nla_put_failure;
|
2018-04-18 00:33:16 +00:00
|
|
|
} else if (dest) {
|
2005-04-16 22:20:36 +00:00
|
|
|
struct in6_addr saddr_buf;
|
2018-04-18 00:33:16 +00:00
|
|
|
if (ip6_route_get_saddr(net, rt, dest, 0, &saddr_buf) == 0 &&
|
2015-03-29 14:59:25 +00:00
|
|
|
nla_put_in6_addr(skb, RTA_PREFSRC, &saddr_buf))
|
2012-04-02 00:27:33 +00:00
|
|
|
goto nla_put_failure;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2006-08-22 07:01:27 +00:00
|
|
|
|
2018-04-18 22:38:59 +00:00
|
|
|
if (rt->fib6_prefsrc.plen) {
|
2011-04-13 21:10:57 +00:00
|
|
|
struct in6_addr saddr_buf;
|
2018-04-18 22:38:59 +00:00
|
|
|
saddr_buf = rt->fib6_prefsrc.addr;
|
2015-03-29 14:59:25 +00:00
|
|
|
if (nla_put_in6_addr(skb, RTA_PREFSRC, &saddr_buf))
|
2012-04-02 00:27:33 +00:00
|
|
|
goto nla_put_failure;
|
2011-04-13 21:10:57 +00:00
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
pmetrics = dst ? dst_metrics_ptr(dst) : rt->fib6_metrics->metrics;
|
|
|
|
if (rtnetlink_put_metrics(skb, pmetrics) < 0)
|
2006-08-22 07:01:27 +00:00
|
|
|
goto nla_put_failure;
|
|
|
|
|
2018-04-18 22:38:59 +00:00
|
|
|
if (nla_put_u32(skb, RTA_PRIORITY, rt->fib6_metric))
|
2012-04-02 00:27:33 +00:00
|
|
|
goto nla_put_failure;
|
2012-07-29 16:01:30 +00:00
|
|
|
|
2017-02-02 20:37:09 +00:00
|
|
|
/* For multipath routes, walk the siblings list and add
|
|
|
|
* each as a nexthop within RTA_MULTIPATH.
|
|
|
|
*/
|
2018-09-11 06:33:58 +00:00
|
|
|
if (rt6) {
|
|
|
|
if (rt6_flags & RTF_GATEWAY &&
|
|
|
|
nla_put_in6_addr(skb, RTA_GATEWAY, &rt6->rt6i_gateway))
|
|
|
|
goto nla_put_failure;
|
|
|
|
|
|
|
|
if (dst->dev && nla_put_u32(skb, RTA_OIF, dst->dev->ifindex))
|
|
|
|
goto nla_put_failure;
|
2020-11-18 23:06:51 +00:00
|
|
|
|
|
|
|
if (dst->lwtstate &&
|
|
|
|
lwtunnel_fill_encap(skb, dst->lwtstate, RTA_ENCAP, RTA_ENCAP_TYPE) < 0)
|
|
|
|
goto nla_put_failure;
|
2018-09-11 06:33:58 +00:00
|
|
|
} else if (rt->fib6_nsiblings) {
|
2018-04-18 00:33:26 +00:00
|
|
|
struct fib6_info *sibling, *next_sibling;
|
2017-02-02 20:37:09 +00:00
|
|
|
struct nlattr *mp;
|
|
|
|
|
2019-04-26 09:13:06 +00:00
|
|
|
mp = nla_nest_start_noflag(skb, RTA_MULTIPATH);
|
2017-02-02 20:37:09 +00:00
|
|
|
if (!mp)
|
|
|
|
goto nla_put_failure;
|
|
|
|
|
2019-05-23 03:27:59 +00:00
|
|
|
if (fib_add_nexthop(skb, &rt->fib6_nh->nh_common,
|
2021-09-23 15:03:19 +00:00
|
|
|
rt->fib6_nh->fib_nh_weight, AF_INET6,
|
|
|
|
0) < 0)
|
2017-02-02 20:37:09 +00:00
|
|
|
goto nla_put_failure;
|
|
|
|
|
|
|
|
list_for_each_entry_safe(sibling, next_sibling,
|
2018-04-18 22:38:59 +00:00
|
|
|
&rt->fib6_siblings, fib6_siblings) {
|
2019-05-23 03:27:59 +00:00
|
|
|
if (fib_add_nexthop(skb, &sibling->fib6_nh->nh_common,
|
2019-09-04 14:11:58 +00:00
|
|
|
sibling->fib6_nh->fib_nh_weight,
|
2021-09-23 15:03:19 +00:00
|
|
|
AF_INET6, 0) < 0)
|
2017-02-02 20:37:09 +00:00
|
|
|
goto nla_put_failure;
|
|
|
|
}
|
|
|
|
|
|
|
|
nla_nest_end(skb, mp);
|
2019-06-04 03:19:52 +00:00
|
|
|
} else if (rt->nh) {
|
|
|
|
if (nla_put_u32(skb, RTA_NH_ID, rt->nh->id))
|
|
|
|
goto nla_put_failure;
|
2019-04-23 15:48:09 +00:00
|
|
|
|
2019-06-04 03:19:52 +00:00
|
|
|
if (nexthop_is_blackhole(rt->nh))
|
|
|
|
rtm->rtm_type = RTN_BLACKHOLE;
|
|
|
|
|
2020-04-27 20:56:46 +00:00
|
|
|
if (net->ipv4.sysctl_nexthop_compat_mode &&
|
|
|
|
rt6_fill_node_nexthop(skb, rt->nh, &nh_flags) < 0)
|
2019-06-04 03:19:52 +00:00
|
|
|
goto nla_put_failure;
|
|
|
|
|
|
|
|
rtm->rtm_flags |= nh_flags;
|
|
|
|
} else {
|
2019-09-04 14:11:58 +00:00
|
|
|
if (fib_nexthop_info(skb, &rt->fib6_nh->nh_common, AF_INET6,
|
2019-04-23 15:48:09 +00:00
|
|
|
&nh_flags, false) < 0)
|
2017-02-02 20:37:09 +00:00
|
|
|
goto nla_put_failure;
|
2019-04-23 15:48:09 +00:00
|
|
|
|
|
|
|
rtm->rtm_flags |= nh_flags;
|
2017-02-02 20:37:09 +00:00
|
|
|
}
|
|
|
|
|
2018-09-11 06:33:58 +00:00
|
|
|
if (rt6_flags & RTF_EXPIRES) {
|
2018-04-18 00:33:17 +00:00
|
|
|
expires = dst ? dst->expires : rt->expires;
|
|
|
|
expires -= jiffies;
|
|
|
|
}
|
2008-05-19 23:55:13 +00:00
|
|
|
|
2020-01-14 11:23:12 +00:00
|
|
|
if (!dst) {
|
|
|
|
if (rt->offload)
|
|
|
|
rtm->rtm_flags |= RTM_F_OFFLOAD;
|
|
|
|
if (rt->trap)
|
|
|
|
rtm->rtm_flags |= RTM_F_TRAP;
|
2021-02-07 08:22:52 +00:00
|
|
|
if (rt->offload_failed)
|
|
|
|
rtm->rtm_flags |= RTM_F_OFFLOAD_FAILED;
|
2020-01-14 11:23:12 +00:00
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
if (rtnl_put_cacheinfo(skb, dst, 0, expires, dst ? dst->error : 0) < 0)
|
2006-11-27 17:27:07 +00:00
|
|
|
goto nla_put_failure;
|
2006-08-22 07:01:27 +00:00
|
|
|
|
2018-09-11 06:33:58 +00:00
|
|
|
if (nla_put_u8(skb, RTA_PREF, IPV6_EXTRACT_PREF(rt6_flags)))
|
2015-03-11 14:39:21 +00:00
|
|
|
goto nla_put_failure;
|
|
|
|
|
2015-07-21 08:43:48 +00:00
|
|
|
|
2015-01-16 21:09:00 +00:00
|
|
|
nlmsg_end(skb, nlh);
|
|
|
|
return 0;
|
2006-08-22 07:01:27 +00:00
|
|
|
|
|
|
|
nla_put_failure:
|
2007-02-01 07:16:40 +00:00
|
|
|
nlmsg_cancel(skb, nlh);
|
|
|
|
return -EMSGSIZE;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2019-06-08 21:53:27 +00:00
|
|
|
static int fib6_info_nh_uses_dev(struct fib6_nh *nh, void *arg)
|
|
|
|
{
|
|
|
|
const struct net_device *dev = arg;
|
|
|
|
|
|
|
|
if (nh->fib_nh_dev == dev)
|
|
|
|
return 1;
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2018-10-16 01:56:44 +00:00
|
|
|
static bool fib6_info_uses_dev(const struct fib6_info *f6i,
|
|
|
|
const struct net_device *dev)
|
|
|
|
{
|
2019-06-08 21:53:27 +00:00
|
|
|
if (f6i->nh) {
|
|
|
|
struct net_device *_dev = (struct net_device *)dev;
|
|
|
|
|
|
|
|
return !!nexthop_for_each_fib6_nh(f6i->nh,
|
|
|
|
fib6_info_nh_uses_dev,
|
|
|
|
_dev);
|
|
|
|
}
|
|
|
|
|
2019-05-23 03:27:59 +00:00
|
|
|
if (f6i->fib6_nh->fib_nh_dev == dev)
|
2018-10-16 01:56:44 +00:00
|
|
|
return true;
|
|
|
|
|
|
|
|
if (f6i->fib6_nsiblings) {
|
|
|
|
struct fib6_info *sibling, *next_sibling;
|
|
|
|
|
|
|
|
list_for_each_entry_safe(sibling, next_sibling,
|
|
|
|
&f6i->fib6_siblings, fib6_siblings) {
|
2019-05-23 03:27:59 +00:00
|
|
|
if (sibling->fib6_nh->fib_nh_dev == dev)
|
2018-10-16 01:56:44 +00:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
ipv6: Dump route exceptions if requested
Since commit 2b760fcf5cfb ("ipv6: hook up exception table to store dst
cache"), route exceptions reside in a separate hash table, and won't be
found by walking the FIB, so they won't be dumped to userspace on a
RTM_GETROUTE message.
This causes 'ip -6 route list cache' and 'ip -6 route flush cache' to
have no function anymore:
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 539sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 536sec mtu 1500 pref medium
# ip -6 route list cache
# ip -6 route flush cache
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 520sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 519sec mtu 1500 pref medium
because iproute2 lists cached routes using RTM_GETROUTE, and flushes them
by listing all the routes, and deleting them with RTM_DELROUTE one by one.
If cached routes are requested using the RTM_F_CLONED flag together with
strict checking, or if no strict checking is requested (and hence we can't
consistently apply filters), look up exceptions in the hash table
associated with the current fib6_info in rt6_dump_route(), and, if present
and not expired, add them to the dump.
We might be unable to dump all the entries for a given node in a single
message, so keep track of how many entries were handled for the current
node in fib6_walker, and skip that amount in case we start from the same
partially dumped node.
When a partial dump restarts, as the starting node might change when
'sernum' changes, we have no guarantee that we need to skip the same
amount of in-node entries. Therefore, we need two counters, and we need to
zero the in-node counter if the node from which the dump is resumed
differs.
Note that, with the current version of iproute2, this only fixes the
'ip -6 route list cache': on a flush command, iproute2 doesn't pass
RTM_F_CLONED and, due to this inconsistency, 'ip -6 route flush cache' is
still unable to fetch the routes to be flushed. This will be addressed in
a patch for iproute2.
To flush cached routes, a procfs entry could be introduced instead: that's
how it works for IPv4. We already have a rt6_flush_exception() function
ready to be wired to it. However, this would not solve the issue for
listing.
Versions of iproute2 and kernel tested:
iproute2
kernel 4.14.0 4.15.0 4.19.0 5.0.0 5.1.0 5.1.0, patched
3.18 list + + + + + +
flush + + + + + +
4.4 list + + + + + +
flush + + + + + +
4.9 list + + + + + +
flush + + + + + +
4.14 list + + + + + +
flush + + + + + +
4.15 list
flush
4.19 list
flush
5.0 list
flush
5.1 list
flush
with list + + + + + +
fix flush + + + +
v7:
- Explain usage of "skip" counters in commit message (suggested by
David Ahern)
v6:
- Rebase onto net-next, use recently introduced nexthop walker
- Make rt6_nh_dump_exceptions() a separate function (suggested by David
Ahern)
v5:
- Use dump_routes and dump_exceptions from filter, ignore NLM_F_MATCH,
update test results (flushing works with iproute2 < 5.0.0 now)
v4:
- Split NLM_F_MATCH and strict check handling in separate patches
- Filter routes using RTM_F_CLONED: if it's not set, only return
non-cached routes, and if it's set, only return cached routes:
change requested by David Ahern and Martin Lau. This implies that
iproute2 needs a separate patch to be able to flush IPv6 cached
routes. This is not ideal because we can't fix the breakage caused
by 2b760fcf5cfb entirely in kernel. However, two years have passed
since then, and this makes it more tolerable
v3:
- More descriptive comment about expired exceptions in rt6_dump_route()
- Swap return values of rt6_dump_route() (suggested by Martin Lau)
- Don't zero skip_in_node in case we don't dump anything in a given pass
(also suggested by Martin Lau)
- Remove check on RTM_F_CLONED altogether: in the current UAPI semantic,
it's just a flag to indicate the route was cloned, not to filter on
routes
v2: Add tracking of number of entries to be skipped in current node after
a partial dump. As we restart from the same node, if not all the
exceptions for a given node fit in a single message, the dump will
not terminate, as suggested by Martin Lau. This is a concrete
possibility, setting up a big number of exceptions for the same route
actually causes the issue, suggested by David Ahern.
Reported-by: Jianlin Shi <jishi@redhat.com>
Fixes: 2b760fcf5cfb ("ipv6: hook up exception table to store dst cache")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-21 15:45:27 +00:00
|
|
|
struct fib6_nh_exception_dump_walker {
|
|
|
|
struct rt6_rtnl_dump_arg *dump;
|
|
|
|
struct fib6_info *rt;
|
|
|
|
unsigned int flags;
|
|
|
|
unsigned int skip;
|
|
|
|
unsigned int count;
|
|
|
|
};
|
|
|
|
|
|
|
|
static int rt6_nh_dump_exceptions(struct fib6_nh *nh, void *arg)
|
|
|
|
{
|
|
|
|
struct fib6_nh_exception_dump_walker *w = arg;
|
|
|
|
struct rt6_rtnl_dump_arg *dump = w->dump;
|
|
|
|
struct rt6_exception_bucket *bucket;
|
|
|
|
struct rt6_exception *rt6_ex;
|
|
|
|
int i, err;
|
|
|
|
|
|
|
|
bucket = fib6_nh_get_excptn_bucket(nh, NULL);
|
|
|
|
if (!bucket)
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
for (i = 0; i < FIB6_EXCEPTION_BUCKET_SIZE; i++) {
|
|
|
|
hlist_for_each_entry(rt6_ex, &bucket->chain, hlist) {
|
|
|
|
if (w->skip) {
|
|
|
|
w->skip--;
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Expiration of entries doesn't bump sernum, insertion
|
|
|
|
* does. Removal is triggered by insertion, so we can
|
|
|
|
* rely on the fact that if entries change between two
|
|
|
|
* partial dumps, this node is scanned again completely,
|
|
|
|
* see rt6_insert_exception() and fib6_dump_table().
|
|
|
|
*
|
|
|
|
* Count expired entries we go through as handled
|
|
|
|
* entries that we'll skip next time, in case of partial
|
|
|
|
* node dump. Otherwise, if entries expire meanwhile,
|
|
|
|
* we'll skip the wrong amount.
|
|
|
|
*/
|
|
|
|
if (rt6_check_expired(rt6_ex->rt6i)) {
|
|
|
|
w->count++;
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
|
|
|
err = rt6_fill_node(dump->net, dump->skb, w->rt,
|
|
|
|
&rt6_ex->rt6i->dst, NULL, NULL, 0,
|
|
|
|
RTM_NEWROUTE,
|
|
|
|
NETLINK_CB(dump->cb->skb).portid,
|
|
|
|
dump->cb->nlh->nlmsg_seq, w->flags);
|
|
|
|
if (err)
|
|
|
|
return err;
|
|
|
|
|
|
|
|
w->count++;
|
|
|
|
}
|
|
|
|
bucket++;
|
|
|
|
}
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2019-06-21 15:45:26 +00:00
|
|
|
/* Return -1 if done with node, number of handled routes on partial dump */
|
ipv6: Dump route exceptions if requested
Since commit 2b760fcf5cfb ("ipv6: hook up exception table to store dst
cache"), route exceptions reside in a separate hash table, and won't be
found by walking the FIB, so they won't be dumped to userspace on a
RTM_GETROUTE message.
This causes 'ip -6 route list cache' and 'ip -6 route flush cache' to
have no function anymore:
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 539sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 536sec mtu 1500 pref medium
# ip -6 route list cache
# ip -6 route flush cache
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 520sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 519sec mtu 1500 pref medium
because iproute2 lists cached routes using RTM_GETROUTE, and flushes them
by listing all the routes, and deleting them with RTM_DELROUTE one by one.
If cached routes are requested using the RTM_F_CLONED flag together with
strict checking, or if no strict checking is requested (and hence we can't
consistently apply filters), look up exceptions in the hash table
associated with the current fib6_info in rt6_dump_route(), and, if present
and not expired, add them to the dump.
We might be unable to dump all the entries for a given node in a single
message, so keep track of how many entries were handled for the current
node in fib6_walker, and skip that amount in case we start from the same
partially dumped node.
When a partial dump restarts, as the starting node might change when
'sernum' changes, we have no guarantee that we need to skip the same
amount of in-node entries. Therefore, we need two counters, and we need to
zero the in-node counter if the node from which the dump is resumed
differs.
Note that, with the current version of iproute2, this only fixes the
'ip -6 route list cache': on a flush command, iproute2 doesn't pass
RTM_F_CLONED and, due to this inconsistency, 'ip -6 route flush cache' is
still unable to fetch the routes to be flushed. This will be addressed in
a patch for iproute2.
To flush cached routes, a procfs entry could be introduced instead: that's
how it works for IPv4. We already have a rt6_flush_exception() function
ready to be wired to it. However, this would not solve the issue for
listing.
Versions of iproute2 and kernel tested:
iproute2
kernel 4.14.0 4.15.0 4.19.0 5.0.0 5.1.0 5.1.0, patched
3.18 list + + + + + +
flush + + + + + +
4.4 list + + + + + +
flush + + + + + +
4.9 list + + + + + +
flush + + + + + +
4.14 list + + + + + +
flush + + + + + +
4.15 list
flush
4.19 list
flush
5.0 list
flush
5.1 list
flush
with list + + + + + +
fix flush + + + +
v7:
- Explain usage of "skip" counters in commit message (suggested by
David Ahern)
v6:
- Rebase onto net-next, use recently introduced nexthop walker
- Make rt6_nh_dump_exceptions() a separate function (suggested by David
Ahern)
v5:
- Use dump_routes and dump_exceptions from filter, ignore NLM_F_MATCH,
update test results (flushing works with iproute2 < 5.0.0 now)
v4:
- Split NLM_F_MATCH and strict check handling in separate patches
- Filter routes using RTM_F_CLONED: if it's not set, only return
non-cached routes, and if it's set, only return cached routes:
change requested by David Ahern and Martin Lau. This implies that
iproute2 needs a separate patch to be able to flush IPv6 cached
routes. This is not ideal because we can't fix the breakage caused
by 2b760fcf5cfb entirely in kernel. However, two years have passed
since then, and this makes it more tolerable
v3:
- More descriptive comment about expired exceptions in rt6_dump_route()
- Swap return values of rt6_dump_route() (suggested by Martin Lau)
- Don't zero skip_in_node in case we don't dump anything in a given pass
(also suggested by Martin Lau)
- Remove check on RTM_F_CLONED altogether: in the current UAPI semantic,
it's just a flag to indicate the route was cloned, not to filter on
routes
v2: Add tracking of number of entries to be skipped in current node after
a partial dump. As we restart from the same node, if not all the
exceptions for a given node fit in a single message, the dump will
not terminate, as suggested by Martin Lau. This is a concrete
possibility, setting up a big number of exceptions for the same route
actually causes the issue, suggested by David Ahern.
Reported-by: Jianlin Shi <jishi@redhat.com>
Fixes: 2b760fcf5cfb ("ipv6: hook up exception table to store dst cache")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-21 15:45:27 +00:00
|
|
|
int rt6_dump_route(struct fib6_info *rt, void *p_arg, unsigned int skip)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
|
|
|
struct rt6_rtnl_dump_arg *arg = (struct rt6_rtnl_dump_arg *) p_arg;
|
2018-10-16 01:56:44 +00:00
|
|
|
struct fib_dump_filter *filter = &arg->filter;
|
|
|
|
unsigned int flags = NLM_F_MULTI;
|
2017-01-26 21:54:08 +00:00
|
|
|
struct net *net = arg->net;
|
ipv6: Dump route exceptions if requested
Since commit 2b760fcf5cfb ("ipv6: hook up exception table to store dst
cache"), route exceptions reside in a separate hash table, and won't be
found by walking the FIB, so they won't be dumped to userspace on a
RTM_GETROUTE message.
This causes 'ip -6 route list cache' and 'ip -6 route flush cache' to
have no function anymore:
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 539sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 536sec mtu 1500 pref medium
# ip -6 route list cache
# ip -6 route flush cache
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 520sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 519sec mtu 1500 pref medium
because iproute2 lists cached routes using RTM_GETROUTE, and flushes them
by listing all the routes, and deleting them with RTM_DELROUTE one by one.
If cached routes are requested using the RTM_F_CLONED flag together with
strict checking, or if no strict checking is requested (and hence we can't
consistently apply filters), look up exceptions in the hash table
associated with the current fib6_info in rt6_dump_route(), and, if present
and not expired, add them to the dump.
We might be unable to dump all the entries for a given node in a single
message, so keep track of how many entries were handled for the current
node in fib6_walker, and skip that amount in case we start from the same
partially dumped node.
When a partial dump restarts, as the starting node might change when
'sernum' changes, we have no guarantee that we need to skip the same
amount of in-node entries. Therefore, we need two counters, and we need to
zero the in-node counter if the node from which the dump is resumed
differs.
Note that, with the current version of iproute2, this only fixes the
'ip -6 route list cache': on a flush command, iproute2 doesn't pass
RTM_F_CLONED and, due to this inconsistency, 'ip -6 route flush cache' is
still unable to fetch the routes to be flushed. This will be addressed in
a patch for iproute2.
To flush cached routes, a procfs entry could be introduced instead: that's
how it works for IPv4. We already have a rt6_flush_exception() function
ready to be wired to it. However, this would not solve the issue for
listing.
Versions of iproute2 and kernel tested:
iproute2
kernel 4.14.0 4.15.0 4.19.0 5.0.0 5.1.0 5.1.0, patched
3.18 list + + + + + +
flush + + + + + +
4.4 list + + + + + +
flush + + + + + +
4.9 list + + + + + +
flush + + + + + +
4.14 list + + + + + +
flush + + + + + +
4.15 list
flush
4.19 list
flush
5.0 list
flush
5.1 list
flush
with list + + + + + +
fix flush + + + +
v7:
- Explain usage of "skip" counters in commit message (suggested by
David Ahern)
v6:
- Rebase onto net-next, use recently introduced nexthop walker
- Make rt6_nh_dump_exceptions() a separate function (suggested by David
Ahern)
v5:
- Use dump_routes and dump_exceptions from filter, ignore NLM_F_MATCH,
update test results (flushing works with iproute2 < 5.0.0 now)
v4:
- Split NLM_F_MATCH and strict check handling in separate patches
- Filter routes using RTM_F_CLONED: if it's not set, only return
non-cached routes, and if it's set, only return cached routes:
change requested by David Ahern and Martin Lau. This implies that
iproute2 needs a separate patch to be able to flush IPv6 cached
routes. This is not ideal because we can't fix the breakage caused
by 2b760fcf5cfb entirely in kernel. However, two years have passed
since then, and this makes it more tolerable
v3:
- More descriptive comment about expired exceptions in rt6_dump_route()
- Swap return values of rt6_dump_route() (suggested by Martin Lau)
- Don't zero skip_in_node in case we don't dump anything in a given pass
(also suggested by Martin Lau)
- Remove check on RTM_F_CLONED altogether: in the current UAPI semantic,
it's just a flag to indicate the route was cloned, not to filter on
routes
v2: Add tracking of number of entries to be skipped in current node after
a partial dump. As we restart from the same node, if not all the
exceptions for a given node fit in a single message, the dump will
not terminate, as suggested by Martin Lau. This is a concrete
possibility, setting up a big number of exceptions for the same route
actually causes the issue, suggested by David Ahern.
Reported-by: Jianlin Shi <jishi@redhat.com>
Fixes: 2b760fcf5cfb ("ipv6: hook up exception table to store dst cache")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-21 15:45:27 +00:00
|
|
|
int count = 0;
|
2017-01-26 21:54:08 +00:00
|
|
|
|
2018-04-18 00:33:18 +00:00
|
|
|
if (rt == net->ipv6.fib6_null_entry)
|
2019-06-21 15:45:26 +00:00
|
|
|
return -1;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-10-16 01:56:44 +00:00
|
|
|
if ((filter->flags & RTM_F_PREFIX) &&
|
|
|
|
!(rt->fib6_flags & RTF_PREFIX_RT)) {
|
|
|
|
/* success since this is not a prefix route */
|
2019-06-21 15:45:26 +00:00
|
|
|
return -1;
|
2018-10-16 01:56:44 +00:00
|
|
|
}
|
ipv6: Dump route exceptions if requested
Since commit 2b760fcf5cfb ("ipv6: hook up exception table to store dst
cache"), route exceptions reside in a separate hash table, and won't be
found by walking the FIB, so they won't be dumped to userspace on a
RTM_GETROUTE message.
This causes 'ip -6 route list cache' and 'ip -6 route flush cache' to
have no function anymore:
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 539sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 536sec mtu 1500 pref medium
# ip -6 route list cache
# ip -6 route flush cache
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 520sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 519sec mtu 1500 pref medium
because iproute2 lists cached routes using RTM_GETROUTE, and flushes them
by listing all the routes, and deleting them with RTM_DELROUTE one by one.
If cached routes are requested using the RTM_F_CLONED flag together with
strict checking, or if no strict checking is requested (and hence we can't
consistently apply filters), look up exceptions in the hash table
associated with the current fib6_info in rt6_dump_route(), and, if present
and not expired, add them to the dump.
We might be unable to dump all the entries for a given node in a single
message, so keep track of how many entries were handled for the current
node in fib6_walker, and skip that amount in case we start from the same
partially dumped node.
When a partial dump restarts, as the starting node might change when
'sernum' changes, we have no guarantee that we need to skip the same
amount of in-node entries. Therefore, we need two counters, and we need to
zero the in-node counter if the node from which the dump is resumed
differs.
Note that, with the current version of iproute2, this only fixes the
'ip -6 route list cache': on a flush command, iproute2 doesn't pass
RTM_F_CLONED and, due to this inconsistency, 'ip -6 route flush cache' is
still unable to fetch the routes to be flushed. This will be addressed in
a patch for iproute2.
To flush cached routes, a procfs entry could be introduced instead: that's
how it works for IPv4. We already have a rt6_flush_exception() function
ready to be wired to it. However, this would not solve the issue for
listing.
Versions of iproute2 and kernel tested:
iproute2
kernel 4.14.0 4.15.0 4.19.0 5.0.0 5.1.0 5.1.0, patched
3.18 list + + + + + +
flush + + + + + +
4.4 list + + + + + +
flush + + + + + +
4.9 list + + + + + +
flush + + + + + +
4.14 list + + + + + +
flush + + + + + +
4.15 list
flush
4.19 list
flush
5.0 list
flush
5.1 list
flush
with list + + + + + +
fix flush + + + +
v7:
- Explain usage of "skip" counters in commit message (suggested by
David Ahern)
v6:
- Rebase onto net-next, use recently introduced nexthop walker
- Make rt6_nh_dump_exceptions() a separate function (suggested by David
Ahern)
v5:
- Use dump_routes and dump_exceptions from filter, ignore NLM_F_MATCH,
update test results (flushing works with iproute2 < 5.0.0 now)
v4:
- Split NLM_F_MATCH and strict check handling in separate patches
- Filter routes using RTM_F_CLONED: if it's not set, only return
non-cached routes, and if it's set, only return cached routes:
change requested by David Ahern and Martin Lau. This implies that
iproute2 needs a separate patch to be able to flush IPv6 cached
routes. This is not ideal because we can't fix the breakage caused
by 2b760fcf5cfb entirely in kernel. However, two years have passed
since then, and this makes it more tolerable
v3:
- More descriptive comment about expired exceptions in rt6_dump_route()
- Swap return values of rt6_dump_route() (suggested by Martin Lau)
- Don't zero skip_in_node in case we don't dump anything in a given pass
(also suggested by Martin Lau)
- Remove check on RTM_F_CLONED altogether: in the current UAPI semantic,
it's just a flag to indicate the route was cloned, not to filter on
routes
v2: Add tracking of number of entries to be skipped in current node after
a partial dump. As we restart from the same node, if not all the
exceptions for a given node fit in a single message, the dump will
not terminate, as suggested by Martin Lau. This is a concrete
possibility, setting up a big number of exceptions for the same route
actually causes the issue, suggested by David Ahern.
Reported-by: Jianlin Shi <jishi@redhat.com>
Fixes: 2b760fcf5cfb ("ipv6: hook up exception table to store dst cache")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-21 15:45:27 +00:00
|
|
|
if (filter->filter_set &&
|
|
|
|
((filter->rt_type && rt->fib6_type != filter->rt_type) ||
|
|
|
|
(filter->dev && !fib6_info_uses_dev(rt, filter->dev)) ||
|
|
|
|
(filter->protocol && rt->fib6_protocol != filter->protocol))) {
|
|
|
|
return -1;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (filter->filter_set ||
|
|
|
|
!filter->dump_routes || !filter->dump_exceptions) {
|
2018-10-16 01:56:44 +00:00
|
|
|
flags |= NLM_F_DUMP_FILTERED;
|
2017-01-17 23:51:08 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
|
ipv6: Dump route exceptions if requested
Since commit 2b760fcf5cfb ("ipv6: hook up exception table to store dst
cache"), route exceptions reside in a separate hash table, and won't be
found by walking the FIB, so they won't be dumped to userspace on a
RTM_GETROUTE message.
This causes 'ip -6 route list cache' and 'ip -6 route flush cache' to
have no function anymore:
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 539sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 536sec mtu 1500 pref medium
# ip -6 route list cache
# ip -6 route flush cache
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 520sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 519sec mtu 1500 pref medium
because iproute2 lists cached routes using RTM_GETROUTE, and flushes them
by listing all the routes, and deleting them with RTM_DELROUTE one by one.
If cached routes are requested using the RTM_F_CLONED flag together with
strict checking, or if no strict checking is requested (and hence we can't
consistently apply filters), look up exceptions in the hash table
associated with the current fib6_info in rt6_dump_route(), and, if present
and not expired, add them to the dump.
We might be unable to dump all the entries for a given node in a single
message, so keep track of how many entries were handled for the current
node in fib6_walker, and skip that amount in case we start from the same
partially dumped node.
When a partial dump restarts, as the starting node might change when
'sernum' changes, we have no guarantee that we need to skip the same
amount of in-node entries. Therefore, we need two counters, and we need to
zero the in-node counter if the node from which the dump is resumed
differs.
Note that, with the current version of iproute2, this only fixes the
'ip -6 route list cache': on a flush command, iproute2 doesn't pass
RTM_F_CLONED and, due to this inconsistency, 'ip -6 route flush cache' is
still unable to fetch the routes to be flushed. This will be addressed in
a patch for iproute2.
To flush cached routes, a procfs entry could be introduced instead: that's
how it works for IPv4. We already have a rt6_flush_exception() function
ready to be wired to it. However, this would not solve the issue for
listing.
Versions of iproute2 and kernel tested:
iproute2
kernel 4.14.0 4.15.0 4.19.0 5.0.0 5.1.0 5.1.0, patched
3.18 list + + + + + +
flush + + + + + +
4.4 list + + + + + +
flush + + + + + +
4.9 list + + + + + +
flush + + + + + +
4.14 list + + + + + +
flush + + + + + +
4.15 list
flush
4.19 list
flush
5.0 list
flush
5.1 list
flush
with list + + + + + +
fix flush + + + +
v7:
- Explain usage of "skip" counters in commit message (suggested by
David Ahern)
v6:
- Rebase onto net-next, use recently introduced nexthop walker
- Make rt6_nh_dump_exceptions() a separate function (suggested by David
Ahern)
v5:
- Use dump_routes and dump_exceptions from filter, ignore NLM_F_MATCH,
update test results (flushing works with iproute2 < 5.0.0 now)
v4:
- Split NLM_F_MATCH and strict check handling in separate patches
- Filter routes using RTM_F_CLONED: if it's not set, only return
non-cached routes, and if it's set, only return cached routes:
change requested by David Ahern and Martin Lau. This implies that
iproute2 needs a separate patch to be able to flush IPv6 cached
routes. This is not ideal because we can't fix the breakage caused
by 2b760fcf5cfb entirely in kernel. However, two years have passed
since then, and this makes it more tolerable
v3:
- More descriptive comment about expired exceptions in rt6_dump_route()
- Swap return values of rt6_dump_route() (suggested by Martin Lau)
- Don't zero skip_in_node in case we don't dump anything in a given pass
(also suggested by Martin Lau)
- Remove check on RTM_F_CLONED altogether: in the current UAPI semantic,
it's just a flag to indicate the route was cloned, not to filter on
routes
v2: Add tracking of number of entries to be skipped in current node after
a partial dump. As we restart from the same node, if not all the
exceptions for a given node fit in a single message, the dump will
not terminate, as suggested by Martin Lau. This is a concrete
possibility, setting up a big number of exceptions for the same route
actually causes the issue, suggested by David Ahern.
Reported-by: Jianlin Shi <jishi@redhat.com>
Fixes: 2b760fcf5cfb ("ipv6: hook up exception table to store dst cache")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-21 15:45:27 +00:00
|
|
|
if (filter->dump_routes) {
|
|
|
|
if (skip) {
|
|
|
|
skip--;
|
|
|
|
} else {
|
|
|
|
if (rt6_fill_node(net, arg->skb, rt, NULL, NULL, NULL,
|
|
|
|
0, RTM_NEWROUTE,
|
|
|
|
NETLINK_CB(arg->cb->skb).portid,
|
|
|
|
arg->cb->nlh->nlmsg_seq, flags)) {
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
count++;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if (filter->dump_exceptions) {
|
|
|
|
struct fib6_nh_exception_dump_walker w = { .dump = arg,
|
|
|
|
.rt = rt,
|
|
|
|
.flags = flags,
|
|
|
|
.skip = skip,
|
|
|
|
.count = 0 };
|
|
|
|
int err;
|
|
|
|
|
2019-06-26 10:05:28 +00:00
|
|
|
rcu_read_lock();
|
ipv6: Dump route exceptions if requested
Since commit 2b760fcf5cfb ("ipv6: hook up exception table to store dst
cache"), route exceptions reside in a separate hash table, and won't be
found by walking the FIB, so they won't be dumped to userspace on a
RTM_GETROUTE message.
This causes 'ip -6 route list cache' and 'ip -6 route flush cache' to
have no function anymore:
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 539sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 536sec mtu 1500 pref medium
# ip -6 route list cache
# ip -6 route flush cache
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 520sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 519sec mtu 1500 pref medium
because iproute2 lists cached routes using RTM_GETROUTE, and flushes them
by listing all the routes, and deleting them with RTM_DELROUTE one by one.
If cached routes are requested using the RTM_F_CLONED flag together with
strict checking, or if no strict checking is requested (and hence we can't
consistently apply filters), look up exceptions in the hash table
associated with the current fib6_info in rt6_dump_route(), and, if present
and not expired, add them to the dump.
We might be unable to dump all the entries for a given node in a single
message, so keep track of how many entries were handled for the current
node in fib6_walker, and skip that amount in case we start from the same
partially dumped node.
When a partial dump restarts, as the starting node might change when
'sernum' changes, we have no guarantee that we need to skip the same
amount of in-node entries. Therefore, we need two counters, and we need to
zero the in-node counter if the node from which the dump is resumed
differs.
Note that, with the current version of iproute2, this only fixes the
'ip -6 route list cache': on a flush command, iproute2 doesn't pass
RTM_F_CLONED and, due to this inconsistency, 'ip -6 route flush cache' is
still unable to fetch the routes to be flushed. This will be addressed in
a patch for iproute2.
To flush cached routes, a procfs entry could be introduced instead: that's
how it works for IPv4. We already have a rt6_flush_exception() function
ready to be wired to it. However, this would not solve the issue for
listing.
Versions of iproute2 and kernel tested:
iproute2
kernel 4.14.0 4.15.0 4.19.0 5.0.0 5.1.0 5.1.0, patched
3.18 list + + + + + +
flush + + + + + +
4.4 list + + + + + +
flush + + + + + +
4.9 list + + + + + +
flush + + + + + +
4.14 list + + + + + +
flush + + + + + +
4.15 list
flush
4.19 list
flush
5.0 list
flush
5.1 list
flush
with list + + + + + +
fix flush + + + +
v7:
- Explain usage of "skip" counters in commit message (suggested by
David Ahern)
v6:
- Rebase onto net-next, use recently introduced nexthop walker
- Make rt6_nh_dump_exceptions() a separate function (suggested by David
Ahern)
v5:
- Use dump_routes and dump_exceptions from filter, ignore NLM_F_MATCH,
update test results (flushing works with iproute2 < 5.0.0 now)
v4:
- Split NLM_F_MATCH and strict check handling in separate patches
- Filter routes using RTM_F_CLONED: if it's not set, only return
non-cached routes, and if it's set, only return cached routes:
change requested by David Ahern and Martin Lau. This implies that
iproute2 needs a separate patch to be able to flush IPv6 cached
routes. This is not ideal because we can't fix the breakage caused
by 2b760fcf5cfb entirely in kernel. However, two years have passed
since then, and this makes it more tolerable
v3:
- More descriptive comment about expired exceptions in rt6_dump_route()
- Swap return values of rt6_dump_route() (suggested by Martin Lau)
- Don't zero skip_in_node in case we don't dump anything in a given pass
(also suggested by Martin Lau)
- Remove check on RTM_F_CLONED altogether: in the current UAPI semantic,
it's just a flag to indicate the route was cloned, not to filter on
routes
v2: Add tracking of number of entries to be skipped in current node after
a partial dump. As we restart from the same node, if not all the
exceptions for a given node fit in a single message, the dump will
not terminate, as suggested by Martin Lau. This is a concrete
possibility, setting up a big number of exceptions for the same route
actually causes the issue, suggested by David Ahern.
Reported-by: Jianlin Shi <jishi@redhat.com>
Fixes: 2b760fcf5cfb ("ipv6: hook up exception table to store dst cache")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-21 15:45:27 +00:00
|
|
|
if (rt->nh) {
|
|
|
|
err = nexthop_for_each_fib6_nh(rt->nh,
|
|
|
|
rt6_nh_dump_exceptions,
|
|
|
|
&w);
|
|
|
|
} else {
|
|
|
|
err = rt6_nh_dump_exceptions(rt->fib6_nh, &w);
|
|
|
|
}
|
2019-06-26 10:05:28 +00:00
|
|
|
rcu_read_unlock();
|
ipv6: Dump route exceptions if requested
Since commit 2b760fcf5cfb ("ipv6: hook up exception table to store dst
cache"), route exceptions reside in a separate hash table, and won't be
found by walking the FIB, so they won't be dumped to userspace on a
RTM_GETROUTE message.
This causes 'ip -6 route list cache' and 'ip -6 route flush cache' to
have no function anymore:
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 539sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 536sec mtu 1500 pref medium
# ip -6 route list cache
# ip -6 route flush cache
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 520sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 519sec mtu 1500 pref medium
because iproute2 lists cached routes using RTM_GETROUTE, and flushes them
by listing all the routes, and deleting them with RTM_DELROUTE one by one.
If cached routes are requested using the RTM_F_CLONED flag together with
strict checking, or if no strict checking is requested (and hence we can't
consistently apply filters), look up exceptions in the hash table
associated with the current fib6_info in rt6_dump_route(), and, if present
and not expired, add them to the dump.
We might be unable to dump all the entries for a given node in a single
message, so keep track of how many entries were handled for the current
node in fib6_walker, and skip that amount in case we start from the same
partially dumped node.
When a partial dump restarts, as the starting node might change when
'sernum' changes, we have no guarantee that we need to skip the same
amount of in-node entries. Therefore, we need two counters, and we need to
zero the in-node counter if the node from which the dump is resumed
differs.
Note that, with the current version of iproute2, this only fixes the
'ip -6 route list cache': on a flush command, iproute2 doesn't pass
RTM_F_CLONED and, due to this inconsistency, 'ip -6 route flush cache' is
still unable to fetch the routes to be flushed. This will be addressed in
a patch for iproute2.
To flush cached routes, a procfs entry could be introduced instead: that's
how it works for IPv4. We already have a rt6_flush_exception() function
ready to be wired to it. However, this would not solve the issue for
listing.
Versions of iproute2 and kernel tested:
iproute2
kernel 4.14.0 4.15.0 4.19.0 5.0.0 5.1.0 5.1.0, patched
3.18 list + + + + + +
flush + + + + + +
4.4 list + + + + + +
flush + + + + + +
4.9 list + + + + + +
flush + + + + + +
4.14 list + + + + + +
flush + + + + + +
4.15 list
flush
4.19 list
flush
5.0 list
flush
5.1 list
flush
with list + + + + + +
fix flush + + + +
v7:
- Explain usage of "skip" counters in commit message (suggested by
David Ahern)
v6:
- Rebase onto net-next, use recently introduced nexthop walker
- Make rt6_nh_dump_exceptions() a separate function (suggested by David
Ahern)
v5:
- Use dump_routes and dump_exceptions from filter, ignore NLM_F_MATCH,
update test results (flushing works with iproute2 < 5.0.0 now)
v4:
- Split NLM_F_MATCH and strict check handling in separate patches
- Filter routes using RTM_F_CLONED: if it's not set, only return
non-cached routes, and if it's set, only return cached routes:
change requested by David Ahern and Martin Lau. This implies that
iproute2 needs a separate patch to be able to flush IPv6 cached
routes. This is not ideal because we can't fix the breakage caused
by 2b760fcf5cfb entirely in kernel. However, two years have passed
since then, and this makes it more tolerable
v3:
- More descriptive comment about expired exceptions in rt6_dump_route()
- Swap return values of rt6_dump_route() (suggested by Martin Lau)
- Don't zero skip_in_node in case we don't dump anything in a given pass
(also suggested by Martin Lau)
- Remove check on RTM_F_CLONED altogether: in the current UAPI semantic,
it's just a flag to indicate the route was cloned, not to filter on
routes
v2: Add tracking of number of entries to be skipped in current node after
a partial dump. As we restart from the same node, if not all the
exceptions for a given node fit in a single message, the dump will
not terminate, as suggested by Martin Lau. This is a concrete
possibility, setting up a big number of exceptions for the same route
actually causes the issue, suggested by David Ahern.
Reported-by: Jianlin Shi <jishi@redhat.com>
Fixes: 2b760fcf5cfb ("ipv6: hook up exception table to store dst cache")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-06-21 15:45:27 +00:00
|
|
|
|
|
|
|
if (err)
|
|
|
|
return count += w.count;
|
|
|
|
}
|
2019-06-21 15:45:26 +00:00
|
|
|
|
|
|
|
return -1;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2019-01-18 18:46:24 +00:00
|
|
|
static int inet6_rtm_valid_getroute_req(struct sk_buff *skb,
|
|
|
|
const struct nlmsghdr *nlh,
|
|
|
|
struct nlattr **tb,
|
|
|
|
struct netlink_ext_ack *extack)
|
|
|
|
{
|
|
|
|
struct rtmsg *rtm;
|
|
|
|
int i, err;
|
|
|
|
|
|
|
|
if (nlh->nlmsg_len < nlmsg_msg_size(sizeof(*rtm))) {
|
|
|
|
NL_SET_ERR_MSG_MOD(extack,
|
|
|
|
"Invalid header for get route request");
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (!netlink_strict_get_check(skb))
|
netlink: make validation more configurable for future strictness
We currently have two levels of strict validation:
1) liberal (default)
- undefined (type >= max) & NLA_UNSPEC attributes accepted
- attribute length >= expected accepted
- garbage at end of message accepted
2) strict (opt-in)
- NLA_UNSPEC attributes accepted
- attribute length >= expected accepted
Split out parsing strictness into four different options:
* TRAILING - check that there's no trailing data after parsing
attributes (in message or nested)
* MAXTYPE - reject attrs > max known type
* UNSPEC - reject attributes with NLA_UNSPEC policy entries
* STRICT_ATTRS - strictly validate attribute size
The default for future things should be *everything*.
The current *_strict() is a combination of TRAILING and MAXTYPE,
and is renamed to _deprecated_strict().
The current regular parsing has none of this, and is renamed to
*_parse_deprecated().
Additionally it allows us to selectively set one of the new flags
even on old policies. Notably, the UNSPEC flag could be useful in
this case, since it can be arranged (by filling in the policy) to
not be an incompatible userspace ABI change, but would then going
forward prevent forgetting attribute entries. Similar can apply
to the POLICY flag.
We end up with the following renames:
* nla_parse -> nla_parse_deprecated
* nla_parse_strict -> nla_parse_deprecated_strict
* nlmsg_parse -> nlmsg_parse_deprecated
* nlmsg_parse_strict -> nlmsg_parse_deprecated_strict
* nla_parse_nested -> nla_parse_nested_deprecated
* nla_validate_nested -> nla_validate_nested_deprecated
Using spatch, of course:
@@
expression TB, MAX, HEAD, LEN, POL, EXT;
@@
-nla_parse(TB, MAX, HEAD, LEN, POL, EXT)
+nla_parse_deprecated(TB, MAX, HEAD, LEN, POL, EXT)
@@
expression NLH, HDRLEN, TB, MAX, POL, EXT;
@@
-nlmsg_parse(NLH, HDRLEN, TB, MAX, POL, EXT)
+nlmsg_parse_deprecated(NLH, HDRLEN, TB, MAX, POL, EXT)
@@
expression NLH, HDRLEN, TB, MAX, POL, EXT;
@@
-nlmsg_parse_strict(NLH, HDRLEN, TB, MAX, POL, EXT)
+nlmsg_parse_deprecated_strict(NLH, HDRLEN, TB, MAX, POL, EXT)
@@
expression TB, MAX, NLA, POL, EXT;
@@
-nla_parse_nested(TB, MAX, NLA, POL, EXT)
+nla_parse_nested_deprecated(TB, MAX, NLA, POL, EXT)
@@
expression START, MAX, POL, EXT;
@@
-nla_validate_nested(START, MAX, POL, EXT)
+nla_validate_nested_deprecated(START, MAX, POL, EXT)
@@
expression NLH, HDRLEN, MAX, POL, EXT;
@@
-nlmsg_validate(NLH, HDRLEN, MAX, POL, EXT)
+nlmsg_validate_deprecated(NLH, HDRLEN, MAX, POL, EXT)
For this patch, don't actually add the strict, non-renamed versions
yet so that it breaks compile if I get it wrong.
Also, while at it, make nla_validate and nla_parse go down to a
common __nla_validate_parse() function to avoid code duplication.
Ultimately, this allows us to have very strict validation for every
new caller of nla_parse()/nlmsg_parse() etc as re-introduced in the
next patch, while existing things will continue to work as is.
In effect then, this adds fully strict validation for any new command.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-26 12:07:28 +00:00
|
|
|
return nlmsg_parse_deprecated(nlh, sizeof(*rtm), tb, RTA_MAX,
|
|
|
|
rtm_ipv6_policy, extack);
|
2019-01-18 18:46:24 +00:00
|
|
|
|
|
|
|
rtm = nlmsg_data(nlh);
|
|
|
|
if ((rtm->rtm_src_len && rtm->rtm_src_len != 128) ||
|
|
|
|
(rtm->rtm_dst_len && rtm->rtm_dst_len != 128) ||
|
|
|
|
rtm->rtm_table || rtm->rtm_protocol || rtm->rtm_scope ||
|
|
|
|
rtm->rtm_type) {
|
|
|
|
NL_SET_ERR_MSG_MOD(extack, "Invalid values in header for get route request");
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
if (rtm->rtm_flags & ~RTM_F_FIB_MATCH) {
|
|
|
|
NL_SET_ERR_MSG_MOD(extack,
|
|
|
|
"Invalid flags for get route request");
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
netlink: make validation more configurable for future strictness
We currently have two levels of strict validation:
1) liberal (default)
- undefined (type >= max) & NLA_UNSPEC attributes accepted
- attribute length >= expected accepted
- garbage at end of message accepted
2) strict (opt-in)
- NLA_UNSPEC attributes accepted
- attribute length >= expected accepted
Split out parsing strictness into four different options:
* TRAILING - check that there's no trailing data after parsing
attributes (in message or nested)
* MAXTYPE - reject attrs > max known type
* UNSPEC - reject attributes with NLA_UNSPEC policy entries
* STRICT_ATTRS - strictly validate attribute size
The default for future things should be *everything*.
The current *_strict() is a combination of TRAILING and MAXTYPE,
and is renamed to _deprecated_strict().
The current regular parsing has none of this, and is renamed to
*_parse_deprecated().
Additionally it allows us to selectively set one of the new flags
even on old policies. Notably, the UNSPEC flag could be useful in
this case, since it can be arranged (by filling in the policy) to
not be an incompatible userspace ABI change, but would then going
forward prevent forgetting attribute entries. Similar can apply
to the POLICY flag.
We end up with the following renames:
* nla_parse -> nla_parse_deprecated
* nla_parse_strict -> nla_parse_deprecated_strict
* nlmsg_parse -> nlmsg_parse_deprecated
* nlmsg_parse_strict -> nlmsg_parse_deprecated_strict
* nla_parse_nested -> nla_parse_nested_deprecated
* nla_validate_nested -> nla_validate_nested_deprecated
Using spatch, of course:
@@
expression TB, MAX, HEAD, LEN, POL, EXT;
@@
-nla_parse(TB, MAX, HEAD, LEN, POL, EXT)
+nla_parse_deprecated(TB, MAX, HEAD, LEN, POL, EXT)
@@
expression NLH, HDRLEN, TB, MAX, POL, EXT;
@@
-nlmsg_parse(NLH, HDRLEN, TB, MAX, POL, EXT)
+nlmsg_parse_deprecated(NLH, HDRLEN, TB, MAX, POL, EXT)
@@
expression NLH, HDRLEN, TB, MAX, POL, EXT;
@@
-nlmsg_parse_strict(NLH, HDRLEN, TB, MAX, POL, EXT)
+nlmsg_parse_deprecated_strict(NLH, HDRLEN, TB, MAX, POL, EXT)
@@
expression TB, MAX, NLA, POL, EXT;
@@
-nla_parse_nested(TB, MAX, NLA, POL, EXT)
+nla_parse_nested_deprecated(TB, MAX, NLA, POL, EXT)
@@
expression START, MAX, POL, EXT;
@@
-nla_validate_nested(START, MAX, POL, EXT)
+nla_validate_nested_deprecated(START, MAX, POL, EXT)
@@
expression NLH, HDRLEN, MAX, POL, EXT;
@@
-nlmsg_validate(NLH, HDRLEN, MAX, POL, EXT)
+nlmsg_validate_deprecated(NLH, HDRLEN, MAX, POL, EXT)
For this patch, don't actually add the strict, non-renamed versions
yet so that it breaks compile if I get it wrong.
Also, while at it, make nla_validate and nla_parse go down to a
common __nla_validate_parse() function to avoid code duplication.
Ultimately, this allows us to have very strict validation for every
new caller of nla_parse()/nlmsg_parse() etc as re-introduced in the
next patch, while existing things will continue to work as is.
In effect then, this adds fully strict validation for any new command.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-26 12:07:28 +00:00
|
|
|
err = nlmsg_parse_deprecated_strict(nlh, sizeof(*rtm), tb, RTA_MAX,
|
|
|
|
rtm_ipv6_policy, extack);
|
2019-01-18 18:46:24 +00:00
|
|
|
if (err)
|
|
|
|
return err;
|
|
|
|
|
|
|
|
if ((tb[RTA_SRC] && !rtm->rtm_src_len) ||
|
|
|
|
(tb[RTA_DST] && !rtm->rtm_dst_len)) {
|
|
|
|
NL_SET_ERR_MSG_MOD(extack, "rtm_src_len and rtm_dst_len must be 128 for IPv6");
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
|
|
|
for (i = 0; i <= RTA_MAX; i++) {
|
|
|
|
if (!tb[i])
|
|
|
|
continue;
|
|
|
|
|
|
|
|
switch (i) {
|
|
|
|
case RTA_SRC:
|
|
|
|
case RTA_DST:
|
|
|
|
case RTA_IIF:
|
|
|
|
case RTA_OIF:
|
|
|
|
case RTA_MARK:
|
|
|
|
case RTA_UID:
|
|
|
|
case RTA_SPORT:
|
|
|
|
case RTA_DPORT:
|
|
|
|
case RTA_IP_PROTO:
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
NL_SET_ERR_MSG_MOD(extack, "Unsupported attribute in get route request");
|
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2017-04-16 16:48:24 +00:00
|
|
|
static int inet6_rtm_getroute(struct sk_buff *in_skb, struct nlmsghdr *nlh,
|
|
|
|
struct netlink_ext_ack *extack)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2008-03-25 17:26:21 +00:00
|
|
|
struct net *net = sock_net(in_skb->sk);
|
2006-08-22 07:01:47 +00:00
|
|
|
struct nlattr *tb[RTA_MAX+1];
|
2017-05-25 17:42:40 +00:00
|
|
|
int err, iif = 0, oif = 0;
|
2018-04-20 22:38:02 +00:00
|
|
|
struct fib6_info *from;
|
2017-05-25 17:42:40 +00:00
|
|
|
struct dst_entry *dst;
|
2006-08-22 07:01:47 +00:00
|
|
|
struct rt6_info *rt;
|
2005-04-16 22:20:36 +00:00
|
|
|
struct sk_buff *skb;
|
2006-08-22 07:01:47 +00:00
|
|
|
struct rtmsg *rtm;
|
2018-09-30 06:44:54 +00:00
|
|
|
struct flowi6 fl6 = {};
|
2017-05-25 17:42:40 +00:00
|
|
|
bool fibmatch;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-01-18 18:46:24 +00:00
|
|
|
err = inet6_rtm_valid_getroute_req(in_skb, nlh, tb, extack);
|
2006-08-22 07:01:47 +00:00
|
|
|
if (err < 0)
|
|
|
|
goto errout;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2006-08-22 07:01:47 +00:00
|
|
|
err = -EINVAL;
|
2016-06-11 18:08:19 +00:00
|
|
|
rtm = nlmsg_data(nlh);
|
|
|
|
fl6.flowlabel = ip6_make_flowinfo(rtm->rtm_tos, 0);
|
2017-05-25 17:42:40 +00:00
|
|
|
fibmatch = !!(rtm->rtm_flags & RTM_F_FIB_MATCH);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2006-08-22 07:01:47 +00:00
|
|
|
if (tb[RTA_SRC]) {
|
|
|
|
if (nla_len(tb[RTA_SRC]) < sizeof(struct in6_addr))
|
|
|
|
goto errout;
|
|
|
|
|
2011-11-21 03:39:03 +00:00
|
|
|
fl6.saddr = *(struct in6_addr *)nla_data(tb[RTA_SRC]);
|
2006-08-22 07:01:47 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if (tb[RTA_DST]) {
|
|
|
|
if (nla_len(tb[RTA_DST]) < sizeof(struct in6_addr))
|
|
|
|
goto errout;
|
|
|
|
|
2011-11-21 03:39:03 +00:00
|
|
|
fl6.daddr = *(struct in6_addr *)nla_data(tb[RTA_DST]);
|
2006-08-22 07:01:47 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if (tb[RTA_IIF])
|
|
|
|
iif = nla_get_u32(tb[RTA_IIF]);
|
|
|
|
|
|
|
|
if (tb[RTA_OIF])
|
2012-04-01 04:03:45 +00:00
|
|
|
oif = nla_get_u32(tb[RTA_OIF]);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2014-05-15 23:38:41 +00:00
|
|
|
if (tb[RTA_MARK])
|
|
|
|
fl6.flowi6_mark = nla_get_u32(tb[RTA_MARK]);
|
|
|
|
|
2016-11-03 17:23:42 +00:00
|
|
|
if (tb[RTA_UID])
|
|
|
|
fl6.flowi6_uid = make_kuid(current_user_ns(),
|
|
|
|
nla_get_u32(tb[RTA_UID]));
|
|
|
|
else
|
|
|
|
fl6.flowi6_uid = iif ? INVALID_UID : current_uid();
|
|
|
|
|
2018-05-22 21:03:28 +00:00
|
|
|
if (tb[RTA_SPORT])
|
|
|
|
fl6.fl6_sport = nla_get_be16(tb[RTA_SPORT]);
|
|
|
|
|
|
|
|
if (tb[RTA_DPORT])
|
|
|
|
fl6.fl6_dport = nla_get_be16(tb[RTA_DPORT]);
|
|
|
|
|
|
|
|
if (tb[RTA_IP_PROTO]) {
|
|
|
|
err = rtm_getroute_parse_ip_proto(tb[RTA_IP_PROTO],
|
2019-02-27 08:15:29 +00:00
|
|
|
&fl6.flowi6_proto, AF_INET6,
|
|
|
|
extack);
|
2018-05-22 21:03:28 +00:00
|
|
|
if (err)
|
|
|
|
goto errout;
|
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
if (iif) {
|
|
|
|
struct net_device *dev;
|
2012-04-01 04:03:45 +00:00
|
|
|
int flags = 0;
|
|
|
|
|
2017-08-15 14:34:42 +00:00
|
|
|
rcu_read_lock();
|
|
|
|
|
|
|
|
dev = dev_get_by_index_rcu(net, iif);
|
2005-04-16 22:20:36 +00:00
|
|
|
if (!dev) {
|
2017-08-15 14:34:42 +00:00
|
|
|
rcu_read_unlock();
|
2005-04-16 22:20:36 +00:00
|
|
|
err = -ENODEV;
|
2006-08-22 07:01:47 +00:00
|
|
|
goto errout;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2012-04-01 04:03:45 +00:00
|
|
|
|
|
|
|
fl6.flowi6_iif = iif;
|
|
|
|
|
|
|
|
if (!ipv6_addr_any(&fl6.saddr))
|
|
|
|
flags |= RT6_LOOKUP_F_HAS_SADDR;
|
|
|
|
|
2018-03-02 16:32:17 +00:00
|
|
|
dst = ip6_route_input_lookup(net, dev, &fl6, NULL, flags);
|
2017-08-15 14:34:42 +00:00
|
|
|
|
|
|
|
rcu_read_unlock();
|
2012-04-01 04:03:45 +00:00
|
|
|
} else {
|
|
|
|
fl6.flowi6_oif = oif;
|
|
|
|
|
2017-12-20 10:28:25 +00:00
|
|
|
dst = ip6_route_output(net, NULL, &fl6);
|
2017-05-25 17:42:40 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
rt = container_of(dst, struct rt6_info, dst);
|
|
|
|
if (rt->dst.error) {
|
|
|
|
err = rt->dst.error;
|
|
|
|
ip6_rt_put(rt);
|
|
|
|
goto errout;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2017-03-02 04:48:39 +00:00
|
|
|
if (rt == net->ipv6.ip6_null_entry) {
|
|
|
|
err = rt->dst.error;
|
|
|
|
ip6_rt_put(rt);
|
|
|
|
goto errout;
|
|
|
|
}
|
|
|
|
|
2006-08-22 07:01:47 +00:00
|
|
|
skb = alloc_skb(NLMSG_GOODSIZE, GFP_KERNEL);
|
2011-12-03 23:02:47 +00:00
|
|
|
if (!skb) {
|
2012-10-29 00:13:19 +00:00
|
|
|
ip6_rt_put(rt);
|
2006-08-22 07:01:47 +00:00
|
|
|
err = -ENOBUFS;
|
|
|
|
goto errout;
|
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2010-06-11 06:31:35 +00:00
|
|
|
skb_dst_set(skb, &rt->dst);
|
2018-04-20 22:38:02 +00:00
|
|
|
|
|
|
|
rcu_read_lock();
|
|
|
|
from = rcu_dereference(rt->from);
|
2019-04-30 17:45:12 +00:00
|
|
|
if (from) {
|
|
|
|
if (fibmatch)
|
|
|
|
err = rt6_fill_node(net, skb, from, NULL, NULL, NULL,
|
|
|
|
iif, RTM_NEWROUTE,
|
|
|
|
NETLINK_CB(in_skb).portid,
|
|
|
|
nlh->nlmsg_seq, 0);
|
|
|
|
else
|
|
|
|
err = rt6_fill_node(net, skb, from, dst, &fl6.daddr,
|
|
|
|
&fl6.saddr, iif, RTM_NEWROUTE,
|
|
|
|
NETLINK_CB(in_skb).portid,
|
|
|
|
nlh->nlmsg_seq, 0);
|
|
|
|
} else {
|
|
|
|
err = -ENETUNREACH;
|
|
|
|
}
|
2018-04-20 22:38:02 +00:00
|
|
|
rcu_read_unlock();
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
if (err < 0) {
|
2006-08-22 07:01:47 +00:00
|
|
|
kfree_skb(skb);
|
|
|
|
goto errout;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2012-09-07 20:12:54 +00:00
|
|
|
err = rtnl_unicast(skb, net, NETLINK_CB(in_skb).portid);
|
2006-08-22 07:01:47 +00:00
|
|
|
errout:
|
2005-04-16 22:20:36 +00:00
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
2018-04-18 00:33:26 +00:00
|
|
|
void inet6_rt_notify(int event, struct fib6_info *rt, struct nl_info *info,
|
2015-09-13 17:18:33 +00:00
|
|
|
unsigned int nlm_flags)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
|
|
|
struct sk_buff *skb;
|
2008-03-04 21:47:47 +00:00
|
|
|
struct net *net = info->nl_net;
|
2007-12-13 17:45:12 +00:00
|
|
|
u32 seq;
|
|
|
|
int err;
|
|
|
|
|
|
|
|
err = -ENOBUFS;
|
2011-12-03 23:02:47 +00:00
|
|
|
seq = info->nlh ? info->nlh->nlmsg_seq : 0;
|
2006-08-22 07:01:08 +00:00
|
|
|
|
2015-07-21 08:43:48 +00:00
|
|
|
skb = nlmsg_new(rt6_nlmsg_size(rt), gfp_any());
|
2011-12-03 23:02:47 +00:00
|
|
|
if (!skb)
|
2006-08-15 07:35:24 +00:00
|
|
|
goto errout;
|
|
|
|
|
2018-04-18 00:33:16 +00:00
|
|
|
err = rt6_fill_node(net, skb, rt, NULL, NULL, NULL, 0,
|
|
|
|
event, info->portid, seq, nlm_flags);
|
2007-02-01 07:16:40 +00:00
|
|
|
if (err < 0) {
|
|
|
|
/* -EMSGSIZE implies BUG in rt6_nlmsg_size() */
|
|
|
|
WARN_ON(err == -EMSGSIZE);
|
|
|
|
kfree_skb(skb);
|
|
|
|
goto errout;
|
|
|
|
}
|
2012-09-07 20:12:54 +00:00
|
|
|
rtnl_notify(skb, net, info->portid, RTNLGRP_IPV6_ROUTE,
|
2009-02-25 07:18:28 +00:00
|
|
|
info->nlh, gfp_any());
|
|
|
|
return;
|
2006-08-15 07:35:24 +00:00
|
|
|
errout:
|
|
|
|
if (err < 0)
|
2008-03-04 21:47:47 +00:00
|
|
|
rtnl_set_sk_err(net, RTNLGRP_IPV6_ROUTE, err);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2019-05-22 19:04:41 +00:00
|
|
|
void fib6_rt_update(struct net *net, struct fib6_info *rt,
|
|
|
|
struct nl_info *info)
|
|
|
|
{
|
|
|
|
u32 seq = info->nlh ? info->nlh->nlmsg_seq : 0;
|
|
|
|
struct sk_buff *skb;
|
|
|
|
int err = -ENOBUFS;
|
|
|
|
|
|
|
|
skb = nlmsg_new(rt6_nlmsg_size(rt), gfp_any());
|
|
|
|
if (!skb)
|
|
|
|
goto errout;
|
|
|
|
|
|
|
|
err = rt6_fill_node(net, skb, rt, NULL, NULL, NULL, 0,
|
|
|
|
RTM_NEWROUTE, info->portid, seq, NLM_F_REPLACE);
|
|
|
|
if (err < 0) {
|
|
|
|
/* -EMSGSIZE implies BUG in rt6_nlmsg_size() */
|
|
|
|
WARN_ON(err == -EMSGSIZE);
|
|
|
|
kfree_skb(skb);
|
|
|
|
goto errout;
|
|
|
|
}
|
|
|
|
rtnl_notify(skb, net, info->portid, RTNLGRP_IPV6_ROUTE,
|
|
|
|
info->nlh, gfp_any());
|
|
|
|
return;
|
|
|
|
errout:
|
|
|
|
if (err < 0)
|
|
|
|
rtnl_set_sk_err(net, RTNLGRP_IPV6_ROUTE, err);
|
|
|
|
}
|
|
|
|
|
2021-02-01 19:47:55 +00:00
|
|
|
void fib6_info_hw_flags_set(struct net *net, struct fib6_info *f6i,
|
2021-02-07 08:22:52 +00:00
|
|
|
bool offload, bool trap, bool offload_failed)
|
2021-02-01 19:47:55 +00:00
|
|
|
{
|
|
|
|
struct sk_buff *skb;
|
|
|
|
int err;
|
|
|
|
|
2021-02-07 08:22:52 +00:00
|
|
|
if (f6i->offload == offload && f6i->trap == trap &&
|
|
|
|
f6i->offload_failed == offload_failed)
|
2021-02-01 19:47:55 +00:00
|
|
|
return;
|
|
|
|
|
|
|
|
f6i->offload = offload;
|
|
|
|
f6i->trap = trap;
|
2021-02-07 08:22:53 +00:00
|
|
|
|
|
|
|
/* 2 means send notifications only if offload_failed was changed. */
|
|
|
|
if (net->ipv6.sysctl.fib_notify_on_flag_change == 2 &&
|
|
|
|
f6i->offload_failed == offload_failed)
|
|
|
|
return;
|
|
|
|
|
2021-02-07 08:22:52 +00:00
|
|
|
f6i->offload_failed = offload_failed;
|
2021-02-01 19:47:55 +00:00
|
|
|
|
|
|
|
if (!rcu_access_pointer(f6i->fib6_node))
|
|
|
|
/* The route was removed from the tree, do not send
|
2021-03-26 23:12:41 +00:00
|
|
|
* notification.
|
2021-02-01 19:47:55 +00:00
|
|
|
*/
|
|
|
|
return;
|
|
|
|
|
|
|
|
if (!net->ipv6.sysctl.fib_notify_on_flag_change)
|
|
|
|
return;
|
|
|
|
|
|
|
|
skb = nlmsg_new(rt6_nlmsg_size(f6i), GFP_KERNEL);
|
|
|
|
if (!skb) {
|
|
|
|
err = -ENOBUFS;
|
|
|
|
goto errout;
|
|
|
|
}
|
|
|
|
|
|
|
|
err = rt6_fill_node(net, skb, f6i, NULL, NULL, NULL, 0, RTM_NEWROUTE, 0,
|
|
|
|
0, 0);
|
|
|
|
if (err < 0) {
|
|
|
|
/* -EMSGSIZE implies BUG in rt6_nlmsg_size() */
|
|
|
|
WARN_ON(err == -EMSGSIZE);
|
|
|
|
kfree_skb(skb);
|
|
|
|
goto errout;
|
|
|
|
}
|
|
|
|
|
|
|
|
rtnl_notify(skb, net, 0, RTNLGRP_IPV6_ROUTE, NULL, GFP_KERNEL);
|
|
|
|
return;
|
|
|
|
|
|
|
|
errout:
|
|
|
|
rtnl_set_sk_err(net, RTNLGRP_IPV6_ROUTE, err);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL(fib6_info_hw_flags_set);
|
|
|
|
|
2008-03-04 21:48:30 +00:00
|
|
|
static int ip6_route_dev_notify(struct notifier_block *this,
|
2013-05-28 01:30:21 +00:00
|
|
|
unsigned long event, void *ptr)
|
2008-03-04 21:48:30 +00:00
|
|
|
{
|
2013-05-28 01:30:21 +00:00
|
|
|
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
|
2008-03-25 12:47:49 +00:00
|
|
|
struct net *net = dev_net(dev);
|
2008-03-04 21:48:30 +00:00
|
|
|
|
2017-05-08 17:12:13 +00:00
|
|
|
if (!(dev->flags & IFF_LOOPBACK))
|
|
|
|
return NOTIFY_OK;
|
|
|
|
|
|
|
|
if (event == NETDEV_REGISTER) {
|
2019-05-23 03:27:59 +00:00
|
|
|
net->ipv6.fib6_null_entry->fib6_nh->fib_nh_dev = dev;
|
2010-06-11 06:31:35 +00:00
|
|
|
net->ipv6.ip6_null_entry->dst.dev = dev;
|
2008-03-04 21:48:30 +00:00
|
|
|
net->ipv6.ip6_null_entry->rt6i_idev = in6_dev_get(dev);
|
|
|
|
#ifdef CONFIG_IPV6_MULTIPLE_TABLES
|
2010-06-11 06:31:35 +00:00
|
|
|
net->ipv6.ip6_prohibit_entry->dst.dev = dev;
|
2008-03-04 21:48:30 +00:00
|
|
|
net->ipv6.ip6_prohibit_entry->rt6i_idev = in6_dev_get(dev);
|
2010-06-11 06:31:35 +00:00
|
|
|
net->ipv6.ip6_blk_hole_entry->dst.dev = dev;
|
2008-03-04 21:48:30 +00:00
|
|
|
net->ipv6.ip6_blk_hole_entry->rt6i_idev = in6_dev_get(dev);
|
2017-05-08 17:12:13 +00:00
|
|
|
#endif
|
2017-06-20 18:42:27 +00:00
|
|
|
} else if (event == NETDEV_UNREGISTER &&
|
|
|
|
dev->reg_state != NETREG_UNREGISTERED) {
|
|
|
|
/* NETDEV_UNREGISTER could be fired for multiple times by
|
|
|
|
* netdev_wait_allrefs(). Make sure we only call this once.
|
|
|
|
*/
|
2017-08-15 11:09:51 +00:00
|
|
|
in6_dev_put_clear(&net->ipv6.ip6_null_entry->rt6i_idev);
|
2017-05-08 17:12:13 +00:00
|
|
|
#ifdef CONFIG_IPV6_MULTIPLE_TABLES
|
2017-08-15 11:09:51 +00:00
|
|
|
in6_dev_put_clear(&net->ipv6.ip6_prohibit_entry->rt6i_idev);
|
|
|
|
in6_dev_put_clear(&net->ipv6.ip6_blk_hole_entry->rt6i_idev);
|
2008-03-04 21:48:30 +00:00
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
|
|
|
return NOTIFY_OK;
|
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/*
|
|
|
|
* /proc
|
|
|
|
*/
|
|
|
|
|
|
|
|
#ifdef CONFIG_PROC_FS
|
|
|
|
static int rt6_stats_seq_show(struct seq_file *seq, void *v)
|
|
|
|
{
|
2008-03-04 21:46:23 +00:00
|
|
|
struct net *net = (struct net *)seq->private;
|
2005-04-16 22:20:36 +00:00
|
|
|
seq_printf(seq, "%04x %04x %04x %04x %04x %04x %04x\n",
|
2008-03-04 21:46:23 +00:00
|
|
|
net->ipv6.rt6_stats->fib_nodes,
|
|
|
|
net->ipv6.rt6_stats->fib_route_nodes,
|
2017-10-06 19:06:11 +00:00
|
|
|
atomic_read(&net->ipv6.rt6_stats->fib_rt_alloc),
|
2008-03-04 21:46:23 +00:00
|
|
|
net->ipv6.rt6_stats->fib_rt_entries,
|
|
|
|
net->ipv6.rt6_stats->fib_rt_cache,
|
2010-10-08 06:37:34 +00:00
|
|
|
dst_entries_get_slow(&net->ipv6.ip6_dst_ops),
|
2008-03-04 21:46:23 +00:00
|
|
|
net->ipv6.rt6_stats->fib_discarded_routes);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
#endif /* CONFIG_PROC_FS */
|
|
|
|
|
|
|
|
#ifdef CONFIG_SYSCTL
|
|
|
|
|
2020-04-24 06:43:38 +00:00
|
|
|
static int ipv6_sysctl_rtcache_flush(struct ctl_table *ctl, int write,
|
|
|
|
void *buffer, size_t *lenp, loff_t *ppos)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2011-02-24 19:48:03 +00:00
|
|
|
struct net *net;
|
|
|
|
int delay;
|
2018-12-24 16:30:17 +00:00
|
|
|
int ret;
|
2011-02-24 19:48:03 +00:00
|
|
|
if (!write)
|
2005-04-16 22:20:36 +00:00
|
|
|
return -EINVAL;
|
2011-02-24 19:48:03 +00:00
|
|
|
|
|
|
|
net = (struct net *)ctl->extra1;
|
|
|
|
delay = net->ipv6.sysctl.flush_delay;
|
2018-12-24 16:30:17 +00:00
|
|
|
ret = proc_dointvec(ctl, write, buffer, lenp, ppos);
|
|
|
|
if (ret)
|
|
|
|
return ret;
|
|
|
|
|
2013-08-01 08:04:14 +00:00
|
|
|
fib6_run_gc(delay <= 0 ? 0 : (unsigned long)delay, net, delay > 0);
|
2011-02-24 19:48:03 +00:00
|
|
|
return 0;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2018-10-08 21:06:34 +00:00
|
|
|
static struct ctl_table ipv6_route_table_template[] = {
|
2007-02-09 14:24:49 +00:00
|
|
|
{
|
2021-10-27 08:00:08 +00:00
|
|
|
.procname = "max_size",
|
|
|
|
.data = &init_net.ipv6.sysctl.ip6_rt_max_size,
|
2005-04-16 22:20:36 +00:00
|
|
|
.maxlen = sizeof(int),
|
2021-10-27 08:00:08 +00:00
|
|
|
.mode = 0644,
|
|
|
|
.proc_handler = proc_dointvec,
|
2005-04-16 22:20:36 +00:00
|
|
|
},
|
|
|
|
{
|
|
|
|
.procname = "gc_thresh",
|
2008-03-04 21:48:53 +00:00
|
|
|
.data = &ip6_dst_ops_template.gc_thresh,
|
2005-04-16 22:20:36 +00:00
|
|
|
.maxlen = sizeof(int),
|
|
|
|
.mode = 0644,
|
2008-11-04 02:21:05 +00:00
|
|
|
.proc_handler = proc_dointvec,
|
2005-04-16 22:20:36 +00:00
|
|
|
},
|
|
|
|
{
|
2021-10-27 08:00:08 +00:00
|
|
|
.procname = "flush",
|
|
|
|
.data = &init_net.ipv6.sysctl.flush_delay,
|
2005-04-16 22:20:36 +00:00
|
|
|
.maxlen = sizeof(int),
|
2021-10-27 08:00:08 +00:00
|
|
|
.mode = 0200,
|
|
|
|
.proc_handler = ipv6_sysctl_rtcache_flush
|
2005-04-16 22:20:36 +00:00
|
|
|
},
|
|
|
|
{
|
|
|
|
.procname = "gc_min_interval",
|
2008-01-10 11:01:01 +00:00
|
|
|
.data = &init_net.ipv6.sysctl.ip6_rt_gc_min_interval,
|
2005-04-16 22:20:36 +00:00
|
|
|
.maxlen = sizeof(int),
|
|
|
|
.mode = 0644,
|
2008-11-04 02:21:05 +00:00
|
|
|
.proc_handler = proc_dointvec_jiffies,
|
2005-04-16 22:20:36 +00:00
|
|
|
},
|
|
|
|
{
|
|
|
|
.procname = "gc_timeout",
|
2008-01-10 11:01:01 +00:00
|
|
|
.data = &init_net.ipv6.sysctl.ip6_rt_gc_timeout,
|
2005-04-16 22:20:36 +00:00
|
|
|
.maxlen = sizeof(int),
|
|
|
|
.mode = 0644,
|
2008-11-04 02:21:05 +00:00
|
|
|
.proc_handler = proc_dointvec_jiffies,
|
2005-04-16 22:20:36 +00:00
|
|
|
},
|
|
|
|
{
|
|
|
|
.procname = "gc_interval",
|
2008-01-10 11:01:01 +00:00
|
|
|
.data = &init_net.ipv6.sysctl.ip6_rt_gc_interval,
|
2005-04-16 22:20:36 +00:00
|
|
|
.maxlen = sizeof(int),
|
|
|
|
.mode = 0644,
|
2008-11-04 02:21:05 +00:00
|
|
|
.proc_handler = proc_dointvec_jiffies,
|
2005-04-16 22:20:36 +00:00
|
|
|
},
|
|
|
|
{
|
|
|
|
.procname = "gc_elasticity",
|
2008-01-10 11:01:01 +00:00
|
|
|
.data = &init_net.ipv6.sysctl.ip6_rt_gc_elasticity,
|
2005-04-16 22:20:36 +00:00
|
|
|
.maxlen = sizeof(int),
|
|
|
|
.mode = 0644,
|
2010-08-15 05:42:51 +00:00
|
|
|
.proc_handler = proc_dointvec,
|
2005-04-16 22:20:36 +00:00
|
|
|
},
|
|
|
|
{
|
|
|
|
.procname = "mtu_expires",
|
2008-01-10 11:01:01 +00:00
|
|
|
.data = &init_net.ipv6.sysctl.ip6_rt_mtu_expires,
|
2005-04-16 22:20:36 +00:00
|
|
|
.maxlen = sizeof(int),
|
|
|
|
.mode = 0644,
|
2008-11-04 02:21:05 +00:00
|
|
|
.proc_handler = proc_dointvec_jiffies,
|
2005-04-16 22:20:36 +00:00
|
|
|
},
|
|
|
|
{
|
|
|
|
.procname = "min_adv_mss",
|
2008-01-10 11:01:01 +00:00
|
|
|
.data = &init_net.ipv6.sysctl.ip6_rt_min_advmss,
|
2005-04-16 22:20:36 +00:00
|
|
|
.maxlen = sizeof(int),
|
|
|
|
.mode = 0644,
|
2010-08-15 05:42:51 +00:00
|
|
|
.proc_handler = proc_dointvec,
|
2005-04-16 22:20:36 +00:00
|
|
|
},
|
|
|
|
{
|
|
|
|
.procname = "gc_min_interval_ms",
|
2008-01-10 11:01:01 +00:00
|
|
|
.data = &init_net.ipv6.sysctl.ip6_rt_gc_min_interval,
|
2005-04-16 22:20:36 +00:00
|
|
|
.maxlen = sizeof(int),
|
|
|
|
.mode = 0644,
|
2008-11-04 02:21:05 +00:00
|
|
|
.proc_handler = proc_dointvec_ms_jiffies,
|
2005-04-16 22:20:36 +00:00
|
|
|
},
|
2018-10-12 03:17:21 +00:00
|
|
|
{
|
|
|
|
.procname = "skip_notify_on_dev_down",
|
|
|
|
.data = &init_net.ipv6.sysctl.skip_notify_on_dev_down,
|
|
|
|
.maxlen = sizeof(int),
|
|
|
|
.mode = 0644,
|
2019-06-25 03:08:01 +00:00
|
|
|
.proc_handler = proc_dointvec_minmax,
|
proc/sysctl: add shared variables for range check
In the sysctl code the proc_dointvec_minmax() function is often used to
validate the user supplied value between an allowed range. This
function uses the extra1 and extra2 members from struct ctl_table as
minimum and maximum allowed value.
On sysctl handler declaration, in every source file there are some
readonly variables containing just an integer which address is assigned
to the extra1 and extra2 members, so the sysctl range is enforced.
The special values 0, 1 and INT_MAX are very often used as range
boundary, leading duplication of variables like zero=0, one=1,
int_max=INT_MAX in different source files:
$ git grep -E '\.extra[12].*&(zero|one|int_max)' |wc -l
248
Add a const int array containing the most commonly used values, some
macros to refer more easily to the correct array member, and use them
instead of creating a local one for every object file.
This is the bloat-o-meter output comparing the old and new binary
compiled with the default Fedora config:
# scripts/bloat-o-meter -d vmlinux.o.old vmlinux.o
add/remove: 2/2 grow/shrink: 0/2 up/down: 24/-188 (-164)
Data old new delta
sysctl_vals - 12 +12
__kstrtab_sysctl_vals - 12 +12
max 14 10 -4
int_max 16 - -16
one 68 - -68
zero 128 28 -100
Total: Before=20583249, After=20583085, chg -0.00%
[mcroce@redhat.com: tipc: remove two unused variables]
Link: http://lkml.kernel.org/r/20190530091952.4108-1-mcroce@redhat.com
[akpm@linux-foundation.org: fix net/ipv6/sysctl_net_ipv6.c]
[arnd@arndb.de: proc/sysctl: make firmware loader table conditional]
Link: http://lkml.kernel.org/r/20190617130014.1713870-1-arnd@arndb.de
[akpm@linux-foundation.org: fix fs/eventpoll.c]
Link: http://lkml.kernel.org/r/20190430180111.10688-1-mcroce@redhat.com
Signed-off-by: Matteo Croce <mcroce@redhat.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Aaron Tomlin <atomlin@redhat.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-07-18 22:58:50 +00:00
|
|
|
.extra1 = SYSCTL_ZERO,
|
|
|
|
.extra2 = SYSCTL_ONE,
|
2018-10-12 03:17:21 +00:00
|
|
|
},
|
2009-11-05 21:32:03 +00:00
|
|
|
{ }
|
2005-04-16 22:20:36 +00:00
|
|
|
};
|
|
|
|
|
2010-01-17 03:35:32 +00:00
|
|
|
struct ctl_table * __net_init ipv6_route_sysctl_init(struct net *net)
|
2008-01-10 10:53:43 +00:00
|
|
|
{
|
|
|
|
struct ctl_table *table;
|
|
|
|
|
|
|
|
table = kmemdup(ipv6_route_table_template,
|
|
|
|
sizeof(ipv6_route_table_template),
|
|
|
|
GFP_KERNEL);
|
2008-02-27 15:24:28 +00:00
|
|
|
|
|
|
|
if (table) {
|
2021-10-27 08:00:08 +00:00
|
|
|
table[0].data = &net->ipv6.sysctl.ip6_rt_max_size;
|
2009-08-29 01:34:49 +00:00
|
|
|
table[1].data = &net->ipv6.ip6_dst_ops.gc_thresh;
|
2021-10-27 08:00:08 +00:00
|
|
|
table[2].data = &net->ipv6.sysctl.flush_delay;
|
|
|
|
table[2].extra1 = net;
|
2008-02-27 15:24:28 +00:00
|
|
|
table[3].data = &net->ipv6.sysctl.ip6_rt_gc_min_interval;
|
|
|
|
table[4].data = &net->ipv6.sysctl.ip6_rt_gc_timeout;
|
|
|
|
table[5].data = &net->ipv6.sysctl.ip6_rt_gc_interval;
|
|
|
|
table[6].data = &net->ipv6.sysctl.ip6_rt_gc_elasticity;
|
|
|
|
table[7].data = &net->ipv6.sysctl.ip6_rt_mtu_expires;
|
|
|
|
table[8].data = &net->ipv6.sysctl.ip6_rt_min_advmss;
|
2009-12-19 04:11:03 +00:00
|
|
|
table[9].data = &net->ipv6.sysctl.ip6_rt_gc_min_interval;
|
2018-10-12 03:17:21 +00:00
|
|
|
table[10].data = &net->ipv6.sysctl.skip_notify_on_dev_down;
|
2012-11-16 03:02:59 +00:00
|
|
|
|
|
|
|
/* Don't export sysctls to unprivileged users */
|
|
|
|
if (net->user_ns != &init_user_ns)
|
2021-10-27 08:00:08 +00:00
|
|
|
table[1].procname = NULL;
|
2008-02-27 15:24:28 +00:00
|
|
|
}
|
|
|
|
|
2008-01-10 10:53:43 +00:00
|
|
|
return table;
|
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
#endif
|
|
|
|
|
2010-01-17 03:35:32 +00:00
|
|
|
static int __net_init ip6_route_net_init(struct net *net)
|
2008-03-04 21:45:33 +00:00
|
|
|
{
|
2008-04-21 21:25:23 +00:00
|
|
|
int ret = -ENOMEM;
|
2008-03-04 21:48:30 +00:00
|
|
|
|
2009-08-29 01:34:49 +00:00
|
|
|
memcpy(&net->ipv6.ip6_dst_ops, &ip6_dst_ops_template,
|
|
|
|
sizeof(net->ipv6.ip6_dst_ops));
|
2008-03-04 21:49:23 +00:00
|
|
|
|
2010-10-08 06:37:34 +00:00
|
|
|
if (dst_entries_init(&net->ipv6.ip6_dst_ops) < 0)
|
|
|
|
goto out_ip6_dst_ops;
|
|
|
|
|
2019-05-23 03:27:59 +00:00
|
|
|
net->ipv6.fib6_null_entry = fib6_info_alloc(GFP_KERNEL, true);
|
2018-04-18 00:33:18 +00:00
|
|
|
if (!net->ipv6.fib6_null_entry)
|
|
|
|
goto out_ip6_dst_entries;
|
2019-05-23 03:27:59 +00:00
|
|
|
memcpy(net->ipv6.fib6_null_entry, &fib6_null_entry_template,
|
|
|
|
sizeof(*net->ipv6.fib6_null_entry));
|
2018-04-18 00:33:18 +00:00
|
|
|
|
2008-03-04 21:48:30 +00:00
|
|
|
net->ipv6.ip6_null_entry = kmemdup(&ip6_null_entry_template,
|
|
|
|
sizeof(*net->ipv6.ip6_null_entry),
|
|
|
|
GFP_KERNEL);
|
|
|
|
if (!net->ipv6.ip6_null_entry)
|
2018-04-18 00:33:18 +00:00
|
|
|
goto out_fib6_null_entry;
|
2010-06-11 06:31:35 +00:00
|
|
|
net->ipv6.ip6_null_entry->dst.ops = &net->ipv6.ip6_dst_ops;
|
net: Implement read-only protection and COW'ing of metrics.
Routing metrics are now copy-on-write.
Initially a route entry points it's metrics at a read-only location.
If a routing table entry exists, it will point there. Else it will
point at the all zero metric place-holder called 'dst_default_metrics'.
The writeability state of the metrics is stored in the low bits of the
metrics pointer, we have two bits left to spare if we want to store
more states.
For the initial implementation, COW is implemented simply via kmalloc.
However future enhancements will change this to place the writable
metrics somewhere else, in order to increase sharing. Very likely
this "somewhere else" will be the inetpeer cache.
Note also that this means that metrics updates may transiently fail
if we cannot COW the metrics successfully.
But even by itself, this patch should decrease memory usage and
increase cache locality especially for routing workloads. In those
cases the read-only metric copies stay in place and never get written
to.
TCP workloads where metrics get updated, and those rare cases where
PMTU triggers occur, will take a very slight performance hit. But
that hit will be alleviated when the long-term writable metrics
move to a more sharable location.
Since the metrics storage went from a u32 array of RTAX_MAX entries to
what is essentially a pointer, some retooling of the dst_entry layout
was necessary.
Most importantly, we need to preserve the alignment of the reference
count so that it doesn't share cache lines with the read-mostly state,
as per Eric Dumazet's alignment assertion checks.
The only non-trivial bit here is the move of the 'flags' member into
the writeable cacheline. This is OK since we are always accessing the
flags around the same moment when we made a modification to the
reference count.
Signed-off-by: David S. Miller <davem@davemloft.net>
2011-01-27 04:51:05 +00:00
|
|
|
dst_init_metrics(&net->ipv6.ip6_null_entry->dst,
|
|
|
|
ip6_template_metrics, true);
|
2019-06-21 00:36:38 +00:00
|
|
|
INIT_LIST_HEAD(&net->ipv6.ip6_null_entry->rt6i_uncached);
|
2008-03-04 21:48:30 +00:00
|
|
|
|
|
|
|
#ifdef CONFIG_IPV6_MULTIPLE_TABLES
|
2017-08-08 18:23:49 +00:00
|
|
|
net->ipv6.fib6_has_custom_rules = false;
|
2008-03-04 21:48:30 +00:00
|
|
|
net->ipv6.ip6_prohibit_entry = kmemdup(&ip6_prohibit_entry_template,
|
|
|
|
sizeof(*net->ipv6.ip6_prohibit_entry),
|
|
|
|
GFP_KERNEL);
|
2008-10-07 21:12:10 +00:00
|
|
|
if (!net->ipv6.ip6_prohibit_entry)
|
|
|
|
goto out_ip6_null_entry;
|
2010-06-11 06:31:35 +00:00
|
|
|
net->ipv6.ip6_prohibit_entry->dst.ops = &net->ipv6.ip6_dst_ops;
|
net: Implement read-only protection and COW'ing of metrics.
Routing metrics are now copy-on-write.
Initially a route entry points it's metrics at a read-only location.
If a routing table entry exists, it will point there. Else it will
point at the all zero metric place-holder called 'dst_default_metrics'.
The writeability state of the metrics is stored in the low bits of the
metrics pointer, we have two bits left to spare if we want to store
more states.
For the initial implementation, COW is implemented simply via kmalloc.
However future enhancements will change this to place the writable
metrics somewhere else, in order to increase sharing. Very likely
this "somewhere else" will be the inetpeer cache.
Note also that this means that metrics updates may transiently fail
if we cannot COW the metrics successfully.
But even by itself, this patch should decrease memory usage and
increase cache locality especially for routing workloads. In those
cases the read-only metric copies stay in place and never get written
to.
TCP workloads where metrics get updated, and those rare cases where
PMTU triggers occur, will take a very slight performance hit. But
that hit will be alleviated when the long-term writable metrics
move to a more sharable location.
Since the metrics storage went from a u32 array of RTAX_MAX entries to
what is essentially a pointer, some retooling of the dst_entry layout
was necessary.
Most importantly, we need to preserve the alignment of the reference
count so that it doesn't share cache lines with the read-mostly state,
as per Eric Dumazet's alignment assertion checks.
The only non-trivial bit here is the move of the 'flags' member into
the writeable cacheline. This is OK since we are always accessing the
flags around the same moment when we made a modification to the
reference count.
Signed-off-by: David S. Miller <davem@davemloft.net>
2011-01-27 04:51:05 +00:00
|
|
|
dst_init_metrics(&net->ipv6.ip6_prohibit_entry->dst,
|
|
|
|
ip6_template_metrics, true);
|
2019-06-21 00:36:38 +00:00
|
|
|
INIT_LIST_HEAD(&net->ipv6.ip6_prohibit_entry->rt6i_uncached);
|
2008-03-04 21:48:30 +00:00
|
|
|
|
|
|
|
net->ipv6.ip6_blk_hole_entry = kmemdup(&ip6_blk_hole_entry_template,
|
|
|
|
sizeof(*net->ipv6.ip6_blk_hole_entry),
|
|
|
|
GFP_KERNEL);
|
2008-10-07 21:12:10 +00:00
|
|
|
if (!net->ipv6.ip6_blk_hole_entry)
|
|
|
|
goto out_ip6_prohibit_entry;
|
2010-06-11 06:31:35 +00:00
|
|
|
net->ipv6.ip6_blk_hole_entry->dst.ops = &net->ipv6.ip6_dst_ops;
|
net: Implement read-only protection and COW'ing of metrics.
Routing metrics are now copy-on-write.
Initially a route entry points it's metrics at a read-only location.
If a routing table entry exists, it will point there. Else it will
point at the all zero metric place-holder called 'dst_default_metrics'.
The writeability state of the metrics is stored in the low bits of the
metrics pointer, we have two bits left to spare if we want to store
more states.
For the initial implementation, COW is implemented simply via kmalloc.
However future enhancements will change this to place the writable
metrics somewhere else, in order to increase sharing. Very likely
this "somewhere else" will be the inetpeer cache.
Note also that this means that metrics updates may transiently fail
if we cannot COW the metrics successfully.
But even by itself, this patch should decrease memory usage and
increase cache locality especially for routing workloads. In those
cases the read-only metric copies stay in place and never get written
to.
TCP workloads where metrics get updated, and those rare cases where
PMTU triggers occur, will take a very slight performance hit. But
that hit will be alleviated when the long-term writable metrics
move to a more sharable location.
Since the metrics storage went from a u32 array of RTAX_MAX entries to
what is essentially a pointer, some retooling of the dst_entry layout
was necessary.
Most importantly, we need to preserve the alignment of the reference
count so that it doesn't share cache lines with the read-mostly state,
as per Eric Dumazet's alignment assertion checks.
The only non-trivial bit here is the move of the 'flags' member into
the writeable cacheline. This is OK since we are always accessing the
flags around the same moment when we made a modification to the
reference count.
Signed-off-by: David S. Miller <davem@davemloft.net>
2011-01-27 04:51:05 +00:00
|
|
|
dst_init_metrics(&net->ipv6.ip6_blk_hole_entry->dst,
|
|
|
|
ip6_template_metrics, true);
|
2019-06-21 00:36:38 +00:00
|
|
|
INIT_LIST_HEAD(&net->ipv6.ip6_blk_hole_entry->rt6i_uncached);
|
2019-11-20 12:47:34 +00:00
|
|
|
#ifdef CONFIG_IPV6_SUBTREES
|
|
|
|
net->ipv6.fib6_routes_require_src = 0;
|
|
|
|
#endif
|
2008-03-04 21:48:30 +00:00
|
|
|
#endif
|
|
|
|
|
2008-10-07 21:15:00 +00:00
|
|
|
net->ipv6.sysctl.flush_delay = 0;
|
|
|
|
net->ipv6.sysctl.ip6_rt_max_size = 4096;
|
|
|
|
net->ipv6.sysctl.ip6_rt_gc_min_interval = HZ / 2;
|
|
|
|
net->ipv6.sysctl.ip6_rt_gc_timeout = 60*HZ;
|
|
|
|
net->ipv6.sysctl.ip6_rt_gc_interval = 30*HZ;
|
|
|
|
net->ipv6.sysctl.ip6_rt_gc_elasticity = 9;
|
|
|
|
net->ipv6.sysctl.ip6_rt_mtu_expires = 10*60*HZ;
|
|
|
|
net->ipv6.sysctl.ip6_rt_min_advmss = IPV6_MIN_MTU - 20 - 40;
|
2018-10-12 03:17:21 +00:00
|
|
|
net->ipv6.sysctl.skip_notify_on_dev_down = 0;
|
2008-10-07 21:15:00 +00:00
|
|
|
|
2008-03-04 21:49:47 +00:00
|
|
|
net->ipv6.ip6_rt_gc_expire = 30*HZ;
|
|
|
|
|
2008-03-04 21:48:30 +00:00
|
|
|
ret = 0;
|
|
|
|
out:
|
|
|
|
return ret;
|
2008-03-04 21:49:23 +00:00
|
|
|
|
2008-10-07 21:12:10 +00:00
|
|
|
#ifdef CONFIG_IPV6_MULTIPLE_TABLES
|
|
|
|
out_ip6_prohibit_entry:
|
|
|
|
kfree(net->ipv6.ip6_prohibit_entry);
|
|
|
|
out_ip6_null_entry:
|
|
|
|
kfree(net->ipv6.ip6_null_entry);
|
|
|
|
#endif
|
2018-04-18 00:33:18 +00:00
|
|
|
out_fib6_null_entry:
|
|
|
|
kfree(net->ipv6.fib6_null_entry);
|
2010-10-08 06:37:34 +00:00
|
|
|
out_ip6_dst_entries:
|
|
|
|
dst_entries_destroy(&net->ipv6.ip6_dst_ops);
|
2008-03-04 21:49:23 +00:00
|
|
|
out_ip6_dst_ops:
|
|
|
|
goto out;
|
2008-03-04 21:45:33 +00:00
|
|
|
}
|
|
|
|
|
2010-01-17 03:35:32 +00:00
|
|
|
static void __net_exit ip6_route_net_exit(struct net *net)
|
2008-03-04 21:45:33 +00:00
|
|
|
{
|
2018-04-18 00:33:18 +00:00
|
|
|
kfree(net->ipv6.fib6_null_entry);
|
2008-03-04 21:48:30 +00:00
|
|
|
kfree(net->ipv6.ip6_null_entry);
|
|
|
|
#ifdef CONFIG_IPV6_MULTIPLE_TABLES
|
|
|
|
kfree(net->ipv6.ip6_prohibit_entry);
|
|
|
|
kfree(net->ipv6.ip6_blk_hole_entry);
|
|
|
|
#endif
|
net dst: fix percpu_counter list corruption and poison overwritten
There're some percpu_counter list corruption and poison overwritten warnings
in recent kernel, which is resulted by fc66f95c.
commit fc66f95c switches to use percpu_counter, in ip6_route_net_init, kernel
init the percpu_counter for dst entries, but, the percpu_counter is never destroyed
in ip6_route_net_exit. So if the related data is freed by kernel, the freed percpu_counter
is still on the list, then if we insert/remove other percpu_counter, list corruption
resulted. Also, if the insert/remove option modifies the ->prev,->next pointer of
the freed value, the poison overwritten is resulted then.
With the following patch, the percpu_counter list corruption and poison overwritten
warnings disappeared.
Signed-off-by: Xiaotian Feng <dfeng@redhat.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>
Cc: "Pekka Savola (ipv6)" <pekkas@netcore.fi>
Cc: James Morris <jmorris@namei.org>
Cc: Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>
Cc: Patrick McHardy <kaber@trash.net>
Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2010-11-02 16:11:05 +00:00
|
|
|
dst_entries_destroy(&net->ipv6.ip6_dst_ops);
|
2008-03-04 21:45:33 +00:00
|
|
|
}
|
|
|
|
|
2012-06-18 12:08:33 +00:00
|
|
|
static int __net_init ip6_route_net_init_late(struct net *net)
|
|
|
|
{
|
|
|
|
#ifdef CONFIG_PROC_FS
|
2018-04-10 17:42:55 +00:00
|
|
|
proc_create_net("ipv6_route", 0, net->proc_net, &ipv6_route_seq_ops,
|
|
|
|
sizeof(struct ipv6_route_iter));
|
2018-04-13 18:38:35 +00:00
|
|
|
proc_create_net_single("rt6_stats", 0444, net->proc_net,
|
|
|
|
rt6_stats_seq_show, NULL);
|
2012-06-18 12:08:33 +00:00
|
|
|
#endif
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static void __net_exit ip6_route_net_exit_late(struct net *net)
|
|
|
|
{
|
|
|
|
#ifdef CONFIG_PROC_FS
|
2013-02-18 01:34:56 +00:00
|
|
|
remove_proc_entry("ipv6_route", net->proc_net);
|
|
|
|
remove_proc_entry("rt6_stats", net->proc_net);
|
2012-06-18 12:08:33 +00:00
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
2008-03-04 21:45:33 +00:00
|
|
|
static struct pernet_operations ip6_route_net_ops = {
|
|
|
|
.init = ip6_route_net_init,
|
|
|
|
.exit = ip6_route_net_exit,
|
|
|
|
};
|
|
|
|
|
2012-06-09 23:27:05 +00:00
|
|
|
static int __net_init ipv6_inetpeer_init(struct net *net)
|
|
|
|
{
|
|
|
|
struct inet_peer_base *bp = kmalloc(sizeof(*bp), GFP_KERNEL);
|
|
|
|
|
|
|
|
if (!bp)
|
|
|
|
return -ENOMEM;
|
|
|
|
inet_peer_base_init(bp);
|
|
|
|
net->ipv6.peers = bp;
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static void __net_exit ipv6_inetpeer_exit(struct net *net)
|
|
|
|
{
|
|
|
|
struct inet_peer_base *bp = net->ipv6.peers;
|
|
|
|
|
|
|
|
net->ipv6.peers = NULL;
|
2012-06-09 23:32:41 +00:00
|
|
|
inetpeer_invalidate_tree(bp);
|
2012-06-09 23:27:05 +00:00
|
|
|
kfree(bp);
|
|
|
|
}
|
|
|
|
|
2012-06-10 02:00:16 +00:00
|
|
|
static struct pernet_operations ipv6_inetpeer_ops = {
|
2012-06-09 23:27:05 +00:00
|
|
|
.init = ipv6_inetpeer_init,
|
|
|
|
.exit = ipv6_inetpeer_exit,
|
|
|
|
};
|
|
|
|
|
2012-06-18 12:08:33 +00:00
|
|
|
static struct pernet_operations ip6_route_net_late_ops = {
|
|
|
|
.init = ip6_route_net_init_late,
|
|
|
|
.exit = ip6_route_net_exit_late,
|
|
|
|
};
|
|
|
|
|
2008-03-04 21:48:30 +00:00
|
|
|
static struct notifier_block ip6_route_dev_notifier = {
|
|
|
|
.notifier_call = ip6_route_dev_notify,
|
2017-05-08 17:12:13 +00:00
|
|
|
.priority = ADDRCONF_NOTIFY_PRIORITY - 10,
|
2008-03-04 21:48:30 +00:00
|
|
|
};
|
|
|
|
|
2017-05-04 05:07:31 +00:00
|
|
|
void __init ip6_route_init_special_entries(void)
|
|
|
|
{
|
|
|
|
/* Registering of the loopback is done before this portion of code,
|
|
|
|
* the loopback reference in rt6_info will not be taken, do it
|
|
|
|
* manually for init_net */
|
2019-05-23 03:27:59 +00:00
|
|
|
init_net.ipv6.fib6_null_entry->fib6_nh->fib_nh_dev = init_net.loopback_dev;
|
2017-05-04 05:07:31 +00:00
|
|
|
init_net.ipv6.ip6_null_entry->dst.dev = init_net.loopback_dev;
|
|
|
|
init_net.ipv6.ip6_null_entry->rt6i_idev = in6_dev_get(init_net.loopback_dev);
|
|
|
|
#ifdef CONFIG_IPV6_MULTIPLE_TABLES
|
|
|
|
init_net.ipv6.ip6_prohibit_entry->dst.dev = init_net.loopback_dev;
|
|
|
|
init_net.ipv6.ip6_prohibit_entry->rt6i_idev = in6_dev_get(init_net.loopback_dev);
|
|
|
|
init_net.ipv6.ip6_blk_hole_entry->dst.dev = init_net.loopback_dev;
|
|
|
|
init_net.ipv6.ip6_blk_hole_entry->rt6i_idev = in6_dev_get(init_net.loopback_dev);
|
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
2020-05-09 17:59:10 +00:00
|
|
|
#if IS_BUILTIN(CONFIG_IPV6)
|
|
|
|
#if defined(CONFIG_BPF_SYSCALL) && defined(CONFIG_PROC_FS)
|
|
|
|
DEFINE_BPF_ITER_FUNC(ipv6_route, struct bpf_iter_meta *meta, struct fib6_info *rt)
|
|
|
|
|
2020-07-20 16:34:03 +00:00
|
|
|
BTF_ID_LIST(btf_fib6_info_id)
|
|
|
|
BTF_ID(struct, fib6_info)
|
|
|
|
|
2020-07-23 18:41:09 +00:00
|
|
|
static const struct bpf_iter_seq_info ipv6_route_seq_info = {
|
2020-05-13 18:02:19 +00:00
|
|
|
.seq_ops = &ipv6_route_seq_ops,
|
|
|
|
.init_seq_private = bpf_iter_init_seq_net,
|
|
|
|
.fini_seq_private = bpf_iter_fini_seq_net,
|
|
|
|
.seq_priv_size = sizeof(struct ipv6_route_iter),
|
2020-07-23 18:41:09 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
static struct bpf_iter_reg ipv6_route_reg_info = {
|
|
|
|
.target = "ipv6_route",
|
2020-05-13 18:02:21 +00:00
|
|
|
.ctx_arg_info_size = 1,
|
|
|
|
.ctx_arg_info = {
|
|
|
|
{ offsetof(struct bpf_iter__ipv6_route, rt),
|
|
|
|
PTR_TO_BTF_ID_OR_NULL },
|
|
|
|
},
|
2020-07-23 18:41:09 +00:00
|
|
|
.seq_info = &ipv6_route_seq_info,
|
2020-05-13 18:02:19 +00:00
|
|
|
};
|
|
|
|
|
2020-05-09 17:59:10 +00:00
|
|
|
static int __init bpf_iter_register(void)
|
|
|
|
{
|
2020-07-20 16:34:03 +00:00
|
|
|
ipv6_route_reg_info.ctx_arg_info[0].btf_id = *btf_fib6_info_id;
|
2020-05-13 18:02:19 +00:00
|
|
|
return bpf_iter_reg_target(&ipv6_route_reg_info);
|
2020-05-09 17:59:10 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static void bpf_iter_unregister(void)
|
|
|
|
{
|
2020-05-13 18:02:20 +00:00
|
|
|
bpf_iter_unreg_target(&ipv6_route_reg_info);
|
2020-05-09 17:59:10 +00:00
|
|
|
}
|
|
|
|
#endif
|
|
|
|
#endif
|
|
|
|
|
2007-12-07 08:43:48 +00:00
|
|
|
int __init ip6_route_init(void)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2007-12-07 08:43:48 +00:00
|
|
|
int ret;
|
2015-05-23 03:56:04 +00:00
|
|
|
int cpu;
|
2007-12-07 08:43:48 +00:00
|
|
|
|
2008-03-04 21:48:53 +00:00
|
|
|
ret = -ENOMEM;
|
|
|
|
ip6_dst_ops_template.kmem_cachep =
|
2006-08-27 02:25:52 +00:00
|
|
|
kmem_cache_create("ip6_dst_cache", sizeof(struct rt6_info), 0,
|
memcg: enable accounting for IP address and routing-related objects
An netadmin inside container can use 'ip a a' and 'ip r a'
to assign a large number of ipv4/ipv6 addresses and routing entries
and force kernel to allocate megabytes of unaccounted memory
for long-lived per-netdevice related kernel objects:
'struct in_ifaddr', 'struct inet6_ifaddr', 'struct fib6_node',
'struct rt6_info', 'struct fib_rules' and ip_fib caches.
These objects can be manually removed, though usually they lives
in memory till destroy of its net namespace.
It makes sense to account for them to restrict the host's memory
consumption from inside the memcg-limited container.
One of such objects is the 'struct fib6_node' mostly allocated in
net/ipv6/route.c::__ip6_ins_rt() inside the lock_bh()/unlock_bh() section:
write_lock_bh(&table->tb6_lock);
err = fib6_add(&table->tb6_root, rt, info, mxc);
write_unlock_bh(&table->tb6_lock);
In this case it is not enough to simply add SLAB_ACCOUNT to corresponding
kmem cache. The proper memory cgroup still cannot be found due to the
incorrect 'in_interrupt()' check used in memcg_kmem_bypass().
Obsoleted in_interrupt() does not describe real execution context properly.
>From include/linux/preempt.h:
The following macros are deprecated and should not be used in new code:
in_interrupt() - We're in NMI,IRQ,SoftIRQ context or have BH disabled
To verify the current execution context new macro should be used instead:
in_task() - We're in task context
Signed-off-by: Vasily Averin <vvs@virtuozzo.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2021-07-19 10:44:31 +00:00
|
|
|
SLAB_HWCACHE_ALIGN | SLAB_ACCOUNT, NULL);
|
2008-03-04 21:48:53 +00:00
|
|
|
if (!ip6_dst_ops_template.kmem_cachep)
|
2009-01-08 02:09:08 +00:00
|
|
|
goto out;
|
2007-05-25 01:17:54 +00:00
|
|
|
|
2010-10-08 06:37:34 +00:00
|
|
|
ret = dst_entries_init(&ip6_dst_blackhole_ops);
|
2008-03-04 21:48:30 +00:00
|
|
|
if (ret)
|
2008-03-04 21:48:10 +00:00
|
|
|
goto out_kmem_cache;
|
|
|
|
|
2012-06-09 23:27:05 +00:00
|
|
|
ret = register_pernet_subsys(&ipv6_inetpeer_ops);
|
|
|
|
if (ret)
|
2012-06-16 08:12:19 +00:00
|
|
|
goto out_dst_entries;
|
2012-06-14 23:00:17 +00:00
|
|
|
|
2012-06-15 22:51:55 +00:00
|
|
|
ret = register_pernet_subsys(&ip6_route_net_ops);
|
|
|
|
if (ret)
|
|
|
|
goto out_register_inetpeer;
|
2012-06-09 23:27:05 +00:00
|
|
|
|
2008-10-01 09:37:56 +00:00
|
|
|
ip6_dst_blackhole_ops.kmem_cachep = ip6_dst_ops_template.kmem_cachep;
|
|
|
|
|
2012-06-16 08:12:19 +00:00
|
|
|
ret = fib6_init();
|
2007-12-07 08:43:48 +00:00
|
|
|
if (ret)
|
2008-03-04 21:48:30 +00:00
|
|
|
goto out_register_subsys;
|
2007-12-07 08:43:48 +00:00
|
|
|
|
|
|
|
ret = xfrm6_init();
|
|
|
|
if (ret)
|
2012-06-16 08:12:19 +00:00
|
|
|
goto out_fib6_init;
|
2007-12-08 08:14:11 +00:00
|
|
|
|
2007-12-07 08:43:48 +00:00
|
|
|
ret = fib6_rules_init();
|
|
|
|
if (ret)
|
|
|
|
goto xfrm6_init;
|
2007-12-08 08:14:54 +00:00
|
|
|
|
2012-06-18 12:08:33 +00:00
|
|
|
ret = register_pernet_subsys(&ip6_route_net_late_ops);
|
|
|
|
if (ret)
|
|
|
|
goto fib6_rules_init;
|
|
|
|
|
2017-12-02 20:44:08 +00:00
|
|
|
ret = rtnl_register_module(THIS_MODULE, PF_INET6, RTM_NEWROUTE,
|
|
|
|
inet6_rtm_newroute, NULL, 0);
|
|
|
|
if (ret < 0)
|
|
|
|
goto out_register_late_subsys;
|
|
|
|
|
|
|
|
ret = rtnl_register_module(THIS_MODULE, PF_INET6, RTM_DELROUTE,
|
|
|
|
inet6_rtm_delroute, NULL, 0);
|
|
|
|
if (ret < 0)
|
|
|
|
goto out_register_late_subsys;
|
|
|
|
|
|
|
|
ret = rtnl_register_module(THIS_MODULE, PF_INET6, RTM_GETROUTE,
|
|
|
|
inet6_rtm_getroute, NULL,
|
|
|
|
RTNL_FLAG_DOIT_UNLOCKED);
|
|
|
|
if (ret < 0)
|
2012-06-18 12:08:33 +00:00
|
|
|
goto out_register_late_subsys;
|
2007-03-22 18:58:32 +00:00
|
|
|
|
2008-03-04 21:48:30 +00:00
|
|
|
ret = register_netdevice_notifier(&ip6_route_dev_notifier);
|
2008-03-04 21:45:33 +00:00
|
|
|
if (ret)
|
2012-06-18 12:08:33 +00:00
|
|
|
goto out_register_late_subsys;
|
2008-03-04 21:48:30 +00:00
|
|
|
|
2020-05-09 17:59:10 +00:00
|
|
|
#if IS_BUILTIN(CONFIG_IPV6)
|
|
|
|
#if defined(CONFIG_BPF_SYSCALL) && defined(CONFIG_PROC_FS)
|
|
|
|
ret = bpf_iter_register();
|
|
|
|
if (ret)
|
|
|
|
goto out_register_late_subsys;
|
|
|
|
#endif
|
|
|
|
#endif
|
|
|
|
|
2015-05-23 03:56:04 +00:00
|
|
|
for_each_possible_cpu(cpu) {
|
|
|
|
struct uncached_list *ul = per_cpu_ptr(&rt6_uncached_list, cpu);
|
|
|
|
|
|
|
|
INIT_LIST_HEAD(&ul->head);
|
|
|
|
spin_lock_init(&ul->lock);
|
|
|
|
}
|
|
|
|
|
2007-12-07 08:43:48 +00:00
|
|
|
out:
|
|
|
|
return ret;
|
|
|
|
|
2012-06-18 12:08:33 +00:00
|
|
|
out_register_late_subsys:
|
2017-12-02 20:44:08 +00:00
|
|
|
rtnl_unregister_all(PF_INET6);
|
2012-06-18 12:08:33 +00:00
|
|
|
unregister_pernet_subsys(&ip6_route_net_late_ops);
|
2007-12-07 08:43:48 +00:00
|
|
|
fib6_rules_init:
|
|
|
|
fib6_rules_cleanup();
|
|
|
|
xfrm6_init:
|
|
|
|
xfrm6_fini();
|
2012-06-14 23:00:17 +00:00
|
|
|
out_fib6_init:
|
|
|
|
fib6_gc_cleanup();
|
2008-03-04 21:48:30 +00:00
|
|
|
out_register_subsys:
|
|
|
|
unregister_pernet_subsys(&ip6_route_net_ops);
|
2012-06-15 22:51:55 +00:00
|
|
|
out_register_inetpeer:
|
|
|
|
unregister_pernet_subsys(&ipv6_inetpeer_ops);
|
2010-10-08 06:37:34 +00:00
|
|
|
out_dst_entries:
|
|
|
|
dst_entries_destroy(&ip6_dst_blackhole_ops);
|
2007-12-07 08:43:48 +00:00
|
|
|
out_kmem_cache:
|
2008-03-04 21:49:23 +00:00
|
|
|
kmem_cache_destroy(ip6_dst_ops_template.kmem_cachep);
|
2007-12-07 08:43:48 +00:00
|
|
|
goto out;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
void ip6_route_cleanup(void)
|
|
|
|
{
|
2020-05-09 17:59:10 +00:00
|
|
|
#if IS_BUILTIN(CONFIG_IPV6)
|
|
|
|
#if defined(CONFIG_BPF_SYSCALL) && defined(CONFIG_PROC_FS)
|
|
|
|
bpf_iter_unregister();
|
|
|
|
#endif
|
|
|
|
#endif
|
2008-03-04 21:48:30 +00:00
|
|
|
unregister_netdevice_notifier(&ip6_route_dev_notifier);
|
2012-06-18 12:08:33 +00:00
|
|
|
unregister_pernet_subsys(&ip6_route_net_late_ops);
|
2006-08-04 10:39:02 +00:00
|
|
|
fib6_rules_cleanup();
|
2005-04-16 22:20:36 +00:00
|
|
|
xfrm6_fini();
|
|
|
|
fib6_gc_cleanup();
|
2012-06-09 23:27:05 +00:00
|
|
|
unregister_pernet_subsys(&ipv6_inetpeer_ops);
|
2008-03-04 21:48:30 +00:00
|
|
|
unregister_pernet_subsys(&ip6_route_net_ops);
|
net dst: fix percpu_counter list corruption and poison overwritten
There're some percpu_counter list corruption and poison overwritten warnings
in recent kernel, which is resulted by fc66f95c.
commit fc66f95c switches to use percpu_counter, in ip6_route_net_init, kernel
init the percpu_counter for dst entries, but, the percpu_counter is never destroyed
in ip6_route_net_exit. So if the related data is freed by kernel, the freed percpu_counter
is still on the list, then if we insert/remove other percpu_counter, list corruption
resulted. Also, if the insert/remove option modifies the ->prev,->next pointer of
the freed value, the poison overwritten is resulted then.
With the following patch, the percpu_counter list corruption and poison overwritten
warnings disappeared.
Signed-off-by: Xiaotian Feng <dfeng@redhat.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>
Cc: "Pekka Savola (ipv6)" <pekkas@netcore.fi>
Cc: James Morris <jmorris@namei.org>
Cc: Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>
Cc: Patrick McHardy <kaber@trash.net>
Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2010-11-02 16:11:05 +00:00
|
|
|
dst_entries_destroy(&ip6_dst_blackhole_ops);
|
2008-03-04 21:49:23 +00:00
|
|
|
kmem_cache_destroy(ip6_dst_ops_template.kmem_cachep);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|