Add a driver for the SDIO/MMC host found on the Amlogic Meson SoCs. This
is an MMC controller which provides an interface between the application
processor and various memory cards. It supports the SD specification
v2.0 and the eMMC specification v4.41.
The controller provides an internal "mux" which allows connecting up to
three MMC devices to it. Only one device can be used at a time though
since the registers are shared across all devices. The driver takes care
of synchronizing access (similar to the dw_mmc driver).
The maximum supported bus-width is 4-bits.
Amlogic's GPL kernel sources call the corresponding driver "aml_sdio" to
differentiate it from the other MMC controller in (at least the Meson8
and Meson8b) the SoCs (they call the other drivers aml_sdhc and
aml_sdhc_m8, which seem to support a bus-width of up to 8-bits). This
means that there are three different MMC host controller IP blocks from
Amlogic (each of them with completely own register layout and features):
- "SDIO": 1 and 4 bit bus width, support for high-speed modes up to
UHS-I SDR50, part of Meson6, Meson8 and Meson8b (the driver from this
patch targets this controller)
- "SDHC": 1, 4 and 8 bit bus width, compatible with standard iNAND
interface, support for speeds up to HS200 and MMC spec up to version
4.5x, part of Meson8 and Meson8b SoCs (there is no mainline driver
for this controller yet)
- "SDEMMC": 1, 4 and 8 bit bus width, support for speeds up to HS400
and MMC spec up to version 5.0, part of the Meson GX (64-bit) SoCs
(supported by the meson-gx MMC host driver)
Signed-off-by: Carlo Caione <carlo@endlessm.com>
Signed-off-by: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
This documents the devicetree bindings for the SDIO/MMC host found in
Amlogic Meson8 and Meson8b SoCs. It supports the SD specification v2.0
and the eMMC specification v4.41.
It has an internal "mux" which allows connecting up to three MMC devices
to it. The maximum supported bus-width is 4-bits.
Amlogic's GPL kernel sources call it "SDIO" to differentiate it from the
other MMC controller in (at least the Meson8 and Meson8b) the SoCs (they
call the other one "SDHC", which supports a bus-width of up to 8-bits).
Signed-off-by: Carlo Caione <carlo@endlessm.com>
Signed-off-by: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
The function sdhci_at91_set_uhs_signaling is local to the source and does
not need to be in global scope, so make it static.
Cleans up sparse warning:
symbol 'sdhci_at91_set_uhs_signaling' was not declared. Should it be
static?
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
The array hs_timing_cfg is local to the source and does not need to
be in global scope, so make it static.
Cleans up sparse warning:
symbol 'hs_timing_cfg' was not declared. Should it be static?
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Support for non-dt based initialization for Exynos SoCs has been removed,
so there is no need to keep driver IDs for this case. While touching this,
replace odd conditional code for instantiating driver data for Exynos4
SoCs with a simple reference and move that driver data under CONFIG_OF.
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Acked-by: Krzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Without the ThunderX/OcteonTx GPIO driver the MMC driver
would not power up any MMC devices. Therefore add a
dependency to the GPIO driver and remove the unneeded GPIOLIB
dependency.
Signed-off-by: Jan Glauber <jglauber@cavium.com>
Acked-by: David Daney <david.daney@cavium.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Register writes which change voltage of IO lines or turn the IO bus
on/off require controller to be ready before progressing further. When
the controller is ready, it will generate a power irq which needs to be
handled. The thread which initiated the register write should wait for
power irq to complete. This will be done through the new sdhc msm write
APIs which will check whether the particular write can trigger a power
irq and wait for it with a timeout if it is expected.
The SDHC core power control IRQ gets triggered when -
* There is a state change in power control bit (bit 0)
of SDHCI_POWER_CONTROL register.
* There is a state change in 1.8V enable bit (bit 3) of
SDHCI_HOST_CONTROL2 register.
* Bit 1 of SDHCI_SOFTWARE_RESET is set.
Also add support APIs which are used by sdhc msm write APIs to check
if power irq is expected to be generated and wait for the power irq
to come and complete if the irq is expected.
This patch requires CONFIG_MMC_SDHCI_IO_ACCESSORS to be enabled.
Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Signed-off-by: Vijay Viswanath <vviswana@codeaurora.org>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Enable CONFIG_MMC_SDHCI_IO_ACCESSORS so that SDHC controller specific
register read and write APIs, if registered, can be used.
Signed-off-by: Vijay Viswanath <vviswana@codeaurora.org>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
There is a rare scenario in HW, where the first clear pulse could
be lost when the actual reset and clear/read of status register
are happening at the same time. Fix this by retrying upto 10 times
to ensure the status register gets cleared. Otherwise, this will
lead to a spurious power IRQ which results in system instability.
Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Signed-off-by: Vijay Viswanath <vviswana@codeaurora.org>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
SDCC controller reset (SW_RST) during probe may trigger power irq if
previous status of PWRCTL was either BUS_ON or IO_HIGH_V. So before we
enable the power irq interrupt in GIC (by registering the interrupt
handler), we need to ensure that any pending power irq interrupt status
is acknowledged otherwise power irq interrupt handler would be fired
prematurely.
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Vijay Viswanath <vviswana@codeaurora.org>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Some boards have SD card connectors where the power rail cannot be switched
off by the driver. However there are various circumstances when a card
might be re-initialized, such as after system resume, warm re-boot, or
error handling. However, a UHS card will continue to use 1.8V signaling
unless it is power cycled.
If the card has not been power cycled, it may still be using 1.8V
signaling. According to the SD spec., the Bus Speed Mode (function group 1)
bits 2 to 4 are zero if the card is initialized at 3.3V signal level. Thus
they can be used to determine if the card has already switched to 1.8V
signaling. Detect that situation and try to initialize a UHS-I (1.8V)
transfer mode.
Tested with the following cards:
Transcend 4GB High Speed
Kingston 64GB SDR104
Lexar by Micron HIGH-PERFORMANCE 300x 16GB DDR50
SanDisk Ultra 8GB DDR50
Transcend Ultimate 600x 16GB SDR104
Transcend Premium 300x 64GB SDR104
Lexar by Micron Professional 1000x 32GB UHS-II SDR104
SanDisk Extreme Pro 16GB SDR104
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Zhoujie Wu <zjwu@marvell.com>
Reviewed-by: Shawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Factor out mmc_host_set_uhs_voltage() so it can be reused.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
The following functions are needed by the mmc block device driver, once it
converts to blkmq, therefore let's export them.
mmc_start_bkops()
mmc_start_request()
mmc_retune_hold_now()
mmc_retune_release()
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Don't populate the const arrays mszs on the stack, instead make them
static. Makes the object code smaller by over 310 bytes:
Before:
text data bss dec hex filename
47527 8528 320 56375 dc37 drivers/mmc/host/dw_mmc.o
After:
text data bss dec hex filename
47055 8688 320 56063 daff drivers/mmc/host/dw_mmc.o
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Since commit 3fc7eaef44 ("mmc: dw_mmc: Add external dma interface
support") use_dma no longer means only the data transfer mode, and
includes dma transmission channel. So make it more clear.
Signed-off-by: Ziyuan <ziyuan.biubiu@gmail.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Use setup_timer function instead of initializing timer with the
function and data fields.
Signed-off-by: Allen Pais <allen.lkml@gmail.com>
Reviewed-by: Shawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Use setup_timer function instead of initializing timer with the
function and data fields.
Signed-off-by: Allen Pais <allen.lkml@gmail.com>
Reviewed-by: Shawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Use setup_timer function instead of initializing timer with the
function and data fields.
Signed-off-by: Allen Pais <allen.lkml@gmail.com>
Reviewed-by: Shawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Use setup_timer function instead of initializing timer with the
function and data fields.
Signed-off-by: Allen Pais <allen.lkml@gmail.com>
Reviewed-by: Shawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
On gen3 PCI-Express we should send command one by one.
If sending many commands in one packet will lead to a failure.
Signed-off-by: rui_feng <rui_feng@realsil.com.cn>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Factor out some common code that will also be used with blk-mq.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Enhance mmc_blk_data_prep() to support CQE requests. That means adding
some things that for non-CQE requests would be encoded into the command
arguments - such as the block address, reliable-write flag, and data tag
flag. Also the request tag is needed to provide the command queue task id,
and a comment is added to explain the future possibility of defining a
priority.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Use local variables in mmc_blk_data_prep() in preparation for adding CQE
support which doesn't use the output variables.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Enable or disable CQE when a card is added or removed respectively.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Enable the Command Queue if the host controller supports a command queue
engine. It is not compatible with Packed Commands, so make a note of that in the
comment.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Add core support for handling CQE requests, including starting, completing
and recovering.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Currently the host can be claimed by a task. Change this so that the host
can be claimed by a context that may or may not be a task. This provides
for the host to be claimed by a block driver queue to support blk-mq, while
maintaining compatibility with the existing use of mmc_claim_host().
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Callers already have the host claimed, so remove the unnecessary
calls to mmc_claim_host() and mmc_release_host().
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
I forgot to account for the fact that the device core holds a
reference to a device added with device_initialize() that need
to be released with a corresponding put_device() to reach a 0
refcount at the end of the lifecycle.
This led to a NULL pointer reference when freeing the device
when e.g. unbidning the host device in sysfs.
Fix this and use the device .release() callback to free the
IDA and free:ing the memory used by the RPMB device.
Before this patch:
/sys/bus/amba/drivers/mmci-pl18x$ echo 80114000.sdi4_per2 > unbind
[ 29.797332] mmc3: card 0001 removed
[ 29.810791] Unable to handle kernel NULL pointer dereference at
virtual address 00000050
[ 29.818878] pgd = de70c000
[ 29.821624] [00000050] *pgd=1e70a831, *pte=00000000, *ppte=00000000
[ 29.827911] Internal error: Oops: 17 [#1] PREEMPT SMP ARM
[ 29.833282] Modules linked in:
[ 29.836334] CPU: 1 PID: 154 Comm: sh Not tainted
4.14.0-rc3-00039-g83318e309566-dirty #736
[ 29.844604] Hardware name: ST-Ericsson Ux5x0 platform (Device Tree Support)
[ 29.851562] task: de572700 task.stack: de742000
[ 29.856079] PC is at kernfs_find_ns+0x8/0x100
[ 29.860443] LR is at kernfs_find_and_get_ns+0x30/0x48
After this patch:
/sys/bus/amba/drivers/mmci-pl18x$ echo 80005000.sdi4_per2 > unbind
[ 20.623382] mmc3: card 0001 removed
Fixes: 97548575be ("mmc: block: Convert RPMB to a character device")
Reported-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
This function is used by the block layer queue to bail out of
requests if the current request is towards an RPMB
"block device".
This was done to avoid boot time scanning of this "block
device" which was never really a block device, thus duct-taping
over the fact that it was badly engineered.
This problem is now gone as we removed the offending RPMB block
device in another patch and replaced it with a character
device.
Cc: Tomas Winkler <tomas.winkler@intel.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
The RPMB partition on the eMMC devices is a special area used
for storing cryptographically safe information signed by a
special secret key. To write and read records from this special
area, authentication is needed.
The RPMB area is *only* and *exclusively* accessed using
ioctl():s from userspace. It is not really a block device,
as blocks cannot be read or written from the device, also
the signed chunks that can be stored on the RPMB are actually
256 bytes, not 512 making a block device a real bad fit.
Currently the RPMB partition spawns a separate block device
named /dev/mmcblkNrpmb for each device with an RPMB partition,
including the creation of a block queue with its own kernel
thread and all overhead associated with this. On the Ux500
HREFv60 platform, for example, the two eMMCs means that two
block queues with separate threads are created for no use
whatsoever.
I have concluded that this block device design for RPMB is
actually pretty wrong. The RPMB area should have been designed
to be accessed from /dev/mmcblkN directly, using ioctl()s on
the main block device. It is however way too late to change
that, since userspace expects to open an RPMB device in
/dev/mmcblkNrpmb and we cannot break userspace.
This patch tries to amend the situation using the following
strategy:
- Stop creating a block device for the RPMB partition/area
- Instead create a custom, dynamic character device with
the same name.
- Make this new character device support exactly the same
set of ioctl()s as the old block device.
- Wrap the requests back to the same ioctl() handlers, but
issue them on the block queue of the main partition/area,
i.e. /dev/mmcblkN
We need to create a special "rpmb" bus type in order to get
udev and/or busybox hot/coldplug to instantiate the device
node properly.
Before the patch, this appears in 'ps aux':
101 root 0:00 [mmcqd/2rpmb]
123 root 0:00 [mmcqd/3rpmb]
After applying the patch these surplus block queue threads
are gone, but RPMB is as usable as ever using the userspace
MMC tools, such as 'mmc rpmb read-counter'.
We get instead those dynamice devices in /dev:
brw-rw---- 1 root root 179, 0 Jan 1 2000 mmcblk0
brw-rw---- 1 root root 179, 1 Jan 1 2000 mmcblk0p1
brw-rw---- 1 root root 179, 2 Jan 1 2000 mmcblk0p2
brw-rw---- 1 root root 179, 5 Jan 1 2000 mmcblk0p5
brw-rw---- 1 root root 179, 8 Jan 1 2000 mmcblk2
brw-rw---- 1 root root 179, 16 Jan 1 2000 mmcblk2boot0
brw-rw---- 1 root root 179, 24 Jan 1 2000 mmcblk2boot1
crw-rw---- 1 root root 248, 0 Jan 1 2000 mmcblk2rpmb
brw-rw---- 1 root root 179, 32 Jan 1 2000 mmcblk3
brw-rw---- 1 root root 179, 40 Jan 1 2000 mmcblk3boot0
brw-rw---- 1 root root 179, 48 Jan 1 2000 mmcblk3boot1
brw-rw---- 1 root root 179, 33 Jan 1 2000 mmcblk3p1
crw-rw---- 1 root root 248, 1 Jan 1 2000 mmcblk3rpmb
Notice the (248,0) and (248,1) character devices for RPMB.
Cc: Tomas Winkler <tomas.winkler@intel.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
SD clock should be disabled for clock value 0. It's not
right to just return. This may cause failure of signal
voltage switching.
Signed-off-by: Yangbo Lu <yangbo.lu@nxp.com>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
The function was removed half a year ago, so this declaration can go,
too.
Fixes: 51ced59cc0 ("mmc: sdhci-pci: Use ACPI DSM to get driver strength for some Intel devices")
Signed-off-by: Wolfram Sang <wsa+renesas@sang-engineering.com>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
The delay circuit used to support HS400 is calibrated based on two
additional clocks. When these clocks are not available and
FF_CLK_SW_RST_DIS is not set in CORE_HC_MODE, reset might fail. But on
some platforms this doesn't work properly and below dump can be seen in
the kernel log.
mmc0: Reset 0x1 never completed.
mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00001102
mmc0: sdhci: Blk size: 0x00004000 | Blk cnt: 0x00000000
mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000000
mmc0: sdhci: Present: 0x01f80000 | Host ctl: 0x00000000
mmc0: sdhci: Power: 0x00000000 | Blk gap: 0x00000000
mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00000002
mmc0: sdhci: Timeout: 0x00000000 | Int stat: 0x00000000
mmc0: sdhci: Int enab: 0x00000000 | Sig enab: 0x00000000
mmc0: sdhci: AC12 err: 0x00000000 | Slot int: 0x00000000
mmc0: sdhci: Caps: 0x742dc8b2 | Caps_1: 0x00008007
mmc0: sdhci: Cmd: 0x00000000 | Max curr: 0x00000000
mmc0: sdhci: Resp[0]: 0x00000000 | Resp[1]: 0x00000000
mmc0: sdhci: Resp[2]: 0x00000000 | Resp[3]: 0x00000000
mmc0: sdhci: Host ctl2: 0x00000000
mmc0: sdhci: ============================================
Add support for the additional calibration clocks to allow these
platforms to be configured appropriately.
Cc: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Cc: Ritesh Harjani <riteshh@codeaurora.org>
Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Acked-by: Rob Herring <robh@kernel.org>
Tested-by: Jeremy McNicoll <jeremymc@redhat.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
By stuffing the runtime controlled clocks into a clk_bulk_data array we
can utilize the newly introduced bulk clock operations and clean up the
error paths. This allow us to handle additional clocks in subsequent
patch, without the added complexity.
Cc: Ritesh Harjani <riteshh@codeaurora.org>
Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Tested-by: Jeremy McNicoll <jeremymc@redhat.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
SDHCI controllers on Tegra186 support 40 bit addressing.
IOVA addresses are 48-bit wide on Tegra186.
SDHCI host common code sets dma mask as either 32-bit or 64-bit.
To avoid access issues when SMMU is enabled, disable 64-bit dma.
Signed-off-by: Krishna Reddy <vdumpa@nvidia.com>
Tested-by: Thierry Reding <treding@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Pull networking fixes from David Miller:
1) Fix route leak in xfrm_bundle_create().
2) In mac80211, validate user rate mask before configuring it. From
Johannes Berg.
3) Properly enforce memory limits in fair queueing code, from Toke
Hoiland-Jorgensen.
4) Fix lockdep splat in inet_csk_route_req(), from Eric Dumazet.
5) Fix TSO header allocation and management in mvpp2 driver, from Yan
Markman.
6) Don't take socket lock in BH handler in strparser code, from Tom
Herbert.
7) Don't show sockets from other namespaces in AF_UNIX code, from
Andrei Vagin.
8) Fix double free in error path of tap_open(), from Girish Moodalbail.
9) Fix TX map failure path in igb and ixgbe, from Jean-Philippe Brucker
and Alexander Duyck.
10) Fix DCB mode programming in stmmac driver, from Jose Abreu.
11) Fix err_count handling in various tunnels (ipip, ip6_gre). From Xin
Long.
12) Properly align SKB head before building SKB in tuntap, from Jason
Wang.
13) Avoid matching qdiscs with a zero handle during lookups, from Cong
Wang.
14) Fix various endianness bugs in sctp, from Xin Long.
15) Fix tc filter callback races and add selftests which trigger the
problem, from Cong Wang.
* git://git.kernel.org/pub/scm/linux/kernel/git/davem/net: (73 commits)
selftests: Introduce a new test case to tc testsuite
selftests: Introduce a new script to generate tc batch file
net_sched: fix call_rcu() race on act_sample module removal
net_sched: add rtnl assertion to tcf_exts_destroy()
net_sched: use tcf_queue_work() in tcindex filter
net_sched: use tcf_queue_work() in rsvp filter
net_sched: use tcf_queue_work() in route filter
net_sched: use tcf_queue_work() in u32 filter
net_sched: use tcf_queue_work() in matchall filter
net_sched: use tcf_queue_work() in fw filter
net_sched: use tcf_queue_work() in flower filter
net_sched: use tcf_queue_work() in flow filter
net_sched: use tcf_queue_work() in cgroup filter
net_sched: use tcf_queue_work() in bpf filter
net_sched: use tcf_queue_work() in basic filter
net_sched: introduce a workqueue for RCU callbacks of tc filter
sctp: fix some type cast warnings introduced since very beginning
sctp: fix a type cast warnings that causes a_rwnd gets the wrong value
sctp: fix some type cast warnings introduced by transport rhashtable
sctp: fix some type cast warnings introduced by stream reconf
...
Cong Wang says:
====================
net_sched: fix races with RCU callbacks
Recently, the RCU callbacks used in TC filters and TC actions keep
drawing my attention, they introduce at least 4 race condition bugs:
1. A simple one fixed by Daniel:
commit c78e1746d3
Author: Daniel Borkmann <daniel@iogearbox.net>
Date: Wed May 20 17:13:33 2015 +0200
net: sched: fix call_rcu() race on classifier module unloads
2. A very nasty one fixed by me:
commit 1697c4bb52
Author: Cong Wang <xiyou.wangcong@gmail.com>
Date: Mon Sep 11 16:33:32 2017 -0700
net_sched: carefully handle tcf_block_put()
3. Two more bugs found by Chris:
https://patchwork.ozlabs.org/patch/826696/https://patchwork.ozlabs.org/patch/826695/
Usually RCU callbacks are simple, however for TC filters and actions,
they are complex because at least TC actions could be destroyed
together with the TC filter in one callback. And RCU callbacks are
invoked in BH context, without locking they are parallel too. All of
these contribute to the cause of these nasty bugs.
Alternatively, we could also:
a) Introduce a spinlock to serialize these RCU callbacks. But as I
said in commit 1697c4bb52 ("net_sched: carefully handle
tcf_block_put()"), it is very hard to do because of tcf_chain_dump().
Potentially we need to do a lot of work to make it possible (if not
impossible).
b) Just get rid of these RCU callbacks, because they are not
necessary at all, callers of these call_rcu() are all on slow paths
and holding RTNL lock, so blocking is allowed in their contexts.
However, David and Eric dislike adding synchronize_rcu() here.
As suggested by Paul, we could defer the work to a workqueue and
gain the permission of holding RTNL again without any performance
impact, however, in tcf_block_put() we could have a deadlock when
flushing workqueue while hodling RTNL lock, the trick here is to
defer the work itself in workqueue and make it queued after all
other works so that we keep the same ordering to avoid any
use-after-free. Please see the first patch for details.
Patch 1 introduces the infrastructure, patch 2~12 move each
tc filter to the new tc filter workqueue, patch 13 adds
an assertion to catch potential bugs like this, patch 14
closes another rcu callback race, patch 15 and patch 16 add
new test cases.
====================
Reported-by: Chris Mi <chrism@mellanox.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Pirko <jiri@resnulli.us>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: Jamal Hadi Salim <jhs@mojatatu.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
In this patchset, we fixed a tc bug. This patch adds the test case
that reproduces the bug. To run this test case, user should specify
an existing NIC device:
# sudo ./tdc.py -d enp4s0f0
This test case belongs to category "flower". If user doesn't specify
a NIC device, the test cases belong to "flower" will not be run.
In this test case, we create 1M filters and all filters share the same
action. When destroying all filters, kernel should not panic. It takes
about 18s to run it.
Acked-by: Jamal Hadi Salim <jhs@mojatatu.com>
Acked-by: Lucas Bates <lucasb@mojatatu.com>
Signed-off-by: Chris Mi <chrism@mellanox.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
# ./tdc_batch.py -h
usage: tdc_batch.py [-h] [-n NUMBER] [-o] [-s] [-p] device file
TC batch file generator
positional arguments:
device device name
file batch file name
optional arguments:
-h, --help show this help message and exit
-n NUMBER, --number NUMBER
how many lines in batch file
-o, --skip_sw skip_sw (offload), by default skip_hw
-s, --share_action all filters share the same action
-p, --prio all filters have different prio
Acked-by: Jamal Hadi Salim <jhs@mojatatu.com>
Acked-by: Lucas Bates <lucasb@mojatatu.com>
Signed-off-by: Chris Mi <chrism@mellanox.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Similar to commit c78e1746d3
("net: sched: fix call_rcu() race on classifier module unloads"),
we need to wait for flying RCU callback tcf_sample_cleanup_rcu().
Cc: Yotam Gigi <yotamg@mellanox.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Pirko <jiri@resnulli.us>
Cc: Jamal Hadi Salim <jhs@mojatatu.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
After previous patches, it is now safe to claim that
tcf_exts_destroy() is always called with RTNL lock.
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Pirko <jiri@resnulli.us>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: Jamal Hadi Salim <jhs@mojatatu.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Defer the tcf_exts_destroy() in RCU callback to
tc filter workqueue and get RTNL lock.
Reported-by: Chris Mi <chrism@mellanox.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Pirko <jiri@resnulli.us>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: Jamal Hadi Salim <jhs@mojatatu.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Defer the tcf_exts_destroy() in RCU callback to
tc filter workqueue and get RTNL lock.
Reported-by: Chris Mi <chrism@mellanox.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Pirko <jiri@resnulli.us>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: Jamal Hadi Salim <jhs@mojatatu.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Defer the tcf_exts_destroy() in RCU callback to
tc filter workqueue and get RTNL lock.
Reported-by: Chris Mi <chrism@mellanox.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Pirko <jiri@resnulli.us>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: Jamal Hadi Salim <jhs@mojatatu.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Defer the tcf_exts_destroy() in RCU callback to
tc filter workqueue and get RTNL lock.
Reported-by: Chris Mi <chrism@mellanox.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Pirko <jiri@resnulli.us>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: Jamal Hadi Salim <jhs@mojatatu.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
