Jakub Kicinski pointed out a race condition in ipa_start_xmit() in a
recently-accepted series of patches:
https://lore.kernel.org/netdev/20210812195035.2816276-1-elder@linaro.org/
We are stopping the modem TX queue in that function if the power
state is not active. We restart the TX queue again once hardware
resume is complete.
TX path Power Management
------- ----------------
pm_runtime_get(); no power Start resume
Stop TX queue ...
pm_runtime_put() Resume complete
return NETDEV_TX_BUSY Start TX queue
pm_runtime_get()
Power present, transmit
pm_runtime_put() (auto-suspend)
The issue is that the power management (resume) activity and the
network transmit activity can occur concurrently, and there's a
chance the queue will be stopped *after* it has been started again.
TX path Power Management
------- ----------------
Resume underway
pm_runtime_get(); no power ...
Resume complete
Start TX queue
Stop TX queue <-- No more transmits after this
pm_runtime_put()
return NETDEV_TX_BUSY
We address this using a STARTED flag to indicate when the TX queue
has been started from the resume path, and a spinlock to make the
flag and queue updates happen atomically.
TX path Power Management
------- ----------------
Resume underway
pm_runtime_get(); no power Resume complete
start TX queue \
If STARTED flag is *not* set: > atomic
Stop TX queue set STARTED flag /
pm_runtime_put()
return NETDEV_TX_BUSY
A second flag is used to address a different race that involves
another path requesting power.
TX path Other path Power Management
------- ---------- ----------------
pm_runtime_get_sync() Resume
Start TX queue \ atomic
Set STARTED flag /
(do its thing)
pm_runtime_put()
(auto-suspend)
pm_runtime_get() Mark delayed resume
STARTED *is* set, so
do *not* stop TX queue <-- Queue should be stopped here
pm_runtime_put()
return NETDEV_TX_BUSY Suspend done, resume
Resume complete
pm_runtime_get()
Stop TX queue
(STARTED is *not* set) Start TX queue \ atomic
pm_runtime_put() Set STARTED flag /
return NETDEV_TX_BUSY
So a STOPPED flag is set in the transmit path when it has stopped
the TX queue, and this pair of operations is also protected by the
spinlock. The resume path only restarts the TX queue if the STOPPED
flag is set. This case isn't a major problem, but it avoids the
"non-trivial amount of useless work" done by the networking stack
when NETDEV_TX_BUSY is returned.
Fixes: 6b51f802d6 ("net: ipa: ensure hardware has power in ipa_start_xmit()")
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Currently a clock reference is taken whenever the ->ndo_open
callback for the modem netdev is called. That reference is dropped
when the device is closed, in ipa_stop().
We no longer need this, because ipa_start_xmit() now handles the
situation where the hardware power state is not active.
Drop the clock reference in ipa_open() when we're done, and take a
new reference in ipa_stop() before we begin closing the interface.
Finally (and unrelated, but trivial), change the return type of
ipa_start_xmit() to be netdev_tx_t instead of int.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Currently we stop the modem netdev transmit queue when suspending
the hardware. For system suspend this ensured we'd never attempt
to transmit while attempting to suspend the modem endpoints.
For runtime suspend, the IPA hardware might get suspended while the
system is operating. In that case we want an attempt to transmit a
packet to cause the hardware to resume if necessary. But if we
disable the queue this cannot happen.
So stop disabling the queue on suspend. In case we end up disabling
it in ipa_start_xmit() (see the previous commit), we still arrange
to start the TX queue on resume.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
We need to ensure the hardware is powered when we transmit a packet.
But if it's not, we can't block to wait for it. So asynchronously
request power in ipa_start_xmit(), and only proceed if the return
value indicates the power state is active.
If the hardware is not active, a runtime resume request will have
been initiated. In that case, stop the network stack from further
transmit attempts until the resume completes. Return NETDEV_TX_BUSY,
to retry sending the packet once the queue is restarted.
If the power request returns an error (other than -EINPROGRESS,
which just means a resume requested elsewhere isn't complete), just
drop the packet.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Create a new work structure in the modem private data, and use it to
re-enable the modem network device transmit queue when resuming.
This is needed by the next patch, which stops the TX queue if IPA
power isn't active when a transmit request arrives. Packets will
start arriving the instant the TX queue is enabled, but resuming
isn't complete until ipa_modem_resume() returns. This way we're
sure to be resumed before transmits are allowed again.
Cancel it before calling ipa_stop() in ipa_modem_stop() to ensure
the transmit queue restart completes before it gets stopped there.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
We currently assume no errors occur when enabling or disabling the
IPA core clock and interconnects. And although this commit exposes
errors that could occur, we generally assume this won't happen in
practice.
This commit changes ipa_clock_get() and ipa_clock_put() so each
returns a value. The values returned are meant to mimic what the
runtime power management functions return, so we can set up error
handling here before we make the switch. Have ipa_clock_get()
increment the reference count even if it returns an error, to match
the behavior of pm_runtime_get().
More details follow.
When taking a reference in ipa_clock_get(), return 0 for the first
reference, 1 for subsequent references, or a negative error code if
an error occurs. Note that if ipa_clock_get() returns an error, we
must not touch hardware; in some cases such errors now cause entire
blocks of code to be skipped.
When dropping a reference in ipa_clock_put(), we return 0 or an
error code. The error would come from ipa_clock_disable(), which
now returns what ipa_interconnect_disable() returns (either 0 or a
negative error code). For now, callers ignore the return value;
if an error occurs, a message will have already been logged, and
little more can actually be done to improve the situation.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Assign the ipa->modem_netdev and endpoint->netdev pointers *before*
registering the network device. As soon as the device is
registered it can be opened, and by that time we'll want those
pointers valid.
Similarly, don't make those pointers NULL until *after* the modem
network device is unregistered in ipa_modem_stop().
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
The modem network device is set up by ipa_modem_start(). But its
TX queue is not actually started and endpoints enabled until it is
opened.
So avoid stopping the modem network device TX queue and disabling
endpoints on suspend or stop unless the netdev is marked UP. And
skip attempting to resume unless it is UP.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
The IPA network device can be opened at any time, and an opened
network device can be stopped any time. Both of these callback
functions require access to the hardware, and therefore they need
the IPA clock to be operational. Take an IPA clock reference in
both the ->open and ->stop callback functions, dropping the
reference when they are done accessing hardware.
The ->start_xmit callback requires a little different handling,
and that will be added separately.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
The remoteproc SSR callback function for the modem requires hardware
access when handling a modem crash or shutdown. Take and later
release an IPA clock reference in ipa_modem_crashed(), to ensure the
hardware is operational.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
The first time it's booted, the modem loads and starts the
IPA-resident microcontroller. Once the microcontroller has
completed its initialization, it notifies the AP it's "ready"
by sending an INIT_COMPLETED response message.
Until it receives that microcontroller message, the AP must ensure
the IPA core clock remains operational. Currently, a "proxy" clock
reference is taken in ipa_uc_config(), dropping it again once the
message is received.
However there could be a long delay between when ipa_config()
completes and when modem actually starts. And because the
microcontroller gets loaded by the modem, there's no need to
get the modem "proxy clock" until the first time it starts.
Create a new function ipa_uc_clock() which takes the "proxy" clock
reference for the microcontroller. Call it when we get remoteproc
SSR notification that the modem is about to start. Keep an
additional flag to record whether this proxy clock reference needs
to be dropped at shutdown time, and issue a warning if we get the
microcontroller message either before the clock reference is taken,
or after it has already been dropped.
Drop the nearby use of "hh" length modifiers, which are no longer
encouraged in the kernel.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
The functions ipa_modem_setup() and ipa_modem_teardown() are trivial
wrappers that call ipa_qmi_setup() and ipa_qmi_teardown(). Just
call the QMI functions directly, and get rid of the wrappers.
Improve the documentation of what setting up QMI does.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
In ipa_modem_stop(), if the modem netdev pointer is non-null we call
ipa_stop(). We check for an error and if one is returned we handle
it. But ipa_stop() never returns an error, so this extra handling
is unnecessary. Simplify the code in ipa_modem_stop() based on the
knowledge no error handling is needed at this spot.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
In ipa_modem_start(), we set endpoint netdev pointers before the
network device is registered. If registration fails, we don't undo
those assignments. Instead, wait to assign the netdev pointer until
after registration succeeds.
Set these endpoint netdev pointers to NULL in ipa_modem_stop()
before unregistering the network device.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
At the moment it is quite hard to identify the network interface
provided by IPA in userspace components: The network interface is
created as virtual device, without any link to the IPA device.
The interface name ("rmnet_ipa%d") is the only indication that the
network interface belongs to IPA, but this is not very reliable.
Add SET_NETDEV_DEV() to associate the network interface with the
IPA parent device. This allows userspace services like ModemManager
to properly identify that this network interface is provided by IPA
and belongs to the modem.
Cc: Alex Elder <elder@kernel.org>
Fixes: a646d6ec90 ("soc: qcom: ipa: modem and microcontroller")
Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
Link: https://lore.kernel.org/r/20210106100755.56800-1-stephan@gerhold.net
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Use the new SSR notifier infrastructure to request notifications of
modem events, rather than the remoteproc IPA notification system.
The latter was put in place temporarily with the knowledge that the
new mechanism would become available.
Acked-by: David S. Miller <davem@davemloft.net>
Reviewed-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Signed-off-by: Alex Elder <elder@linaro.org>
Link: https://lore.kernel.org/r/20200724181142.13581-2-elder@linaro.org
Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Message logged by 'dev_xxx()' or 'pr_xxx()' should end with a '\n'.
Fixes: a646d6ec90 ("soc: qcom: ipa: modem and microcontroller")
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
There is a spelling mistake in a dev_err message. Fix it.
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch includes code implementing the modem functionality.
There are several communication paths between the AP and modem,
separate from the main data path provided by IPA. SMP2P provides
primitive messaging and interrupt capability, and QMI allows more
complex out-of-band messaging to occur between entities on the AP
and modem. (SMP2P and QMI support are added by the next patch.)
Management of these (plus the network device implementing the data
path) is done by code within "ipa_modem.c".
Sort of unrelated, this patch also includes the code supporting the
microcontroller CPU present on the IPA. The microcontroller can be
used to implement special handling of packets, but at this time we
don't support that. Still, it is a component that needs to be
initialized, and in the event of a crash we need to do some
synchronization between the AP and the microcontroller.
Signed-off-by: Alex Elder <elder@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
