Commit Graph

45 Commits

Author SHA1 Message Date
Jason A. Donenfeld a251c17aa5 treewide: use get_random_u32() when possible
The prandom_u32() function has been a deprecated inline wrapper around
get_random_u32() for several releases now, and compiles down to the
exact same code. Replace the deprecated wrapper with a direct call to
the real function. The same also applies to get_random_int(), which is
just a wrapper around get_random_u32(). This was done as a basic find
and replace.

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Yury Norov <yury.norov@gmail.com>
Reviewed-by: Jan Kara <jack@suse.cz> # for ext4
Acked-by: Toke Høiland-Jørgensen <toke@toke.dk> # for sch_cake
Acked-by: Chuck Lever <chuck.lever@oracle.com> # for nfsd
Acked-by: Jakub Kicinski <kuba@kernel.org>
Acked-by: Mika Westerberg <mika.westerberg@linux.intel.com> # for thunderbolt
Acked-by: Darrick J. Wong <djwong@kernel.org> # for xfs
Acked-by: Helge Deller <deller@gmx.de> # for parisc
Acked-by: Heiko Carstens <hca@linux.ibm.com> # for s390
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2022-10-11 17:42:58 -06:00
Li zeming 006fd8cb65 habanalabs/gaudi2: Remove unnecessary (void*) conversions
The void pointer object can be directly assigned to different structure
objects, it does not need to be cast.

Signed-off-by: Li zeming <zeming@nfschina.com>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:40 +03:00
Dani Liberman 43657dadfe habanalabs/gaudi2: add handling to pmmu events in eqe handler
In order to get the error cause and the captured address in case of
page fault, added pmmu events to eqe handler.

Signed-off-by: Dani Liberman <dliberman@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:39 +03:00
farah kassabri 04d53cd2a6 habanalabs/gaudi2: get f/w reset status register dynamically
Get the firmware reset status address from the dynamic registers
we read from the firmware instead of using a define.

Signed-off-by: farah kassabri <fkassabri@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:39 +03:00
Tomer Tayar f0b6d3cc29 habanalabs/gaudi2: increase hard-reset sleep time to 2 sec
The access to the device registers is blocked during hard reset, until
preboot runs and allows the access to specific registers, including the
PSOC BTM_FSM register which is used to know when the reset is done.
Between the reset request and until this register is polled there is a
small delay of 500 msec which is not enough for F/W to process the reset
and for preboot to run, so the register might be accessed while it is
blocked.
To avoid it, increase the delay to 2 sec.

Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:39 +03:00
Tomer Tayar cecde184ca habanalabs/gaudi2: print RAZWI info upon PCIe access error
Add the dump of the RAZWI information when a PCIe access is blocked by
RR.

Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:39 +03:00
Oded Gabbay 82736b063f habanalabs: MMU invalidation h/w is per device
The code used the mmu mutex to protect access to the context's page
tables and invalidation of the MMU cache. Because pgt are per
context, the mmu mutex was a member of the context object.

The problem is that the device has a single MMU invalidation h/w
(per MMU). Therefore, the mmu mutex should not be a property of the
context but a property of the device.

Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:39 +03:00
Tal Cohen 6f0818c9fc habanalabs: new notifier events for device state
Add new notifier events that inform several device states.
General H/W error raised on device general H/W error occurs.
User engine error is raised when a device engine informs of an error.

Signed-off-by: Tal Cohen <talcohen@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:38 +03:00
Oded Gabbay c833ac1a5f habanalabs/gaudi2: free event irq if init fails
In case initialization fails after event irq was requested, we need to
release that irq.

Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:38 +03:00
Ofir Bitton 0626fa1a4d habanalabs: add support for new cpucp return codes
Firmware now responds with a more detailed cpucp return codes.
Driver can now distinguish between error and debug return codes.

Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:38 +03:00
Ofir Bitton aee3fd74fe habanalabs/gaudi: rename mme cfg error response print
Current description is misleading hence we rename it to a more
suitable error description.

Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:38 +03:00
farah kassabri 4745b2f0d0 habanalabs: send device active message to f/w
As part of the RAS that is done by the f/w, we should send a message
to the f/w when a user either acquires or releases the device.

Signed-off-by: farah kassabri <fkassabri@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:37 +03:00
Ofir Bitton 0855bf8b17 habanalabs/gaudi2: dump detailed information upon RAZWI
In order to improve debuggability, we add all available information
when a RAZWI event occur.

Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-19 15:08:37 +03:00
farah kassabri 988262ef2f habanalabs/gaudi2: log critical events with no rate limit
When we have a storm of errors of HBM ECC SERR we can reach a situation
where driver start hard reset flow without logging the error cause
that caused the hard reset due to logs rate limiting.

Signed-off-by: farah kassabri <fkassabri@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:53 +03:00
Dafna Hirschfeld 262042af13 habanalabs: set command buffer host VA dynamically
Set the addresses for userspace command buffer dynamically
instead of hard-coded. There is no reason for it to
be hard-coded.

Signed-off-by: Dafna Hirschfeld <dhirschfeld@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:53 +03:00
Rajarama Manjukody Bhat 7b5d13c9ca habanalabs/gaudi2: assigning PQFs for ARC f/w in PDMA
Assigning 3 PQFs in PDMA1 and 2 PQFs in PDMA0 for ARC firmware usage.

Signed-off-by: Rajarama Manjukody Bhat <rmbhat@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:52 +03:00
Dani Liberman 07ecaa0d85 habanalabs: unify hwmon resources clean up
Since hwmon fini code is common for all asics, unified it to common
function.

Signed-off-by: Dani Liberman <dliberman@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:51 +03:00
Tal Cohen 194e515c79 habanalabs/gaudi2: new API to control engine cores running mode
The current flow of halting the engine cores is implemented by command
buffers built by the user space and sent towards the Driver.

This current flow is broken since the user space does not know when
the cores actually halt as sending a workload is async op.

Therefore the application can not free the memory that is mapped
to the engine cores.

This new API allows the user space to control the running mode. The
API call is sync (returns after the cores are set to the
requested mode).

Signed-off-by: Tal Cohen <talcohen@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:51 +03:00
farah kassabri 6419b5232e habanalabs/gaudi2: change device f/w security check
On Gaudi2 the f/w always configures the PCIe iATU and allows access to
scratchpad registers. Therefore, we can know if the f/w is secured
by reading a status bit from the f/w registers.

Signed-off-by: farah kassabri <fkassabri@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:51 +03:00
Tomer Tayar 21fc79336b habanalabs/gaudi2: mark PCIE access error as fatal
F/W events are enabled in a late phase of the device init, so an event
for a PCIE access error during the init, can be received after the init
is already done and considered as successful.
A resulting device reset, which does the same H/W init, can end
similarly with this event right after the reset is done and considered
as successful, and a loop of this sequence can continue.

To avoid it mark the PCIE access error as a fatal event, so after 2
consecutive events no more resets will be done.

Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:50 +03:00
Oded Gabbay 5f92c1e296 habanalabs: remove all kdma locks
We don't use KDMA concurrently in the driver. The only use is through
debugfs and we don't protect concurrent access through it.

Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:50 +03:00
Ohad Sharabi 913bd4179b habanalabs: add return code field to module iterator
Up until now the module iterator called void callback functions
and so caller activating callback that may fail suffered from 2 issues:
1. The need to "plant" return called in the private data. This is a
   drawback since the iterator itself should not be aware of the private
   data of the caller.
2. Due to 1 even in a failure the iterator would keep iterating instead
   of break upon error.

To overcome this an optional rc field added to the iterator context.

Signed-off-by: Ohad Sharabi <osharabi@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:49 +03:00
Tomer Tayar 28742772a0 habanalabs/gaudi2: enable all MMU SPI/SEI interrupts
Currently only part of the MMU SPI/SEI interrupts are enabled, although
there is no real reason to not enable all.
The only exception is "burst_fifo_full" which is expected for PMMU
because it has a 2 entries FIFO, and thus is it not enabled for it.

Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:49 +03:00
Ofir Bitton bc9b271e6c habanalabs: rename non_hard_reset to compute_reset
In order to be more explicit we should use the term compute_reset
for describing the reset in which only the compute engines gets
reset.

Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:49 +03:00
Yang Li 2d4c09e3f9 habanalabs: Simplify bool conversion
Fix the following coccicheck warning:
./drivers/misc/habanalabs/gaudi2/gaudi2.c:9727:48-53: WARNING:
conversion to bool not needed here

Signed-off-by: Yang Li <yang.lee@linux.alibaba.com>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:49 +03:00
Dani Liberman 71386e11f2 habanalabs: removed seq_file parameter from is_idle asic functions
Change is_idle functions so it would be more usable outside debugfs.

Do this by replacing seq_file parameter with regular string.

Signed-off-by: Dani Liberman <dliberman@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-09-18 13:29:49 +03:00
Oded Gabbay a919b823ab habanalabs: move h/w dirty message to debug
H/W being dirty during initialization is completely expected in case
f/w tools are used before loading the driver. As it is not an error,
and as it doesn't give any meaningful information to the user,
no point of printing it.

Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:31 +03:00
Oded Gabbay 0b0ae02440 habanalabs: rename soft reset to compute reset
Doing compute reset can be the traditional inference soft reset
that is supported only in Goya.

Or it can be the new reset upon device release, which is supported
in Gaudi2 and above.

Therefore, wherever suitable, use the terminology of compute reset
instead of soft reset.

Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:31 +03:00
Tomer Tayar 168fc71857 habanalabs/gaudi2: map virtual MSI-X doorbell memory for user
Upon the initialization of a user context, map the host memory page of
the virtual MSI-X doorbell in the device MMU.
A reserved VA is used for this purpose, so user can use it directly
without any allocation/map operation.

Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:31 +03:00
Tomer Tayar 3f043b3192 habanalabs/gaudi2: modify decoder to use virtual MSI-X doorbell
Modify the decoder wrapper blocks to generate interrupts using the
virtual MSI-X doorbell.

As a decoder wrapper block cannot write directly to HBW upon completion,
it writes instead to SOB which is monitored by a master monitor.
When resolved, this monitor will be the one to actually write to the
virtual MSI-X doorbell.

Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:31 +03:00
Tomer Tayar bfbf5a0a71 habanalabs/gaudi2: modify CS completion CQ to use virtual MSI-X doorbell
Modify the CQ which is used for CS completion, to use the virtual MSI-X
doorbell.

Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:30 +03:00
Tomer Tayar 25ad863839 habanalabs/gaudi2: replace defines for reserved sob/mob with enums
Following patches are going to add more reserved sync objects and
monitors.
To make the counting of these reserved resources simpler, replace the
existing RESERVED_* defines with enumerations.

Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:30 +03:00
Tomer Tayar 1cf596c6b9 habanalabs/gaudi2: configure virtual MSI-X doorbell interface
Due to a watchdog timer in the LBW path, writes to the MSI-X doorbell
can return sporadic error responses.
To work-around this issue, a virtual MSI-X doorbell on the HBW path is
configured, using the MSI-X AXI slave interface in the PCIe controller.
Upon an access to a configured HBW host address, the controller will
generate MSI-X interrupt instead of treating the access as regular host
memory access.

This patch allocates the dedicate host memory page, and communicate the
address to F/W, so it will configure the relevant address match
registers in the controller, and will use this address to generate MSI-X
interrupts for F/W events.

Following patches will handle other initiators in the device, to move
them to use the virtual MSI-X doorbell.

Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:30 +03:00
Tomer Tayar af2e650b36 habanalabs: add a value field to hl_fw_send_pci_access_msg()
For gaudi2 we need to send a value to F/W as part of the
PCI_ACCESS packet.
As a preparation, modify hl_fw_send_pci_access_msg() to have a 'value'
field.

Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:30 +03:00
Ohad Sharabi 3fc252670b habanalabs/gaudi2: use DIV_ROUND_UP_SECTOR_T instead of roundup
roundup will create an error in 32-bit architectures as we use
64-bit variables.

Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Ohad Sharabi <osharabi@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:30 +03:00
Oded Gabbay ead36b1981 habanalabs/gaudi2: remove unused defines
There were some defines that are unused in the current upstreamed
code.

Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:30 +03:00
Oded Gabbay e475acabb9 habanalabs/gaudi2: SM mask can only be 8-bit
Otherwise, due to how we calculate it, we might fail in FIELD_PREP
checks.

Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:29 +03:00
Oded Gabbay c979cb5d8b habanalabs/gaudi2: remove unused variable
glbl_sts_clr_val was set but never used

Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:29 +03:00
Ofir Bitton d6a66d5960 habanalabs: add support for common decoder interrupts
User application should be able to get notification for any decoder
completion. Hence, we introduce a new interface in which a user
can wait for all current decoder pending interrupts.

Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:29 +03:00
Ofir Bitton 1a6609cdd4 habanalabs: naming refactor of user interrupt flow
Current naming convention can be misleading. Hence renaming some
variables and defines in order to be more explicit.

Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:28 +03:00
Ohad Sharabi 2b9e583d0a habanalabs: wait for preboot ready after hard reset
Currently we are not waiting for preboot ready after hard reset.
This leads to a race in which COMMs protocol begins but will get no
response from the f/w.

Signed-off-by: Ohad Sharabi <osharabi@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:28 +03:00
Ofir Bitton a85e389a84 habanalabs/gaudi2: reset device upon critical ECC event
Correctable ECC events are not fatal, but as they accumulate, the f/w
can decide that a hard-rest is required. This indication is
propagated to the host using the existing ECC event interface.

Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:28 +03:00
Benjamin Dotan de88aa67af habanalabs/gaudi2: add gaudi2 profiler module
Add the Gaudi2 code to initialize the ASIC's profiler. The profile
receives its initialization values from the user, same as in Gaudi2,
but the code to initialize is in the driver because the configuration
space of the device is not directly exposed to the user.

Signed-off-by: Benjamin Dotan <bdotan@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:28 +03:00
Ofir Bitton 4567214686 habanalabs/gaudi2: add gaudi2 security module
Use the generic security module to block all registers in the ASIC and
then open only those that are needed to be accessed by the user.

Signed-off-by: Ofir Bitton <obitton@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:28 +03:00
Oded Gabbay d7bb1ac89b habanalabs: add gaudi2 asic-specific code
Add the ASIC-specific code for Gaudi2. Supply (almost) all of the
function callbacks that the driver's common code need to initialize,
finalize and submit workloads to the Gaudi2 ASIC.

It also contains the code to initialize the F/W of the Gaudi2 ASIC
and to receive events from the F/W.

It contains new debugfs entry to dump razwi events. razwi is a case
where the device's engines create a transaction that reaches an
invalid destination.

Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
2022-07-12 09:09:27 +03:00