Commit graph

24 commits

Author SHA1 Message Date
Serge Semin
f1678a4c66 NTB: Fix UB/bug in ntb_mw_get_align()
Simple (1 << pidx) operation causes undefined behaviour when
pidx >= 32. It must be casted to u64 to match the actual return
value of ntb_link_is_up() method, so to have all the possible
peer indexes covered and to get rid of undefined behaviour.
Additionally there are special macros in "linux/bitops.h" to perform
the bit-set-shift operations, so it's recommended to have them used
for proper bit setting.

Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2018-01-28 22:17:24 -05:00
Serge Semin
b87ab21935 NTB: Rename NTB messaging API methods
There is a common methods signature form used over all the NTB API
like functions naming scheme, arguments names and order, etc.
Recently added NTB messaging API IO callbacks were named a bit
different so should be renamed to be in compliance with the rest
of the API.

Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2018-01-28 22:17:23 -05:00
Logan Gunthorpe
0175250182 ntb_hw_switchtec: Add initialization code for crosslink
Crosslink is a feature of the Switchtec switches that is similar to
the B2B mode of other NTB devices. It allows a system to be designed
that is perfectly symmetric with two identical switches that link
two hosts together.

In order for the system to be symmetric, there is an empty host-less
partition between the two switches which the host must enumerate and
assign BAR addresses to. The firmware in the switch manages this
specially so that the BAR addresses on both sides of the empty
partition will be identical despite being in the same partition with
the same address space.

The driver determines whether crosslink is enabled by a flag set in
the NTB partition info registers which are set by the switch's
configuration file.

When crosslink is enabled, a reserved LUT window is setup to point to
the peer's switch's NTB registers and the local MWs are set to forward
to the host-less partition's BARs. (Yes, this hurts my brain too.)
Once this is setup, largely the same NTB infrastructure is used to
communicate between the two hosts.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2018-01-28 22:17:23 -05:00
Logan Gunthorpe
e099b45b7c NTB: switchtec_ntb: Add skeleton NTB driver
Add a skeleton NTB driver which will be filled out in subsequent patches.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Reviewed-by: Stephen Bates <sbates@raithlin.com>
Reviewed-by: Kurt Schwemmer <kurt.schwemmer@microsemi.com>
Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-11-18 20:37:12 -05:00
Logan Gunthorpe
fa5ab66e36 NTB: Add check and comment for link up to mw_count() and mw_get_align()
Adds a comment and a check to ntb_mw_get_align() so that it always fails
if the function is called before the link is up.

Also adds a comment to ntb_mw_count() to note that it may return 0 if
it is called before the link is up.

This is to prevent accidental mis-use in clients that are testing
on hardware that this doesn't matter for.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-11-18 20:37:11 -05:00
Serge Semin
3c69f5d673 NTB: Add ntb.h comments
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-07-06 11:30:07 -04:00
Serge Semin
85dce3aaae NTB: Add PCIe Gen4 link speed
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-07-06 11:30:07 -04:00
Serge Semin
bc3e49adc2 NTB: Add Messaging NTB API
Some IDT NTB-capable PCIe-switches have message registers to communicate with
peer devices. This patch adds new NTB API callback methods, which can be used
to utilize these registers functionality:
 ntb_msg_count(); - get number of message registers
 ntb_msg_inbits(); - get bitfield of inbound message registers status
 ntb_msg_outbits(); - get bitfield of outbound message registers status
 ntb_msg_read_sts(); - read the inbound and outbound message registers status
 ntb_msg_clear_sts(); - clear status bits of message registers
 ntb_msg_set_mask(); - mask interrupts raised by status bits of message
registers.
 ntb_msg_clear_mask(); - clear interrupts mask bits of message registers
 ntb_msg_read(midx, *pidx); - read message register with specified index,
additionally getting peer port index which data received from
 ntb_msg_write(midx, pidx); - write data to the specified message register
sending it to the passed peer device connected over a pidx port
 ntb_msg_event(); - notify driver context of a new message event

Of course there is hardware which doesn't support Message registers, so
this API is made optional.

Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-07-06 11:30:07 -04:00
Serge Semin
d67288a395 NTB: Alter Scratchpads API to support multi-ports devices
Even though there is no any real NTB hardware, which would have both more
than two ports and Scratchpad registers, it is logically correct to have
Scratchpad API accepting a peer port index as well. Intel/AMD drivers utilize
Primary and Secondary topology to split Scratchpad between connected root
devices. Since port-index API introduced, Intel/AMD NTB hardware drivers can
use device port to determine which Scratchpad registers actually belong to
local and peer devices. The same approach can be used if some potential
hardware in future will be multi-port and have some set of Scratchpads.
Here are the brief of changes in the API:
 ntb_spad_count() - return number of Scratchpads per each port
 ntb_peer_spad_addr(pidx, sidx) - address of Scratchpad register of the
peer device with pidx-index
 ntb_peer_spad_read(pidx, sidx) - read specified Scratchpad register of the
peer with pidx-index
 ntb_peer_spad_write(pidx, sidx) - write data to Scratchpad register of the
peer with pidx-index

Since there is hardware which doesn't support Scratchpad registers, the
corresponding API methods are now made optional.

Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-07-06 11:30:07 -04:00
Serge Semin
443b9a14ec NTB: Alter MW API to support multi-ports devices
Multi-port NTB devices permit to share a memory between all accessible peers.
Memory Windows API is altered to correspondingly initialize and map memory
windows for such devices:
 ntb_mw_count(pidx); - number of inbound memory windows, which can be allocated
for shared buffer with specified peer device.
 ntb_mw_get_align(pidx, widx); - get alignment and size restriction parameters
to properly allocate inbound memory region.
 ntb_peer_mw_count(); - get number of outbound memory windows.
 ntb_peer_mw_get_addr(widx); - get mapping address of an outbound memory window

If hardware supports inbound translation configured on the local ntb port:
 ntb_mw_set_trans(pidx, widx); - set translation address of allocated inbound
memory window so a peer device could access it.
 ntb_mw_clear_trans(pidx, widx); - clear the translation address of an inbound
memory window.

If hardware supports outbound translation configured on the peer ntb port:
 ntb_peer_mw_set_trans(pidx, widx); - set translation address of a memory
window retrieved from a peer device
 ntb_peer_mw_clear_trans(pidx, widx); - clear the translation address of an
outbound memory window

Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-07-06 11:30:07 -04:00
Serge Semin
4e8c11b7fd NTB: Alter link-state API to support multi-port devices
Multi-port devices permit the NTB connections between multiple domains,
so a local device can have NTB link being up with one peer and being
down with another. NTB link-state API is appropriately altered to return
a bitfield of the link-states between the local device and possible peers.

Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-07-06 11:30:07 -04:00
Serge Semin
1e5301196a NTB: Add indexed ports NTB API
There is some NTB hardware, which can combine more than just two domains
over NTB. For instance, some IDT PCIe-switches can have NTB-functions
activated on more than two-ports. The different domains are distinguished
by ports they are connected to. So the new port-related methods are added to
the NTB API:
 ntb_port_number() - return local port
 ntb_peer_port_count() - return number of peers local port can connect to
 ntb_peer_port_number(pdix) - return port number by it index
 ntb_peer_port_idx(port) - return port index by it number

Current test-drivers aren't changed much. They still support two-ports devices
for the time being while multi-ports hardware drivers aren't added.

By default port-related API is declared for two-ports hardware.
So corresponding hardware drivers won't need to implement it.

Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-07-06 11:30:07 -04:00
Serge Semin
60934b200d NTB: Make link-state API being declared first
Since link operations are usually performed before memory window access
operations, it's logically better to declare link-related API before any
of MW/Doorbell/Scratchpad methods.

Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-07-06 11:30:07 -04:00
Linus Torvalds
01e0d6037d NTB bug fixes for removing an unnecessary call to ntb_peer_spad_read,
and correcting a free_irq inconsistency.  It adds Intel SKX support.
 Finally, it changes the AMD NTB maintainer, and fixes some bugs present
 there.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJYXpoNAAoJEG5mS6x6i9IjKZkP/irnIN87sQQcNKoIJJGco4Xc
 0S4PwU+E4/dP58f1Pi+9EdxhItXnksIBuGFFdvNYkbBH/pC7e6y8TD3oCmMvXBEG
 Qbl9RNB8boI6TZuM2FLtjAUTD6QKqBpubEq9xzRV4/tM/94//JvK8wVvBGAl0z8s
 85kUVrNVY2ahntEgldCXPbX8ONNpYeUdm+YEv0whclAvN1nYxZAfSWA/G21I7VbI
 TH2PjkYy9R+pJglO9Xl0nBj/YG6/LZZnS37sQLAhgIvMb7NIBhpsXCb63lwKo75e
 Sp0p+T+AF9Ax7//g34mljLjtXSEgyOiIfSJrmUa+f/WS1LU8CUI6Z5memo7vEWPW
 FRd9t1NLEd4fZJThQKsFgZpAcKpFQPRT4hf9eGlK5N5e2/jFjJQx1KGEtU3RCHA6
 UrI+ddU57qwwuD4Rbw8PW1fulsEh7lBmyoIFnTiYnkHffzmqL5e5XX9ODSr9Fctj
 c/loucgTMz7T0cgyz3ws8we+HF3o8hzK1gyK3DzRQ46s+PQr46LycLSEFzmqhkHh
 9KqvcJMIJNxxy3nn46Ag1W8yO60Ik6Kz0ujPcaDhpaxpLybVJqHSR5X0n98cf8NZ
 vPktqMN0b4HGVjIracacvfsT/UhLZpooZTrl+jwrd/iGM0Xqu/VpMVkiRNrcNjSN
 Pz6Oxtho25CZVcU1Nxi3
 =F0Rd
 -----END PGP SIGNATURE-----

Merge tag 'ntb-4.10' of git://github.com/jonmason/ntb

Pull NTB update from Jon Mason:

 - NTB bug fixes for removing an unnecessary call to ntb_peer_spad_read,
   and correcting a free_irq inconsistency

 - add Intel SKX support

 - change the AMD NTB maintainer, and fix some bugs present there

* tag 'ntb-4.10' of git://github.com/jonmason/ntb:
  ntb_transport: Remove unnecessary call to ntb_peer_spad_read
  NTB: Fix 'request_irq()' and 'free_irq()' inconsistancy
  ntb: fix SKX NTB config space size register offsets
  NTB: correct ntb_peer_spad_read for case when callback is not supplied.
  MAINTAINERS: Change in maintainer for AMD NTB
  ntb_transport: Limit memory windows based on available, scratchpads
  NTB: Register and offset values fix for memory window
  NTB: add support for hotplug feature
  ntb: Adding Skylake Xeon NTB support
2016-12-24 11:23:24 -08:00
Steven Wahl
5c43c52d5f NTB: correct ntb_peer_spad_read for case when callback is not supplied.
Correct ntb_peer_spad_read for case when callback is not supplied

Signed-off-by: Steve Wahl <Steve.Wahl@dell.com>
Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2016-12-23 16:10:50 -05:00
Aaron Sierra
74dcba3589 NTB: correct ntb_spad_count comment typo
The comment for ntb_spad_count incorrectly referred to ntb_mw_count.

Signed-off-by: Aaron Sierra <asierra@xes-inc.com>
Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>
2016-12-14 10:54:29 +01:00
Allen Hubbe
afc5499229 NTB: Make _addr functions optional in the API
The functions ntb_peer_db_addr and ntb_peer_spad_addr were required by
the api.  The functions already support returning an error, so any
existing calling code should already check for it.  Any existing code
using drivers that implement the functions will not be affected.

Signed-off-by: Allen Hubbe <Allen.Hubbe@emc.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2016-03-21 19:30:06 -04:00
Allen Hubbe
86663c9186 NTB: Fix documentation for ntb_peer_db_clear.
The documentation should say "peer" not "local" when referring to the
peer doorbell register.

Reported-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Allen Hubbe <Allen.Hubbe@emc.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2015-09-07 15:17:09 -04:00
Allen Hubbe
a7c2323748 NTB: Fix documentation for ntb_link_is_up
There was a copy and paste error in the documentation for
ntb_link_is_up.  The long description was mistakenly copied from
ntb_link_set_trans.

This adds the appropriate long description for ntb_link_is_up.

Reported-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Allen Hubbe <Allen.Hubbe@emc.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2015-09-07 15:17:09 -04:00
Allen Hubbe
a1bd3baeb2 NTB: Add NTB hardware abstraction layer
Abstract the NTB device behind a programming interface, so that it can
support different hardware and client drivers.

Signed-off-by: Allen Hubbe <Allen.Hubbe@emc.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2015-07-04 14:04:44 -04:00
Allen Hubbe
ec110bc7cc NTB: Move files in preparation for NTB abstraction
This patch only moves files to their new locations, before applying the
next two patches adding the NTB Abstraction layer.  Splitting this patch
from the next is intended make distinct which code is changed only due
to moving the files, versus which are substantial code changes in adding
the NTB Abstraction layer.

Signed-off-by: Allen Hubbe <Allen.Hubbe@emc.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2015-07-02 10:09:23 -04:00
Jon Mason
53ca4fea0b NTB: Code Style Clean-up
Some white space and 80 char overruns corrected.

Signed-off-by: Jon Mason <jon.mason@intel.com>
2014-04-07 10:59:19 -07:00
Jon Mason
403c63cb6d NTB: client event cleanup
Provide a better event interface between the client and transport

Signed-off-by: Jon Mason <jon.mason@intel.com>
2014-04-07 10:59:19 -07:00
Jon Mason
fce8a7bb5b PCI-Express Non-Transparent Bridge Support
A PCI-Express non-transparent bridge (NTB) is a point-to-point PCIe bus
connecting 2 systems, providing electrical isolation between the two subsystems.
A non-transparent bridge is functionally similar to a transparent bridge except
that both sides of the bridge have their own independent address domains.  The
host on one side of the bridge will not have the visibility of the complete
memory or I/O space on the other side of the bridge.  To communicate across the
non-transparent bridge, each NTB endpoint has one (or more) apertures exposed to
the local system.  Writes to these apertures are mirrored to memory on the
remote system.  Communications can also occur through the use of doorbell
registers that initiate interrupts to the alternate domain, and scratch-pad
registers accessible from both sides.

The NTB device driver is needed to configure these memory windows, doorbell, and
scratch-pad registers as well as use them in such a way as they can be turned
into a viable communication channel to the remote system.  ntb_hw.[ch]
determines the usage model (NTB to NTB or NTB to Root Port) and abstracts away
the underlying hardware to provide access and a common interface to the doorbell
registers, scratch pads, and memory windows.  These hardware interfaces are
exported so that other, non-mainlined kernel drivers can access these.
ntb_transport.[ch] also uses the exported interfaces in ntb_hw.[ch] to setup a
communication channel(s) and provide a reliable way of transferring data from
one side to the other, which it then exports so that "client" drivers can access
them.  These client drivers are used to provide a standard kernel interface
(i.e., Ethernet device) to NTB, such that Linux can transfer data from one
system to the other in a standard way.

Signed-off-by: Jon Mason <jon.mason@intel.com>
Reviewed-by: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-01-17 19:11:14 -08:00