cxl/mem: Introduce a driver for CXL-2.0-Type-3 endpoints

The CXL.mem protocol allows a device to act as a provider of "System
RAM" and/or "Persistent Memory" that is fully coherent as if the memory
was attached to the typical CPU memory controller.

With the CXL-2.0 specification a PCI endpoint can implement a "Type-3"
device interface and give the operating system control over "Host
Managed Device Memory". See section 2.3 Type 3 CXL Device.

The memory range exported by the device may optionally be described by
the platform firmware memory map, or by infrastructure like LIBNVDIMM to
provision persistent memory capacity from one, or more, CXL.mem devices.

A pre-requisite for Linux-managed memory-capacity provisioning is this
cxl_mem driver that can speak the mailbox protocol defined in section
8.2.8.4 Mailbox Registers.

For now just land the initial driver boiler-plate and Documentation/
infrastructure.

Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Acked-by: David Rientjes <rientjes@google.com> (v1)
Cc: Jonathan Corbet <corbet@lwn.net>
Link: https://www.computeexpresslink.org/download-the-specification
Link: https://lore.kernel.org/r/20210217040958.1354670-2-ben.widawsky@intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This commit is contained in:
Dan Williams 2021-02-16 20:09:50 -08:00
parent 1048ba83fb
commit 4cdadfd5e0
10 changed files with 149 additions and 0 deletions

View File

@ -0,0 +1,12 @@
.. SPDX-License-Identifier: GPL-2.0
====================
Compute Express Link
====================
.. toctree::
:maxdepth: 1
memory-devices
.. only:: subproject and html

View File

@ -0,0 +1,15 @@
.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>
===================================
Compute Express Link Memory Devices
===================================
A Compute Express Link Memory Device is a CXL component that implements the
CXL.mem protocol. It contains some amount of volatile memory, persistent memory,
or both. It is enumerated as a PCI device for configuration and passing
messages over an MMIO mailbox. Its contribution to the System Physical
Address space is handled via HDM (Host Managed Device Memory) decoders
that optionally define a device's contribution to an interleaved address
range across multiple devices underneath a host-bridge or interleaved
across host-bridges.

View File

@ -35,6 +35,7 @@ available subsections can be seen below.
usb/index
firewire
pci/index
cxl/index
spi
i2c
ipmb

View File

@ -6,6 +6,7 @@ menu "Device Drivers"
source "drivers/amba/Kconfig"
source "drivers/eisa/Kconfig"
source "drivers/pci/Kconfig"
source "drivers/cxl/Kconfig"
source "drivers/pcmcia/Kconfig"
source "drivers/rapidio/Kconfig"

View File

@ -73,6 +73,7 @@ obj-$(CONFIG_NVM) += lightnvm/
obj-y += base/ block/ misc/ mfd/ nfc/
obj-$(CONFIG_LIBNVDIMM) += nvdimm/
obj-$(CONFIG_DAX) += dax/
obj-$(CONFIG_CXL_BUS) += cxl/
obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf/
obj-$(CONFIG_NUBUS) += nubus/
obj-y += macintosh/

35
drivers/cxl/Kconfig Normal file
View File

@ -0,0 +1,35 @@
# SPDX-License-Identifier: GPL-2.0-only
menuconfig CXL_BUS
tristate "CXL (Compute Express Link) Devices Support"
depends on PCI
help
CXL is a bus that is electrically compatible with PCI Express, but
layers three protocols on that signalling (CXL.io, CXL.cache, and
CXL.mem). The CXL.cache protocol allows devices to hold cachelines
locally, the CXL.mem protocol allows devices to be fully coherent
memory targets, the CXL.io protocol is equivalent to PCI Express.
Say 'y' to enable support for the configuration and management of
devices supporting these protocols.
if CXL_BUS
config CXL_MEM
tristate "CXL.mem: Memory Devices"
help
The CXL.mem protocol allows a device to act as a provider of
"System RAM" and/or "Persistent Memory" that is fully coherent
as if the memory was attached to the typical CPU memory
controller.
Say 'y/m' to enable a driver (named "cxl_mem.ko" when built as
a module) that will attach to CXL.mem devices for
configuration, provisioning, and health monitoring. This
driver is required for dynamic provisioning of CXL.mem
attached memory which is a prerequisite for persistent memory
support. Typically volatile memory is mapped by platform
firmware and included in the platform memory map, but in some
cases the OS is responsible for mapping that memory. See
Chapter 2.3 Type 3 CXL Device in the CXL 2.0 specification.
If unsure say 'm'.
endif

4
drivers/cxl/Makefile Normal file
View File

@ -0,0 +1,4 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_CXL_MEM) += cxl_mem.o
cxl_mem-y := mem.o

62
drivers/cxl/mem.c Normal file
View File

@ -0,0 +1,62 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/io.h>
#include "pci.h"
static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
{
int pos;
pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_DVSEC);
if (!pos)
return 0;
while (pos) {
u16 vendor, id;
pci_read_config_word(pdev, pos + PCI_DVSEC_HEADER1, &vendor);
pci_read_config_word(pdev, pos + PCI_DVSEC_HEADER2, &id);
if (vendor == PCI_DVSEC_VENDOR_ID_CXL && dvsec == id)
return pos;
pos = pci_find_next_ext_capability(pdev, pos,
PCI_EXT_CAP_ID_DVSEC);
}
return 0;
}
static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
struct device *dev = &pdev->dev;
int regloc;
regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC_OFFSET);
if (!regloc) {
dev_err(dev, "register location dvsec not found\n");
return -ENXIO;
}
return 0;
}
static const struct pci_device_id cxl_mem_pci_tbl[] = {
/* PCI class code for CXL.mem Type-3 Devices */
{ PCI_DEVICE_CLASS((PCI_CLASS_MEMORY_CXL << 8 | CXL_MEMORY_PROGIF), ~0)},
{ /* terminate list */ },
};
MODULE_DEVICE_TABLE(pci, cxl_mem_pci_tbl);
static struct pci_driver cxl_mem_driver = {
.name = KBUILD_MODNAME,
.id_table = cxl_mem_pci_tbl,
.probe = cxl_mem_probe,
.driver = {
.probe_type = PROBE_PREFER_ASYNCHRONOUS,
},
};
MODULE_LICENSE("GPL v2");
module_pci_driver(cxl_mem_driver);

17
drivers/cxl/pci.h Normal file
View File

@ -0,0 +1,17 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
#ifndef __CXL_PCI_H__
#define __CXL_PCI_H__
#define CXL_MEMORY_PROGIF 0x10
/*
* See section 8.1 Configuration Space Registers in the CXL 2.0
* Specification
*/
#define PCI_DVSEC_VENDOR_ID_CXL 0x1E98
#define PCI_DVSEC_ID_CXL 0x0
#define PCI_DVSEC_ID_CXL_REGLOC_OFFSET 0x8
#endif /* __CXL_PCI_H__ */

View File

@ -51,6 +51,7 @@
#define PCI_BASE_CLASS_MEMORY 0x05
#define PCI_CLASS_MEMORY_RAM 0x0500
#define PCI_CLASS_MEMORY_FLASH 0x0501
#define PCI_CLASS_MEMORY_CXL 0x0502
#define PCI_CLASS_MEMORY_OTHER 0x0580
#define PCI_BASE_CLASS_BRIDGE 0x06