linux-stable/drivers/iommu
Jason Gunthorpe 916f0958ea iommu/arm-smmu-v3: Hold arm_smmu_asid_lock during all of attach_dev
[ Upstream commit 9f7c689115 ]

The BTM support wants to be able to change the ASID of any smmu_domain.
When it goes to do this it holds the arm_smmu_asid_lock and iterates over
the target domain's devices list.

During attach of a S1 domain we must ensure that the devices list and
CD are in sync, otherwise we could miss CD updates or a parallel CD update
could push an out of date CD.

This is pretty complicated, and almost works today because
arm_smmu_detach_dev() removes the master from the linked list before
working on the CD entries, preventing parallel update of the CD.

However, it does have an issue where the CD can remain programed while the
domain appears to be unattached. arm_smmu_share_asid() will then not clear
any CD entriess and install its own CD entry with the same ASID
concurrently. This creates a small race window where the IOMMU can see two
ASIDs pointing to different translations.

       CPU0                                   CPU1
arm_smmu_attach_dev()
   arm_smmu_detach_dev()
     spin_lock_irqsave(&smmu_domain->devices_lock, flags);
     list_del(&master->domain_head);
     spin_unlock_irqrestore(&smmu_domain->devices_lock, flags);

				      arm_smmu_mmu_notifier_get()
				       arm_smmu_alloc_shared_cd()
					arm_smmu_share_asid():
                                          // Does nothing due to list_del above
					  arm_smmu_update_ctx_desc_devices()
					  arm_smmu_tlb_inv_asid()
				       arm_smmu_write_ctx_desc()
					 ** Now the ASID is in two CDs
					    with different translation

     arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, NULL);

Solve this by wrapping most of the attach flow in the
arm_smmu_asid_lock. This locks more than strictly needed to prepare for
the next patch which will reorganize the order of the linked list, STE and
CD changes.

Move arm_smmu_detach_dev() till after we have initialized the domain so
the lock can be held for less time.

Reviewed-by: Michael Shavit <mshavit@google.com>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Mostafa Saleh <smostafa@google.com>
Tested-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Tested-by: Moritz Fischer <moritzf@google.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Link: https://lore.kernel.org/r/5-v6-96275f25c39d+2d4-smmuv3_newapi_p1_jgg@nvidia.com
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2024-04-13 13:10:01 +02:00
..
amd iommu/amd: Mark interrupt as managed 2024-03-26 18:16:44 -04:00
arm iommu/arm-smmu-v3: Hold arm_smmu_asid_lock during all of attach_dev 2024-04-13 13:10:01 +02:00
intel iommu: Fix compilation without CONFIG_IOMMU_INTEL 2024-03-26 18:16:55 -04:00
iommufd iommufd/selftest: Don't check map/unmap pairing with HUGE_PAGES 2024-02-26 16:59:12 -04:00
Kconfig iommu: Fix compilation without CONFIG_IOMMU_INTEL 2024-03-26 18:16:55 -04:00
Makefile iommu/tegra-gart: Remove tegra-gart 2023-09-25 11:40:54 +02:00
apple-dart.c Merge branches 'apple/dart', 'arm/rockchip', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd' and 'core' into next 2024-01-03 09:59:32 +01:00
dma-iommu.c iommu/dma: Force swiotlb_max_mapping_size on an untrusted device 2024-04-03 15:32:33 +02:00
dma-iommu.h iommu: Optimise PCI SAC address trick 2023-07-14 16:14:17 +02:00
exynos-iommu.c iommu/exynos: Update to {map,unmap}_pages 2023-09-25 12:04:36 +02:00
fsl_pamu.c iommu/fsl: fix all kernel-doc warnings in fsl_pamu.c 2023-03-22 14:50:15 +01:00
fsl_pamu.h
fsl_pamu_domain.c iommu/fsl_pamu: Implement a PLATFORM domain 2023-09-25 11:40:54 +02:00
fsl_pamu_domain.h
hyperv-iommu.c x86/vector: Rename send_cleanup_vector() to vector_schedule_cleanup() 2023-08-06 14:15:09 +02:00
io-pgfault.c iommu: Rename iommu-sva-lib.{c,h} 2022-11-03 15:47:54 +01:00
io-pgtable-arm-v7s.c iommu/io-pgtable-arm-v7s: Remove map/unmap 2022-11-19 10:44:15 +01:00
io-pgtable-arm.c iommu: Extend LPAE page table format to support custom allocators 2023-11-27 11:10:12 +01:00
io-pgtable-arm.h
io-pgtable-dart.c
io-pgtable.c iommu: Allow passing custom allocators to pgtable drivers 2023-11-27 11:10:12 +01:00
iommu-debugfs.c
iommu-priv.h iommu: Move dev_iommu_ops() to private header 2023-08-18 12:52:15 -03:00
iommu-sva.c iommu/sva: Fix SVA handle sharing in multi device case 2024-03-01 13:53:58 +01:00
iommu-sva.h iommu: Remove ioasid infrastructure 2023-03-31 10:03:31 +02:00
iommu-sysfs.c iommu: Do not export iommu_device_link/unlink() 2023-07-14 16:14:15 +02:00
iommu-traces.c iommu: Remove detach_dev callback 2023-01-13 16:39:18 +01:00
iommu.c iommu: Validate the PASID in iommu_attach_device_pasid() 2024-04-10 16:37:59 +02:00
iova.c iommu/iova: Manage the depot list size 2023-09-25 12:07:44 +02:00
ipmmu-vmsa.c iommu/ipmmu-vmsa: Convert to generic_single_device_group() 2023-09-25 11:51:07 +02:00
irq_remapping.c iommu: Fix compilation without CONFIG_IOMMU_INTEL 2024-03-26 18:16:55 -04:00
irq_remapping.h
msm_iommu.c iommu: Allow .iotlb_sync_map to fail and handle s390's -ENOMEM return 2023-10-02 08:42:57 +02:00
msm_iommu.h
msm_iommu_hw-8xxx.h
mtk_iommu.c iommu: Clean up open-coded ownership checks 2023-11-27 11:03:16 +01:00
mtk_iommu_v1.c iommu: Clean up open-coded ownership checks 2023-11-27 11:03:16 +01:00
of_iommu.c Merge branches 'apple/dart', 'arm/rockchip', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd' and 'core' into next 2024-01-03 09:59:32 +01:00
omap-iommu-debug.c
omap-iommu.c iommu: Mark dev_iommu_priv_set() with a lockdep 2023-12-12 10:18:49 +01:00
omap-iommu.h iommu/omap: Convert to generic_single_device_group() 2023-09-25 11:52:08 +02:00
omap-iopgtable.h
rockchip-iommu.c iommu/rockchip: Update to {map,unmap}_pages 2023-09-25 12:04:37 +02:00
s390-iommu.c iommu/dma: Allow a single FQ in addition to per-CPU FQs 2023-10-02 08:43:03 +02:00
sprd-iommu.c iommu: Clean up open-coded ownership checks 2023-11-27 11:03:16 +01:00
sun50i-iommu.c iommu: Allow .iotlb_sync_map to fail and handle s390's -ENOMEM return 2023-10-02 08:42:57 +02:00
tegra-smmu.c Merge branches 'iommu/fixes', 'arm/tegra', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd', 'core' and 's390' into next 2023-10-27 09:13:40 +02:00
virtio-iommu.c Merge branches 'apple/dart', 'arm/rockchip', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd' and 'core' into next 2024-01-03 09:59:32 +01:00