Will Deacon 04867a7a33 swiotlb: Fix double-allocation of slots due to broken alignment handling ... Commit bbb73a103f ("swiotlb: fix a braino in the alignment check fix"), which was a fix for commit 0eee5ae102 ("swiotlb: fix slot alignment checks"), causes a functional regression with vsock in a virtual machine using bouncing via a restricted DMA SWIOTLB pool. When virtio allocates the virtqueues for the vsock device using dma_alloc_coherent(), the SWIOTLB search can return page-unaligned allocations if 'area->index' was left unaligned by a previous allocation from the buffer: # Final address in brackets is the SWIOTLB address returned to the caller | virtio-pci 0000:00:07.0: orig_addr 0x0 alloc_size 0x2000, iotlb_align_mask 0x800 stride 0x2: got slot 1645-1649/7168 (0x98326800) | virtio-pci 0000:00:07.0: orig_addr 0x0 alloc_size 0x2000, iotlb_align_mask 0x800 stride 0x2: got slot 1649-1653/7168 (0x98328800) | virtio-pci 0000:00:07.0: orig_addr 0x0 alloc_size 0x2000, iotlb_align_mask 0x800 stride 0x2: got slot 1653-1657/7168 (0x9832a800) This ends badly (typically buffer corruption and/or a hang) because swiotlb_alloc() is expecting a page-aligned allocation and so blindly returns a pointer to the 'struct page' corresponding to the allocation, therefore double-allocating the first half (2KiB slot) of the 4KiB page. Fix the problem by treating the allocation alignment separately to any additional alignment requirements from the device, using the maximum of the two as the stride to search the buffer slots and taking care to ensure a minimum of page-alignment for buffers larger than a page. This also resolves swiotlb allocation failures occuring due to the inclusion of ~PAGE_MASK in 'iotlb_align_mask' for large allocations and resulting in alignment requirements exceeding swiotlb_max_mapping_size(). Fixes: bbb73a103f ("swiotlb: fix a braino in the alignment check fix") Fixes: 0eee5ae102 ("swiotlb: fix slot alignment checks") Signed-off-by: Will Deacon <will@kernel.org> Reviewed-by: Michael Kelley <mhklinux@outlook.com> Reviewed-by: Petr Tesarik <petr.tesarik1@huawei-partners.com> Tested-by: Nicolin Chen <nicolinc@nvidia.com> Tested-by: Michael Kelley <mhklinux@outlook.com> Signed-off-by: Christoph Hellwig <hch@lst.de>		2024-03-13 11:39:17 -07:00
..
coherent.c	dma-mapping: clear dev->dma_mem to NULL after freeing it	2023-12-15 12:32:45 +01:00
contiguous.c	Revert "dma-contiguous: check for memory region overlap"	2023-09-08 05:58:32 -03:00
debug.c	dma-mapping fixes for Linux 6.8	2024-01-18 16:49:34 -08:00
debug.h	dma-debug: teach add_dma_entry() about DMA_ATTR_SKIP_CPU_SYNC	2021-10-18 12:46:45 +02:00
direct.c	dma-direct: Leak pages on dma_set_decrypted() failure	2024-02-28 05:31:38 -08:00
direct.h	dma-mapping: fix dma_addressing_limited() if dma_range_map can't cover all system RAM	2023-11-06 08:38:16 +01:00
dummy.c	dma-mapping: return error code from dma_dummy_map_sg()	2021-08-09 17:13:06 +02:00
Kconfig	dma-direct: add a CONFIG_ARCH_HAS_DMA_ALLOC symbol	2023-10-22 16:38:54 +02:00
Makefile	dma-mapping: remove CONFIG_DMA_REMAP	2022-03-03 14:00:57 +03:00
map_benchmark.c	dma-mapping: benchmark: remove MODULE_LICENSE in non-modules	2023-04-13 13:13:51 -07:00
mapping.c	dma-mapping: fix dma_addressing_limited() if dma_range_map can't cover all system RAM	2023-11-06 08:38:16 +01:00
ops_helpers.c	dma-mapping: handle vmalloc addresses in dma_common_{mmap,get_sgtable}	2021-07-16 11:30:26 +02:00
pool.c	mm, treewide: rename MAX_ORDER to MAX_PAGE_ORDER	2024-01-08 15:27:15 -08:00
remap.c	dma-remap: use kvmalloc_array/kvfree for larger dma memory remap	2023-06-07 15:06:28 +02:00
swiotlb.c	swiotlb: Fix double-allocation of slots due to broken alignment handling	2024-03-13 11:39:17 -07:00