2019-05-27 06:55:21 +00:00
|
|
|
/* SPDX-License-Identifier: GPL-2.0-only */
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
/*
|
|
|
|
* Fence mechanism for dma-buf to allow for asynchronous dma access
|
|
|
|
*
|
|
|
|
* Copyright (C) 2012 Canonical Ltd
|
|
|
|
* Copyright (C) 2012 Texas Instruments
|
|
|
|
*
|
|
|
|
* Authors:
|
|
|
|
* Rob Clark <robdclark@gmail.com>
|
|
|
|
* Maarten Lankhorst <maarten.lankhorst@canonical.com>
|
|
|
|
*/
|
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
#ifndef __LINUX_DMA_FENCE_H
|
|
|
|
#define __LINUX_DMA_FENCE_H
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
|
|
|
|
#include <linux/err.h>
|
|
|
|
#include <linux/wait.h>
|
|
|
|
#include <linux/list.h>
|
|
|
|
#include <linux/bitops.h>
|
|
|
|
#include <linux/kref.h>
|
|
|
|
#include <linux/sched.h>
|
|
|
|
#include <linux/printk.h>
|
2014-07-01 10:58:00 +00:00
|
|
|
#include <linux/rcupdate.h>
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
struct dma_fence;
|
|
|
|
struct dma_fence_ops;
|
|
|
|
struct dma_fence_cb;
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
|
|
|
|
/**
|
2016-10-25 12:00:45 +00:00
|
|
|
* struct dma_fence - software synchronization primitive
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
* @refcount: refcount for this fence
|
2016-10-25 12:00:45 +00:00
|
|
|
* @ops: dma_fence_ops associated with this fence
|
2014-07-01 10:58:00 +00:00
|
|
|
* @rcu: used for releasing fence with kfree_rcu
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
* @cb_list: list of all callbacks to call
|
|
|
|
* @lock: spin_lock_irqsave used for locking
|
|
|
|
* @context: execution context this fence belongs to, returned by
|
2016-10-25 12:00:45 +00:00
|
|
|
* dma_fence_context_alloc()
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
* @seqno: the sequence number of this fence inside the execution context,
|
|
|
|
* can be compared to decide which fence would be signaled later.
|
2016-10-25 12:00:45 +00:00
|
|
|
* @flags: A mask of DMA_FENCE_FLAG_* defined below
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
* @timestamp: Timestamp when the fence was signaled.
|
2017-01-04 14:12:22 +00:00
|
|
|
* @error: Optional, only valid if < 0, must be set before calling
|
2016-10-25 12:00:45 +00:00
|
|
|
* dma_fence_signal, indicates that the fence has completed with an error.
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*
|
|
|
|
* the flags member must be manipulated and read using the appropriate
|
|
|
|
* atomic ops (bit_*), so taking the spinlock will not be needed most
|
|
|
|
* of the time.
|
|
|
|
*
|
2016-10-25 12:00:45 +00:00
|
|
|
* DMA_FENCE_FLAG_SIGNALED_BIT - fence is already signaled
|
2017-02-14 12:40:01 +00:00
|
|
|
* DMA_FENCE_FLAG_TIMESTAMP_BIT - timestamp recorded for fence signaling
|
2016-10-25 12:00:45 +00:00
|
|
|
* DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT - enable_signaling might have been called
|
|
|
|
* DMA_FENCE_FLAG_USER_BITS - start of the unused bits, can be used by the
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
* implementer of the fence for its own purposes. Can be used in different
|
|
|
|
* ways by different fence implementers, so do not rely on this.
|
|
|
|
*
|
2016-08-11 10:47:58 +00:00
|
|
|
* Since atomic bitops are used, this is not guaranteed to be the case.
|
2016-10-25 12:00:45 +00:00
|
|
|
* Particularly, if the bit was set, but dma_fence_signal was called right
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
* before this bit was set, it would have been able to set the
|
2016-10-25 12:00:45 +00:00
|
|
|
* DMA_FENCE_FLAG_SIGNALED_BIT, before enable_signaling was called.
|
|
|
|
* Adding a check for DMA_FENCE_FLAG_SIGNALED_BIT after setting
|
|
|
|
* DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT closes this race, and makes sure that
|
|
|
|
* after dma_fence_signal was called, any enable_signaling call will have either
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
* been completed, or never called at all.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
struct dma_fence {
|
2019-08-17 14:47:33 +00:00
|
|
|
spinlock_t *lock;
|
2016-10-25 12:00:45 +00:00
|
|
|
const struct dma_fence_ops *ops;
|
2019-08-17 15:30:22 +00:00
|
|
|
/*
|
|
|
|
* We clear the callback list on kref_put so that by the time we
|
|
|
|
* release the fence it is unused. No one should be adding to the
|
|
|
|
* cb_list that they don't themselves hold a reference for.
|
|
|
|
*
|
|
|
|
* The lifetime of the timestamp is similarly tied to both the
|
|
|
|
* rcu freelist and the cb_list. The timestamp is only set upon
|
|
|
|
* signaling while simultaneously notifying the cb_list. Ergo, we
|
|
|
|
* only use either the cb_list of timestamp. Upon destruction,
|
|
|
|
* neither are accessible, and so we can use the rcu. This means
|
|
|
|
* that the cb_list is *only* valid until the signal bit is set,
|
|
|
|
* and to read either you *must* hold a reference to the fence,
|
|
|
|
* and not just the rcu_read_lock.
|
|
|
|
*
|
|
|
|
* Listed in chronological order.
|
2019-08-07 10:31:48 +00:00
|
|
|
*/
|
|
|
|
union {
|
|
|
|
struct list_head cb_list;
|
2019-08-17 15:30:22 +00:00
|
|
|
/* @cb_list replaced by @timestamp on dma_fence_signal() */
|
|
|
|
ktime_t timestamp;
|
|
|
|
/* @timestamp replaced by @rcu on dma_fence_release() */
|
|
|
|
struct rcu_head rcu;
|
2019-08-07 10:31:48 +00:00
|
|
|
};
|
2016-06-01 13:10:02 +00:00
|
|
|
u64 context;
|
2018-11-14 15:11:06 +00:00
|
|
|
u64 seqno;
|
2019-08-17 14:47:33 +00:00
|
|
|
unsigned long flags;
|
|
|
|
struct kref refcount;
|
2017-01-04 14:12:22 +00:00
|
|
|
int error;
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
};
|
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
enum dma_fence_flag_bits {
|
|
|
|
DMA_FENCE_FLAG_SIGNALED_BIT,
|
2017-02-14 12:40:01 +00:00
|
|
|
DMA_FENCE_FLAG_TIMESTAMP_BIT,
|
2016-10-25 12:00:45 +00:00
|
|
|
DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT,
|
|
|
|
DMA_FENCE_FLAG_USER_BITS, /* must always be last member */
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
};
|
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
typedef void (*dma_fence_func_t)(struct dma_fence *fence,
|
|
|
|
struct dma_fence_cb *cb);
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
|
|
|
|
/**
|
2018-04-27 06:17:08 +00:00
|
|
|
* struct dma_fence_cb - callback for dma_fence_add_callback()
|
|
|
|
* @node: used by dma_fence_add_callback() to append this struct to fence::cb_list
|
2016-10-25 12:00:45 +00:00
|
|
|
* @func: dma_fence_func_t to call
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*
|
2018-04-27 06:17:08 +00:00
|
|
|
* This struct will be initialized by dma_fence_add_callback(), additional
|
2016-10-25 12:00:45 +00:00
|
|
|
* data can be passed along by embedding dma_fence_cb in another struct.
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
struct dma_fence_cb {
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
struct list_head node;
|
2016-10-25 12:00:45 +00:00
|
|
|
dma_fence_func_t func;
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
/**
|
2016-10-25 12:00:45 +00:00
|
|
|
* struct dma_fence_ops - operations implemented for fence
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
struct dma_fence_ops {
|
2019-04-15 12:46:34 +00:00
|
|
|
/**
|
|
|
|
* @use_64bit_seqno:
|
|
|
|
*
|
|
|
|
* True if this dma_fence implementation uses 64bit seqno, false
|
|
|
|
* otherwise.
|
|
|
|
*/
|
|
|
|
bool use_64bit_seqno;
|
|
|
|
|
2018-04-27 06:17:08 +00:00
|
|
|
/**
|
|
|
|
* @get_driver_name:
|
|
|
|
*
|
|
|
|
* Returns the driver name. This is a callback to allow drivers to
|
|
|
|
* compute the name at runtime, without having it to store permanently
|
|
|
|
* for each fence, or build a cache of some sort.
|
|
|
|
*
|
|
|
|
* This callback is mandatory.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
const char * (*get_driver_name)(struct dma_fence *fence);
|
2018-04-27 06:17:08 +00:00
|
|
|
|
|
|
|
/**
|
|
|
|
* @get_timeline_name:
|
|
|
|
*
|
|
|
|
* Return the name of the context this fence belongs to. This is a
|
|
|
|
* callback to allow drivers to compute the name at runtime, without
|
|
|
|
* having it to store permanently for each fence, or build a cache of
|
|
|
|
* some sort.
|
|
|
|
*
|
|
|
|
* This callback is mandatory.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
const char * (*get_timeline_name)(struct dma_fence *fence);
|
2018-04-27 06:17:08 +00:00
|
|
|
|
|
|
|
/**
|
|
|
|
* @enable_signaling:
|
|
|
|
*
|
|
|
|
* Enable software signaling of fence.
|
|
|
|
*
|
|
|
|
* For fence implementations that have the capability for hw->hw
|
|
|
|
* signaling, they can implement this op to enable the necessary
|
|
|
|
* interrupts, or insert commands into cmdstream, etc, to avoid these
|
|
|
|
* costly operations for the common case where only hw->hw
|
|
|
|
* synchronization is required. This is called in the first
|
|
|
|
* dma_fence_wait() or dma_fence_add_callback() path to let the fence
|
|
|
|
* implementation know that there is another driver waiting on the
|
|
|
|
* signal (ie. hw->sw case).
|
|
|
|
*
|
|
|
|
* This function can be called from atomic context, but not
|
|
|
|
* from irq context, so normal spinlocks can be used.
|
|
|
|
*
|
|
|
|
* A return value of false indicates the fence already passed,
|
|
|
|
* or some failure occurred that made it impossible to enable
|
|
|
|
* signaling. True indicates successful enabling.
|
|
|
|
*
|
|
|
|
* &dma_fence.error may be set in enable_signaling, but only when false
|
|
|
|
* is returned.
|
|
|
|
*
|
|
|
|
* Since many implementations can call dma_fence_signal() even when before
|
|
|
|
* @enable_signaling has been called there's a race window, where the
|
|
|
|
* dma_fence_signal() might result in the final fence reference being
|
|
|
|
* released and its memory freed. To avoid this, implementations of this
|
|
|
|
* callback should grab their own reference using dma_fence_get(), to be
|
|
|
|
* released when the fence is signalled (through e.g. the interrupt
|
|
|
|
* handler).
|
|
|
|
*
|
2018-05-04 14:10:34 +00:00
|
|
|
* This callback is optional. If this callback is not present, then the
|
|
|
|
* driver must always have signaling enabled.
|
2018-04-27 06:17:08 +00:00
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
bool (*enable_signaling)(struct dma_fence *fence);
|
2018-04-27 06:17:08 +00:00
|
|
|
|
|
|
|
/**
|
|
|
|
* @signaled:
|
|
|
|
*
|
|
|
|
* Peek whether the fence is signaled, as a fastpath optimization for
|
|
|
|
* e.g. dma_fence_wait() or dma_fence_add_callback(). Note that this
|
|
|
|
* callback does not need to make any guarantees beyond that a fence
|
|
|
|
* once indicates as signalled must always return true from this
|
|
|
|
* callback. This callback may return false even if the fence has
|
|
|
|
* completed already, in this case information hasn't propogated throug
|
|
|
|
* the system yet. See also dma_fence_is_signaled().
|
|
|
|
*
|
|
|
|
* May set &dma_fence.error if returning true.
|
|
|
|
*
|
|
|
|
* This callback is optional.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
bool (*signaled)(struct dma_fence *fence);
|
2018-04-27 06:17:08 +00:00
|
|
|
|
|
|
|
/**
|
|
|
|
* @wait:
|
|
|
|
*
|
2018-05-03 14:25:52 +00:00
|
|
|
* Custom wait implementation, defaults to dma_fence_default_wait() if
|
|
|
|
* not set.
|
2018-04-27 06:17:08 +00:00
|
|
|
*
|
2021-07-21 09:01:04 +00:00
|
|
|
* Deprecated and should not be used by new implementations. Only used
|
|
|
|
* by existing implementations which need special handling for their
|
|
|
|
* hardware reset procedure.
|
2018-04-27 06:17:08 +00:00
|
|
|
*
|
|
|
|
* Must return -ERESTARTSYS if the wait is intr = true and the wait was
|
|
|
|
* interrupted, and remaining jiffies if fence has signaled, or 0 if wait
|
|
|
|
* timed out. Can also return other error values on custom implementations,
|
|
|
|
* which should be treated as if the fence is signaled. For example a hardware
|
|
|
|
* lockup could be reported like that.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
signed long (*wait)(struct dma_fence *fence,
|
|
|
|
bool intr, signed long timeout);
|
2018-04-27 06:17:08 +00:00
|
|
|
|
|
|
|
/**
|
|
|
|
* @release:
|
|
|
|
*
|
|
|
|
* Called on destruction of fence to release additional resources.
|
|
|
|
* Can be called from irq context. This callback is optional. If it is
|
|
|
|
* NULL, then dma_fence_free() is instead called as the default
|
|
|
|
* implementation.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
void (*release)(struct dma_fence *fence);
|
|
|
|
|
2018-04-27 06:17:08 +00:00
|
|
|
/**
|
|
|
|
* @fence_value_str:
|
|
|
|
*
|
|
|
|
* Callback to fill in free-form debug info specific to this fence, like
|
|
|
|
* the sequence number.
|
|
|
|
*
|
|
|
|
* This callback is optional.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
void (*fence_value_str)(struct dma_fence *fence, char *str, int size);
|
2018-04-27 06:17:08 +00:00
|
|
|
|
|
|
|
/**
|
|
|
|
* @timeline_value_str:
|
|
|
|
*
|
|
|
|
* Fills in the current value of the timeline as a string, like the
|
2018-05-03 14:25:49 +00:00
|
|
|
* sequence number. Note that the specific fence passed to this function
|
|
|
|
* should not matter, drivers should only use it to look up the
|
|
|
|
* corresponding timeline structures.
|
2018-04-27 06:17:08 +00:00
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
void (*timeline_value_str)(struct dma_fence *fence,
|
|
|
|
char *str, int size);
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
};
|
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
void dma_fence_init(struct dma_fence *fence, const struct dma_fence_ops *ops,
|
2018-11-14 15:11:06 +00:00
|
|
|
spinlock_t *lock, u64 context, u64 seqno);
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
void dma_fence_release(struct kref *kref);
|
|
|
|
void dma_fence_free(struct dma_fence *fence);
|
2021-09-23 11:57:42 +00:00
|
|
|
void dma_fence_describe(struct dma_fence *fence, struct seq_file *seq);
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
|
2016-08-29 07:08:29 +00:00
|
|
|
/**
|
2016-10-25 12:00:45 +00:00
|
|
|
* dma_fence_put - decreases refcount of the fence
|
2018-04-27 06:17:08 +00:00
|
|
|
* @fence: fence to reduce refcount of
|
2016-08-29 07:08:29 +00:00
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
static inline void dma_fence_put(struct dma_fence *fence)
|
2016-08-29 07:08:29 +00:00
|
|
|
{
|
|
|
|
if (fence)
|
2016-10-25 12:00:45 +00:00
|
|
|
kref_put(&fence->refcount, dma_fence_release);
|
2016-08-29 07:08:29 +00:00
|
|
|
}
|
|
|
|
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
/**
|
2016-10-25 12:00:45 +00:00
|
|
|
* dma_fence_get - increases refcount of the fence
|
2018-04-27 06:17:08 +00:00
|
|
|
* @fence: fence to increase refcount of
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*
|
|
|
|
* Returns the same fence, with refcount increased by 1.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
static inline struct dma_fence *dma_fence_get(struct dma_fence *fence)
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
{
|
|
|
|
if (fence)
|
|
|
|
kref_get(&fence->refcount);
|
|
|
|
return fence;
|
|
|
|
}
|
|
|
|
|
2014-07-01 10:58:00 +00:00
|
|
|
/**
|
2019-08-11 08:06:32 +00:00
|
|
|
* dma_fence_get_rcu - get a fence from a dma_resv_list with
|
2016-10-25 12:00:45 +00:00
|
|
|
* rcu read lock
|
2018-04-27 06:17:08 +00:00
|
|
|
* @fence: fence to increase refcount of
|
2014-07-01 10:58:00 +00:00
|
|
|
*
|
|
|
|
* Function returns NULL if no refcount could be obtained, or the fence.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
static inline struct dma_fence *dma_fence_get_rcu(struct dma_fence *fence)
|
2014-07-01 10:58:00 +00:00
|
|
|
{
|
|
|
|
if (kref_get_unless_zero(&fence->refcount))
|
|
|
|
return fence;
|
|
|
|
else
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
/**
|
2016-10-25 12:00:45 +00:00
|
|
|
* dma_fence_get_rcu_safe - acquire a reference to an RCU tracked fence
|
2018-04-27 06:17:08 +00:00
|
|
|
* @fencep: pointer to fence to increase refcount of
|
2016-08-29 07:08:29 +00:00
|
|
|
*
|
|
|
|
* Function returns NULL if no refcount could be obtained, or the fence.
|
|
|
|
* This function handles acquiring a reference to a fence that may be
|
2017-01-18 10:53:44 +00:00
|
|
|
* reallocated within the RCU grace period (such as with SLAB_TYPESAFE_BY_RCU),
|
2016-08-29 07:08:29 +00:00
|
|
|
* so long as the caller is using RCU on the pointer to the fence.
|
|
|
|
*
|
|
|
|
* An alternative mechanism is to employ a seqlock to protect a bunch of
|
2019-08-11 08:06:32 +00:00
|
|
|
* fences, such as used by struct dma_resv. When using a seqlock,
|
2016-08-29 07:08:29 +00:00
|
|
|
* the seqlock must be taken before and checked after a reference to the
|
|
|
|
* fence is acquired (as shown here).
|
|
|
|
*
|
|
|
|
* The caller is required to hold the RCU read lock.
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
static inline struct dma_fence *
|
2017-11-02 20:03:34 +00:00
|
|
|
dma_fence_get_rcu_safe(struct dma_fence __rcu **fencep)
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
{
|
2016-08-29 07:08:29 +00:00
|
|
|
do {
|
2016-10-25 12:00:45 +00:00
|
|
|
struct dma_fence *fence;
|
2016-08-29 07:08:29 +00:00
|
|
|
|
|
|
|
fence = rcu_dereference(*fencep);
|
2017-09-15 09:53:07 +00:00
|
|
|
if (!fence)
|
2016-08-29 07:08:29 +00:00
|
|
|
return NULL;
|
|
|
|
|
2017-09-15 09:53:07 +00:00
|
|
|
if (!dma_fence_get_rcu(fence))
|
|
|
|
continue;
|
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
/* The atomic_inc_not_zero() inside dma_fence_get_rcu()
|
2016-08-29 07:08:29 +00:00
|
|
|
* provides a full memory barrier upon success (such as now).
|
|
|
|
* This is paired with the write barrier from assigning
|
|
|
|
* to the __rcu protected fence pointer so that if that
|
|
|
|
* pointer still matches the current fence, we know we
|
|
|
|
* have successfully acquire a reference to it. If it no
|
|
|
|
* longer matches, we are holding a reference to some other
|
|
|
|
* reallocated pointer. This is possible if the allocator
|
2017-01-18 10:53:44 +00:00
|
|
|
* is using a freelist like SLAB_TYPESAFE_BY_RCU where the
|
2016-08-29 07:08:29 +00:00
|
|
|
* fence remains valid for the RCU grace period, but it
|
|
|
|
* may be reallocated. When using such allocators, we are
|
|
|
|
* responsible for ensuring the reference we get is to
|
|
|
|
* the right fence, as below.
|
|
|
|
*/
|
|
|
|
if (fence == rcu_access_pointer(*fencep))
|
|
|
|
return rcu_pointer_handoff(fence);
|
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
dma_fence_put(fence);
|
2016-08-29 07:08:29 +00:00
|
|
|
} while (1);
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
}
|
|
|
|
|
dma-fence: basic lockdep annotations
Design is similar to the lockdep annotations for workers, but with
some twists:
- We use a read-lock for the execution/worker/completion side, so that
this explicit annotation can be more liberally sprinkled around.
With read locks lockdep isn't going to complain if the read-side
isn't nested the same way under all circumstances, so ABBA deadlocks
are ok. Which they are, since this is an annotation only.
- We're using non-recursive lockdep read lock mode, since in recursive
read lock mode lockdep does not catch read side hazards. And we
_very_ much want read side hazards to be caught. For full details of
this limitation see
commit e91498589746065e3ae95d9a00b068e525eec34f
Author: Peter Zijlstra <peterz@infradead.org>
Date: Wed Aug 23 13:13:11 2017 +0200
locking/lockdep/selftests: Add mixed read-write ABBA tests
- To allow nesting of the read-side explicit annotations we explicitly
keep track of the nesting. lock_is_held() allows us to do that.
- The wait-side annotation is a write lock, and entirely done within
dma_fence_wait() for everyone by default.
- To be able to freely annotate helper functions I want to make it ok
to call dma_fence_begin/end_signalling from soft/hardirq context.
First attempt was using the hardirq locking context for the write
side in lockdep, but this forces all normal spinlocks nested within
dma_fence_begin/end_signalling to be spinlocks. That bollocks.
The approach now is to simple check in_atomic(), and for these cases
entirely rely on the might_sleep() check in dma_fence_wait(). That
will catch any wrong nesting against spinlocks from soft/hardirq
contexts.
The idea here is that every code path that's critical for eventually
signalling a dma_fence should be annotated with
dma_fence_begin/end_signalling. The annotation ideally starts right
after a dma_fence is published (added to a dma_resv, exposed as a
sync_file fd, attached to a drm_syncobj fd, or anything else that
makes the dma_fence visible to other kernel threads), up to and
including the dma_fence_wait(). Examples are irq handlers, the
scheduler rt threads, the tail of execbuf (after the corresponding
fences are visible), any workers that end up signalling dma_fences and
really anything else. Not annotated should be code paths that only
complete fences opportunistically as the gpu progresses, like e.g.
shrinker/eviction code.
The main class of deadlocks this is supposed to catch are:
Thread A:
mutex_lock(A);
mutex_unlock(A);
dma_fence_signal();
Thread B:
mutex_lock(A);
dma_fence_wait();
mutex_unlock(A);
Thread B is blocked on A signalling the fence, but A never gets around
to that because it cannot acquire the lock A.
Note that dma_fence_wait() is allowed to be nested within
dma_fence_begin/end_signalling sections. To allow this to happen the
read lock needs to be upgraded to a write lock, which means that any
other lock is acquired between the dma_fence_begin_signalling() call and
the call to dma_fence_wait(), and still held, this will result in an
immediate lockdep complaint. The only other option would be to not
annotate such calls, defeating the point. Therefore these annotations
cannot be sprinkled over the code entirely mindless to avoid false
positives.
Originally I hope that the cross-release lockdep extensions would
alleviate the need for explicit annotations:
https://lwn.net/Articles/709849/
But there's a few reasons why that's not an option:
- It's not happening in upstream, since it got reverted due to too
many false positives:
commit e966eaeeb623f09975ef362c2866fae6f86844f9
Author: Ingo Molnar <mingo@kernel.org>
Date: Tue Dec 12 12:31:16 2017 +0100
locking/lockdep: Remove the cross-release locking checks
This code (CONFIG_LOCKDEP_CROSSRELEASE=y and CONFIG_LOCKDEP_COMPLETIONS=y),
while it found a number of old bugs initially, was also causing too many
false positives that caused people to disable lockdep - which is arguably
a worse overall outcome.
- cross-release uses the complete() call to annotate the end of
critical sections, for dma_fence that would be dma_fence_signal().
But we do not want all dma_fence_signal() calls to be treated as
critical, since many are opportunistic cleanup of gpu requests. If
these get stuck there's still the main completion interrupt and
workers who can unblock everyone. Automatically annotating all
dma_fence_signal() calls would hence cause false positives.
- cross-release had some educated guesses for when a critical section
starts, like fresh syscall or fresh work callback. This would again
cause false positives without explicit annotations, since for
dma_fence the critical sections only starts when we publish a fence.
- Furthermore there can be cases where a thread never does a
dma_fence_signal, but is still critical for reaching completion of
fences. One example would be a scheduler kthread which picks up jobs
and pushes them into hardware, where the interrupt handler or
another completion thread calls dma_fence_signal(). But if the
scheduler thread hangs, then all the fences hang, hence we need to
manually annotate it. cross-release aimed to solve this by chaining
cross-release dependencies, but the dependency from scheduler thread
to the completion interrupt handler goes through hw where
cross-release code can't observe it.
In short, without manual annotations and careful review of the start
and end of critical sections, cross-relese dependency tracking doesn't
work. We need explicit annotations.
v2: handle soft/hardirq ctx better against write side and dont forget
EXPORT_SYMBOL, drivers can't use this otherwise.
v3: Kerneldoc.
v4: Some spelling fixes from Mika
v5: Amend commit message to explain in detail why cross-release isn't
the solution.
v6: Pull out misplaced .rst hunk.
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Cc: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Cc: Thomas Hellstrom <thomas.hellstrom@intel.com>
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: linux-rdma@vger.kernel.org
Cc: amd-gfx@lists.freedesktop.org
Cc: intel-gfx@lists.freedesktop.org
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200707201229.472834-2-daniel.vetter@ffwll.ch
2020-07-07 20:12:05 +00:00
|
|
|
#ifdef CONFIG_LOCKDEP
|
|
|
|
bool dma_fence_begin_signalling(void);
|
|
|
|
void dma_fence_end_signalling(bool cookie);
|
dma-fence: prime lockdep annotations
Two in one go:
- it is allowed to call dma_fence_wait() while holding a
dma_resv_lock(). This is fundamental to how eviction works with ttm,
so required.
- it is allowed to call dma_fence_wait() from memory reclaim contexts,
specifically from shrinker callbacks (which i915 does), and from mmu
notifier callbacks (which amdgpu does, and which i915 sometimes also
does, and probably always should, but that's kinda a debate). Also
for stuff like HMM we really need to be able to do this, or things
get real dicey.
Consequence is that any critical path necessary to get to a
dma_fence_signal for a fence must never a) call dma_resv_lock nor b)
allocate memory with GFP_KERNEL. Also by implication of
dma_resv_lock(), no userspace faulting allowed. That's some supremely
obnoxious limitations, which is why we need to sprinkle the right
annotations to all relevant paths.
The one big locking context we're leaving out here is mmu notifiers,
added in
commit 23b68395c7c78a764e8963fc15a7cfd318bf187f
Author: Daniel Vetter <daniel.vetter@ffwll.ch>
Date: Mon Aug 26 22:14:21 2019 +0200
mm/mmu_notifiers: add a lockdep map for invalidate_range_start/end
that one covers a lot of other callsites, and it's also allowed to
wait on dma-fences from mmu notifiers. But there's no ready-made
functions exposed to prime this, so I've left it out for now.
v2: Also track against mmu notifier context.
v3: kerneldoc to spec the cross-driver contract. Note that currently
i915 throws in a hard-coded 10s timeout on foreign fences (not sure
why that was done, but it's there), which is why that rule is worded
with SHOULD instead of MUST.
Also some of the mmu_notifier/shrinker rules might surprise SoC
drivers, I haven't fully audited them all. Which is infeasible anyway,
we'll need to run them with lockdep and dma-fence annotations and see
what goes boom.
v4: A spelling fix from Mika
v5: #ifdef for CONFIG_MMU_NOTIFIER. Reported by 0day. Unfortunately
this means lockdep enforcement is slightly inconsistent, it won't spot
GFP_NOIO and GFP_NOFS allocations in the wrong spot if
CONFIG_MMU_NOTIFIER is disabled in the kernel config. Oh well.
v5: Note that only drivers/gpu has a reasonable (or at least
historical) excuse to use dma_fence_wait() from shrinker and mmu
notifier callbacks. Everyone else should either have a better memory
manager model, or better hardware. This reflects discussions with
Jason Gunthorpe.
Cc: Jason Gunthorpe <jgg@mellanox.com>
Cc: Felix Kuehling <Felix.Kuehling@amd.com>
Cc: kernel test robot <lkp@intel.com>
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@intel.com> (v4)
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Cc: Thomas Hellstrom <thomas.hellstrom@intel.com>
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: linux-rdma@vger.kernel.org
Cc: amd-gfx@lists.freedesktop.org
Cc: intel-gfx@lists.freedesktop.org
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200707201229.472834-3-daniel.vetter@ffwll.ch
2020-07-07 20:12:06 +00:00
|
|
|
void __dma_fence_might_wait(void);
|
dma-fence: basic lockdep annotations
Design is similar to the lockdep annotations for workers, but with
some twists:
- We use a read-lock for the execution/worker/completion side, so that
this explicit annotation can be more liberally sprinkled around.
With read locks lockdep isn't going to complain if the read-side
isn't nested the same way under all circumstances, so ABBA deadlocks
are ok. Which they are, since this is an annotation only.
- We're using non-recursive lockdep read lock mode, since in recursive
read lock mode lockdep does not catch read side hazards. And we
_very_ much want read side hazards to be caught. For full details of
this limitation see
commit e91498589746065e3ae95d9a00b068e525eec34f
Author: Peter Zijlstra <peterz@infradead.org>
Date: Wed Aug 23 13:13:11 2017 +0200
locking/lockdep/selftests: Add mixed read-write ABBA tests
- To allow nesting of the read-side explicit annotations we explicitly
keep track of the nesting. lock_is_held() allows us to do that.
- The wait-side annotation is a write lock, and entirely done within
dma_fence_wait() for everyone by default.
- To be able to freely annotate helper functions I want to make it ok
to call dma_fence_begin/end_signalling from soft/hardirq context.
First attempt was using the hardirq locking context for the write
side in lockdep, but this forces all normal spinlocks nested within
dma_fence_begin/end_signalling to be spinlocks. That bollocks.
The approach now is to simple check in_atomic(), and for these cases
entirely rely on the might_sleep() check in dma_fence_wait(). That
will catch any wrong nesting against spinlocks from soft/hardirq
contexts.
The idea here is that every code path that's critical for eventually
signalling a dma_fence should be annotated with
dma_fence_begin/end_signalling. The annotation ideally starts right
after a dma_fence is published (added to a dma_resv, exposed as a
sync_file fd, attached to a drm_syncobj fd, or anything else that
makes the dma_fence visible to other kernel threads), up to and
including the dma_fence_wait(). Examples are irq handlers, the
scheduler rt threads, the tail of execbuf (after the corresponding
fences are visible), any workers that end up signalling dma_fences and
really anything else. Not annotated should be code paths that only
complete fences opportunistically as the gpu progresses, like e.g.
shrinker/eviction code.
The main class of deadlocks this is supposed to catch are:
Thread A:
mutex_lock(A);
mutex_unlock(A);
dma_fence_signal();
Thread B:
mutex_lock(A);
dma_fence_wait();
mutex_unlock(A);
Thread B is blocked on A signalling the fence, but A never gets around
to that because it cannot acquire the lock A.
Note that dma_fence_wait() is allowed to be nested within
dma_fence_begin/end_signalling sections. To allow this to happen the
read lock needs to be upgraded to a write lock, which means that any
other lock is acquired between the dma_fence_begin_signalling() call and
the call to dma_fence_wait(), and still held, this will result in an
immediate lockdep complaint. The only other option would be to not
annotate such calls, defeating the point. Therefore these annotations
cannot be sprinkled over the code entirely mindless to avoid false
positives.
Originally I hope that the cross-release lockdep extensions would
alleviate the need for explicit annotations:
https://lwn.net/Articles/709849/
But there's a few reasons why that's not an option:
- It's not happening in upstream, since it got reverted due to too
many false positives:
commit e966eaeeb623f09975ef362c2866fae6f86844f9
Author: Ingo Molnar <mingo@kernel.org>
Date: Tue Dec 12 12:31:16 2017 +0100
locking/lockdep: Remove the cross-release locking checks
This code (CONFIG_LOCKDEP_CROSSRELEASE=y and CONFIG_LOCKDEP_COMPLETIONS=y),
while it found a number of old bugs initially, was also causing too many
false positives that caused people to disable lockdep - which is arguably
a worse overall outcome.
- cross-release uses the complete() call to annotate the end of
critical sections, for dma_fence that would be dma_fence_signal().
But we do not want all dma_fence_signal() calls to be treated as
critical, since many are opportunistic cleanup of gpu requests. If
these get stuck there's still the main completion interrupt and
workers who can unblock everyone. Automatically annotating all
dma_fence_signal() calls would hence cause false positives.
- cross-release had some educated guesses for when a critical section
starts, like fresh syscall or fresh work callback. This would again
cause false positives without explicit annotations, since for
dma_fence the critical sections only starts when we publish a fence.
- Furthermore there can be cases where a thread never does a
dma_fence_signal, but is still critical for reaching completion of
fences. One example would be a scheduler kthread which picks up jobs
and pushes them into hardware, where the interrupt handler or
another completion thread calls dma_fence_signal(). But if the
scheduler thread hangs, then all the fences hang, hence we need to
manually annotate it. cross-release aimed to solve this by chaining
cross-release dependencies, but the dependency from scheduler thread
to the completion interrupt handler goes through hw where
cross-release code can't observe it.
In short, without manual annotations and careful review of the start
and end of critical sections, cross-relese dependency tracking doesn't
work. We need explicit annotations.
v2: handle soft/hardirq ctx better against write side and dont forget
EXPORT_SYMBOL, drivers can't use this otherwise.
v3: Kerneldoc.
v4: Some spelling fixes from Mika
v5: Amend commit message to explain in detail why cross-release isn't
the solution.
v6: Pull out misplaced .rst hunk.
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Cc: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Cc: Thomas Hellstrom <thomas.hellstrom@intel.com>
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: linux-rdma@vger.kernel.org
Cc: amd-gfx@lists.freedesktop.org
Cc: intel-gfx@lists.freedesktop.org
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200707201229.472834-2-daniel.vetter@ffwll.ch
2020-07-07 20:12:05 +00:00
|
|
|
#else
|
|
|
|
static inline bool dma_fence_begin_signalling(void)
|
|
|
|
{
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
static inline void dma_fence_end_signalling(bool cookie) {}
|
|
|
|
static inline void __dma_fence_might_wait(void) {}
|
|
|
|
#endif
|
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
int dma_fence_signal(struct dma_fence *fence);
|
|
|
|
int dma_fence_signal_locked(struct dma_fence *fence);
|
2021-01-16 00:31:46 +00:00
|
|
|
int dma_fence_signal_timestamp(struct dma_fence *fence, ktime_t timestamp);
|
|
|
|
int dma_fence_signal_timestamp_locked(struct dma_fence *fence,
|
|
|
|
ktime_t timestamp);
|
2016-10-25 12:00:45 +00:00
|
|
|
signed long dma_fence_default_wait(struct dma_fence *fence,
|
|
|
|
bool intr, signed long timeout);
|
|
|
|
int dma_fence_add_callback(struct dma_fence *fence,
|
|
|
|
struct dma_fence_cb *cb,
|
|
|
|
dma_fence_func_t func);
|
|
|
|
bool dma_fence_remove_callback(struct dma_fence *fence,
|
|
|
|
struct dma_fence_cb *cb);
|
|
|
|
void dma_fence_enable_sw_signaling(struct dma_fence *fence);
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
|
|
|
|
/**
|
2016-10-25 12:00:45 +00:00
|
|
|
* dma_fence_is_signaled_locked - Return an indication if the fence
|
|
|
|
* is signaled yet.
|
2018-04-27 06:17:08 +00:00
|
|
|
* @fence: the fence to check
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*
|
|
|
|
* Returns true if the fence was already signaled, false if not. Since this
|
|
|
|
* function doesn't enable signaling, it is not guaranteed to ever return
|
2018-04-27 06:17:08 +00:00
|
|
|
* true if dma_fence_add_callback(), dma_fence_wait() or
|
|
|
|
* dma_fence_enable_sw_signaling() haven't been called before.
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*
|
2018-04-27 06:17:08 +00:00
|
|
|
* This function requires &dma_fence.lock to be held.
|
|
|
|
*
|
|
|
|
* See also dma_fence_is_signaled().
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*/
|
|
|
|
static inline bool
|
2016-10-25 12:00:45 +00:00
|
|
|
dma_fence_is_signaled_locked(struct dma_fence *fence)
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
{
|
2016-10-25 12:00:45 +00:00
|
|
|
if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags))
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
return true;
|
|
|
|
|
|
|
|
if (fence->ops->signaled && fence->ops->signaled(fence)) {
|
2016-10-25 12:00:45 +00:00
|
|
|
dma_fence_signal_locked(fence);
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
2016-10-25 12:00:45 +00:00
|
|
|
* dma_fence_is_signaled - Return an indication if the fence is signaled yet.
|
2018-04-27 06:17:08 +00:00
|
|
|
* @fence: the fence to check
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*
|
|
|
|
* Returns true if the fence was already signaled, false if not. Since this
|
|
|
|
* function doesn't enable signaling, it is not guaranteed to ever return
|
2018-04-27 06:17:08 +00:00
|
|
|
* true if dma_fence_add_callback(), dma_fence_wait() or
|
|
|
|
* dma_fence_enable_sw_signaling() haven't been called before.
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*
|
2016-10-25 12:00:45 +00:00
|
|
|
* It's recommended for seqno fences to call dma_fence_signal when the
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
* operation is complete, it makes it possible to prevent issues from
|
|
|
|
* wraparound between time of issue and time of use by checking the return
|
|
|
|
* value of this function before calling hardware-specific wait instructions.
|
2018-04-27 06:17:08 +00:00
|
|
|
*
|
|
|
|
* See also dma_fence_is_signaled_locked().
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*/
|
|
|
|
static inline bool
|
2016-10-25 12:00:45 +00:00
|
|
|
dma_fence_is_signaled(struct dma_fence *fence)
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
{
|
2016-10-25 12:00:45 +00:00
|
|
|
if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags))
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
return true;
|
|
|
|
|
|
|
|
if (fence->ops->signaled && fence->ops->signaled(fence)) {
|
2016-10-25 12:00:45 +00:00
|
|
|
dma_fence_signal(fence);
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2017-06-29 12:59:24 +00:00
|
|
|
/**
|
|
|
|
* __dma_fence_is_later - return if f1 is chronologically later than f2
|
2018-04-27 06:17:08 +00:00
|
|
|
* @f1: the first fence's seqno
|
|
|
|
* @f2: the second fence's seqno from the same context
|
2019-04-15 12:46:34 +00:00
|
|
|
* @ops: dma_fence_ops associated with the seqno
|
2017-06-29 12:59:24 +00:00
|
|
|
*
|
|
|
|
* Returns true if f1 is chronologically later than f2. Both fences must be
|
|
|
|
* from the same context, since a seqno is not common across contexts.
|
|
|
|
*/
|
2019-04-15 12:46:34 +00:00
|
|
|
static inline bool __dma_fence_is_later(u64 f1, u64 f2,
|
|
|
|
const struct dma_fence_ops *ops)
|
2017-06-29 12:59:24 +00:00
|
|
|
{
|
2018-11-14 15:11:06 +00:00
|
|
|
/* This is for backward compatibility with drivers which can only handle
|
2019-04-15 12:46:34 +00:00
|
|
|
* 32bit sequence numbers. Use a 64bit compare when the driver says to
|
|
|
|
* do so.
|
2018-11-14 15:11:06 +00:00
|
|
|
*/
|
2019-04-15 12:46:34 +00:00
|
|
|
if (ops->use_64bit_seqno)
|
2018-11-14 15:11:06 +00:00
|
|
|
return f1 > f2;
|
|
|
|
|
|
|
|
return (int)(lower_32_bits(f1) - lower_32_bits(f2)) > 0;
|
2017-06-29 12:59:24 +00:00
|
|
|
}
|
|
|
|
|
2015-10-21 10:58:17 +00:00
|
|
|
/**
|
2016-10-25 12:00:45 +00:00
|
|
|
* dma_fence_is_later - return if f1 is chronologically later than f2
|
2018-04-27 06:17:08 +00:00
|
|
|
* @f1: the first fence from the same context
|
|
|
|
* @f2: the second fence from the same context
|
2015-10-21 10:58:17 +00:00
|
|
|
*
|
|
|
|
* Returns true if f1 is chronologically later than f2. Both fences must be
|
|
|
|
* from the same context, since a seqno is not re-used across contexts.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
static inline bool dma_fence_is_later(struct dma_fence *f1,
|
|
|
|
struct dma_fence *f2)
|
2015-10-21 10:58:17 +00:00
|
|
|
{
|
|
|
|
if (WARN_ON(f1->context != f2->context))
|
|
|
|
return false;
|
|
|
|
|
2019-04-15 12:46:34 +00:00
|
|
|
return __dma_fence_is_later(f1->seqno, f2->seqno, f1->ops);
|
2015-10-21 10:58:17 +00:00
|
|
|
}
|
|
|
|
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
/**
|
2016-10-25 12:00:45 +00:00
|
|
|
* dma_fence_later - return the chronologically later fence
|
2018-04-27 06:17:08 +00:00
|
|
|
* @f1: the first fence from the same context
|
|
|
|
* @f2: the second fence from the same context
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*
|
|
|
|
* Returns NULL if both fences are signaled, otherwise the fence that would be
|
|
|
|
* signaled last. Both fences must be from the same context, since a seqno is
|
|
|
|
* not re-used across contexts.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
static inline struct dma_fence *dma_fence_later(struct dma_fence *f1,
|
|
|
|
struct dma_fence *f2)
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
{
|
|
|
|
if (WARN_ON(f1->context != f2->context))
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
/*
|
2016-10-25 12:00:45 +00:00
|
|
|
* Can't check just DMA_FENCE_FLAG_SIGNALED_BIT here, it may never
|
|
|
|
* have been set if enable_signaling wasn't called, and enabling that
|
|
|
|
* here is overkill.
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
if (dma_fence_is_later(f1, f2))
|
|
|
|
return dma_fence_is_signaled(f1) ? NULL : f1;
|
2015-10-21 10:58:17 +00:00
|
|
|
else
|
2016-10-25 12:00:45 +00:00
|
|
|
return dma_fence_is_signaled(f2) ? NULL : f2;
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
}
|
|
|
|
|
2017-01-04 14:12:21 +00:00
|
|
|
/**
|
|
|
|
* dma_fence_get_status_locked - returns the status upon completion
|
2018-04-27 06:17:08 +00:00
|
|
|
* @fence: the dma_fence to query
|
2017-01-04 14:12:21 +00:00
|
|
|
*
|
|
|
|
* Drivers can supply an optional error status condition before they signal
|
|
|
|
* the fence (to indicate whether the fence was completed due to an error
|
|
|
|
* rather than success). The value of the status condition is only valid
|
|
|
|
* if the fence has been signaled, dma_fence_get_status_locked() first checks
|
|
|
|
* the signal state before reporting the error status.
|
|
|
|
*
|
|
|
|
* Returns 0 if the fence has not yet been signaled, 1 if the fence has
|
|
|
|
* been signaled without an error condition, or a negative error code
|
|
|
|
* if the fence has been completed in err.
|
|
|
|
*/
|
|
|
|
static inline int dma_fence_get_status_locked(struct dma_fence *fence)
|
|
|
|
{
|
|
|
|
if (dma_fence_is_signaled_locked(fence))
|
2017-01-04 14:12:22 +00:00
|
|
|
return fence->error ?: 1;
|
2017-01-04 14:12:21 +00:00
|
|
|
else
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
int dma_fence_get_status(struct dma_fence *fence);
|
|
|
|
|
2017-01-04 14:12:22 +00:00
|
|
|
/**
|
|
|
|
* dma_fence_set_error - flag an error condition on the fence
|
2018-04-27 06:17:08 +00:00
|
|
|
* @fence: the dma_fence
|
|
|
|
* @error: the error to store
|
2017-01-04 14:12:22 +00:00
|
|
|
*
|
|
|
|
* Drivers can supply an optional error status condition before they signal
|
|
|
|
* the fence, to indicate that the fence was completed due to an error
|
|
|
|
* rather than success. This must be set before signaling (so that the value
|
|
|
|
* is visible before any waiters on the signal callback are woken). This
|
|
|
|
* helper exists to help catching erroneous setting of #dma_fence.error.
|
|
|
|
*/
|
|
|
|
static inline void dma_fence_set_error(struct dma_fence *fence,
|
|
|
|
int error)
|
|
|
|
{
|
2017-07-20 12:51:07 +00:00
|
|
|
WARN_ON(test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags));
|
|
|
|
WARN_ON(error >= 0 || error < -MAX_ERRNO);
|
2017-01-04 14:12:22 +00:00
|
|
|
|
|
|
|
fence->error = error;
|
|
|
|
}
|
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
signed long dma_fence_wait_timeout(struct dma_fence *,
|
2015-10-20 14:34:16 +00:00
|
|
|
bool intr, signed long timeout);
|
2016-10-25 12:00:45 +00:00
|
|
|
signed long dma_fence_wait_any_timeout(struct dma_fence **fences,
|
|
|
|
uint32_t count,
|
2016-11-04 20:16:09 +00:00
|
|
|
bool intr, signed long timeout,
|
|
|
|
uint32_t *idx);
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
|
|
|
|
/**
|
2016-10-25 12:00:45 +00:00
|
|
|
* dma_fence_wait - sleep until the fence gets signaled
|
2018-04-27 06:17:08 +00:00
|
|
|
* @fence: the fence to wait on
|
|
|
|
* @intr: if true, do an interruptible wait
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*
|
|
|
|
* This function will return -ERESTARTSYS if interrupted by a signal,
|
|
|
|
* or 0 if the fence was signaled. Other error values may be
|
|
|
|
* returned on custom implementations.
|
|
|
|
*
|
|
|
|
* Performs a synchronous wait on this fence. It is assumed the caller
|
|
|
|
* directly or indirectly holds a reference to the fence, otherwise the
|
|
|
|
* fence might be freed before return, resulting in undefined behavior.
|
2018-04-27 06:17:08 +00:00
|
|
|
*
|
|
|
|
* See also dma_fence_wait_timeout() and dma_fence_wait_any_timeout().
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
static inline signed long dma_fence_wait(struct dma_fence *fence, bool intr)
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
{
|
|
|
|
signed long ret;
|
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
/* Since dma_fence_wait_timeout cannot timeout with
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
* MAX_SCHEDULE_TIMEOUT, only valid return values are
|
|
|
|
* -ERESTARTSYS and MAX_SCHEDULE_TIMEOUT.
|
|
|
|
*/
|
2016-10-25 12:00:45 +00:00
|
|
|
ret = dma_fence_wait_timeout(fence, intr, MAX_SCHEDULE_TIMEOUT);
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
|
|
|
|
return ret < 0 ? ret : 0;
|
|
|
|
}
|
|
|
|
|
2018-12-03 12:36:14 +00:00
|
|
|
struct dma_fence *dma_fence_get_stub(void);
|
2021-04-08 09:54:28 +00:00
|
|
|
struct dma_fence *dma_fence_allocate_private_stub(void);
|
2016-10-25 12:00:45 +00:00
|
|
|
u64 dma_fence_context_alloc(unsigned num);
|
fence: dma-buf cross-device synchronization (v18)
A fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A driver must allocate a fence context for each execution ring that can
run in parallel. The function for this takes an argument with how many
contexts to allocate:
+ fence_context_alloc()
A fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence:
+ fence_signal()
To have a rough approximation whether a fence is fired, call:
+ fence_is_signaled()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
The one pending on the fence can add an async callback:
+ fence_add_callback()
The callback can optionally be cancelled with:
+ fence_remove_callback()
To wait synchronously, optionally with a timeout:
+ fence_wait()
+ fence_wait_timeout()
When emitting a fence, call:
+ trace_fence_emit()
To annotate that a fence is blocking on another fence, call:
+ trace_fence_annotate_wait_on(fence, on_fence)
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = custom_get_fence(...);
if ((seqno_fence = to_seqno_fence(fence)) != NULL) {
dma_buf *fence_buf = seqno_fence->sync_buf;
get_dma_buf(fence_buf);
... tell the hw the memory location to wait ...
custom_wait_on(fence_buf, seqno_fence->seqno_ofs, fence->seqno);
} else {
/* fall-back to sw sync * /
fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
enable_signaling callback is used to provide sw signaling in case a cpu
waiter is requested or no compatible hardware signaling could be used.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
v8: [ Maarten Lankhorst ] Tons of tiny fixes, moved __dma_fence_init to
header and fixed include mess. dma-fence.h now includes dma-buf.h
All members are now initialized, so kmalloc can be used for
allocating a dma-fence. More documentation added.
v9: Change compiler bitfields to flags, change return type of
enable_signaling to bool. Rework dma_fence_wait. Added
dma_fence_is_signaled and dma_fence_wait_timeout.
s/dma// and change exports to non GPL. Added fence_is_signaled and
fence_enable_sw_signaling calls, add ability to override default
wait operation.
v10: remove event_queue, use a custom list, export try_to_wake_up from
scheduler. Remove fence lock and use a global spinlock instead,
this should hopefully remove all the locking headaches I was having
on trying to implement this. enable_signaling is called with this
lock held.
v11:
Use atomic ops for flags, lifting the need for some spin_lock_irqsaves.
However I kept the guarantee that after fence_signal returns, it is
guaranteed that enable_signaling has either been called to completion,
or will not be called any more.
Add contexts and seqno to base fence implementation. This allows you
to wait for less fences, by testing for seqno + signaled, and then only
wait on the later fence.
Add FENCE_TRACE, FENCE_WARN, and FENCE_ERR. This makes debugging easier.
An CONFIG_DEBUG_FENCE will be added to turn off the FENCE_TRACE
spam, and another runtime option can turn it off at runtime.
v12:
Add CONFIG_FENCE_TRACE. Add missing documentation for the fence->context
and fence->seqno members.
v13:
Fixup CONFIG_FENCE_TRACE kconfig description.
Move fence_context_alloc to fence.
Simplify fence_later.
Kill priv member to fence_cb.
v14:
Remove priv argument from fence_add_callback, oops!
v15:
Remove priv from documentation.
Explicitly include linux/atomic.h.
v16:
Add trace events.
Import changes required by android syncpoints.
v17:
Use wake_up_state instead of try_to_wake_up. (Colin Cross)
Fix up commit description for seqno_fence. (Rob Clark)
v18:
Rename release_fence to fence_release.
Move to drivers/dma-buf/.
Rename __fence_is_signaled and __fence_signal to *_locked.
Rename __fence_init to fence_init.
Make fence_default_wait return a signed long, and fix wait ops too.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Thierry Reding <thierry.reding@gmail.com> #use smp_mb__before_atomic()
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-07-01 10:57:14 +00:00
|
|
|
|
2016-10-25 12:00:45 +00:00
|
|
|
#endif /* __LINUX_DMA_FENCE_H */
|