mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
synced 2024-09-12 13:55:32 +00:00
![Sakari Ailus](/assets/img/avatar_default.png)
The media_entity_pipeline_start() and media_entity_pipeline_stop() functions are renamed as media_pipeline_start() and media_pipeline_stop(), respectively. The reason is two-fold: the pipeline struct is, rightly, already called media_pipeline (rather than media_entity_pipeline) and what this really is about is a pipeline. A pipeline consists of entities --- and, well, other objects embedded in these entities. As the pipeline object will be in the future moved from entities to pads in order to support multiple pipelines through a single entity, do the renaming now. Similarly, functions operating on struct media_entity_graph as well as the struct itself are renamed by dropping the "entity_" part from the prefix of the function family and the data structure. The graph traversal which is what the functions are about is not specifically about entities only and will operate on pads for the same reason as the media pipeline. The patch has been generated using the following command: git grep -l media_entity |xargs perl -i -pe ' s/media_entity_pipeline/media_pipeline/g; s/media_entity_graph/media_graph/g' And a few manual edits related to line start alignment and line wrapping. Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com> Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
264 lines
9.4 KiB
ReStructuredText
264 lines
9.4 KiB
ReStructuredText
Media Controller devices
|
|
------------------------
|
|
|
|
Media Controller
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
The media controller userspace API is documented in
|
|
:ref:`the Media Controller uAPI book <media_controller>`. This document focus
|
|
on the kernel-side implementation of the media framework.
|
|
|
|
Abstract media device model
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Discovering a device internal topology, and configuring it at runtime, is one
|
|
of the goals of the media framework. To achieve this, hardware devices are
|
|
modelled as an oriented graph of building blocks called entities connected
|
|
through pads.
|
|
|
|
An entity is a basic media hardware building block. It can correspond to
|
|
a large variety of logical blocks such as physical hardware devices
|
|
(CMOS sensor for instance), logical hardware devices (a building block
|
|
in a System-on-Chip image processing pipeline), DMA channels or physical
|
|
connectors.
|
|
|
|
A pad is a connection endpoint through which an entity can interact with
|
|
other entities. Data (not restricted to video) produced by an entity
|
|
flows from the entity's output to one or more entity inputs. Pads should
|
|
not be confused with physical pins at chip boundaries.
|
|
|
|
A link is a point-to-point oriented connection between two pads, either
|
|
on the same entity or on different entities. Data flows from a source
|
|
pad to a sink pad.
|
|
|
|
Media device
|
|
^^^^^^^^^^^^
|
|
|
|
A media device is represented by a struct :c:type:`media_device`
|
|
instance, defined in ``include/media/media-device.h``.
|
|
Allocation of the structure is handled by the media device driver, usually by
|
|
embedding the :c:type:`media_device` instance in a larger driver-specific
|
|
structure.
|
|
|
|
Drivers register media device instances by calling
|
|
:c:func:`__media_device_register()` via the macro ``media_device_register()``
|
|
and unregistered by calling :c:func:`media_device_unregister()`.
|
|
|
|
Entities
|
|
^^^^^^^^
|
|
|
|
Entities are represented by a struct :c:type:`media_entity`
|
|
instance, defined in ``include/media/media-entity.h``. The structure is usually
|
|
embedded into a higher-level structure, such as
|
|
:c:type:`v4l2_subdev` or :c:type:`video_device`
|
|
instances, although drivers can allocate entities directly.
|
|
|
|
Drivers initialize entity pads by calling
|
|
:c:func:`media_entity_pads_init()`.
|
|
|
|
Drivers register entities with a media device by calling
|
|
:c:func:`media_device_register_entity()`
|
|
and unregistred by calling
|
|
:c:func:`media_device_unregister_entity()`.
|
|
|
|
Interfaces
|
|
^^^^^^^^^^
|
|
|
|
Interfaces are represented by a
|
|
struct :c:type:`media_interface` instance, defined in
|
|
``include/media/media-entity.h``. Currently, only one type of interface is
|
|
defined: a device node. Such interfaces are represented by a
|
|
struct :c:type:`media_intf_devnode`.
|
|
|
|
Drivers initialize and create device node interfaces by calling
|
|
:c:func:`media_devnode_create()`
|
|
and remove them by calling:
|
|
:c:func:`media_devnode_remove()`.
|
|
|
|
Pads
|
|
^^^^
|
|
Pads are represented by a struct :c:type:`media_pad` instance,
|
|
defined in ``include/media/media-entity.h``. Each entity stores its pads in
|
|
a pads array managed by the entity driver. Drivers usually embed the array in
|
|
a driver-specific structure.
|
|
|
|
Pads are identified by their entity and their 0-based index in the pads
|
|
array.
|
|
|
|
Both information are stored in the struct :c:type:`media_pad`,
|
|
making the struct :c:type:`media_pad` pointer the canonical way
|
|
to store and pass link references.
|
|
|
|
Pads have flags that describe the pad capabilities and state.
|
|
|
|
``MEDIA_PAD_FL_SINK`` indicates that the pad supports sinking data.
|
|
``MEDIA_PAD_FL_SOURCE`` indicates that the pad supports sourcing data.
|
|
|
|
.. note::
|
|
|
|
One and only one of ``MEDIA_PAD_FL_SINK`` or ``MEDIA_PAD_FL_SOURCE`` must
|
|
be set for each pad.
|
|
|
|
Links
|
|
^^^^^
|
|
|
|
Links are represented by a struct :c:type:`media_link` instance,
|
|
defined in ``include/media/media-entity.h``. There are two types of links:
|
|
|
|
**1. pad to pad links**:
|
|
|
|
Associate two entities via their PADs. Each entity has a list that points
|
|
to all links originating at or targeting any of its pads.
|
|
A given link is thus stored twice, once in the source entity and once in
|
|
the target entity.
|
|
|
|
Drivers create pad to pad links by calling:
|
|
:c:func:`media_create_pad_link()` and remove with
|
|
:c:func:`media_entity_remove_links()`.
|
|
|
|
**2. interface to entity links**:
|
|
|
|
Associate one interface to a Link.
|
|
|
|
Drivers create interface to entity links by calling:
|
|
:c:func:`media_create_intf_link()` and remove with
|
|
:c:func:`media_remove_intf_links()`.
|
|
|
|
.. note::
|
|
|
|
Links can only be created after having both ends already created.
|
|
|
|
Links have flags that describe the link capabilities and state. The
|
|
valid values are described at :c:func:`media_create_pad_link()` and
|
|
:c:func:`media_create_intf_link()`.
|
|
|
|
Graph traversal
|
|
^^^^^^^^^^^^^^^
|
|
|
|
The media framework provides APIs to iterate over entities in a graph.
|
|
|
|
To iterate over all entities belonging to a media device, drivers can use
|
|
the media_device_for_each_entity macro, defined in
|
|
``include/media/media-device.h``.
|
|
|
|
.. code-block:: c
|
|
|
|
struct media_entity *entity;
|
|
|
|
media_device_for_each_entity(entity, mdev) {
|
|
// entity will point to each entity in turn
|
|
...
|
|
}
|
|
|
|
Drivers might also need to iterate over all entities in a graph that can be
|
|
reached only through enabled links starting at a given entity. The media
|
|
framework provides a depth-first graph traversal API for that purpose.
|
|
|
|
.. note::
|
|
|
|
Graphs with cycles (whether directed or undirected) are **NOT**
|
|
supported by the graph traversal API. To prevent infinite loops, the graph
|
|
traversal code limits the maximum depth to ``MEDIA_ENTITY_ENUM_MAX_DEPTH``,
|
|
currently defined as 16.
|
|
|
|
Drivers initiate a graph traversal by calling
|
|
:c:func:`media_graph_walk_start()`
|
|
|
|
The graph structure, provided by the caller, is initialized to start graph
|
|
traversal at the given entity.
|
|
|
|
Drivers can then retrieve the next entity by calling
|
|
:c:func:`media_graph_walk_next()`
|
|
|
|
When the graph traversal is complete the function will return ``NULL``.
|
|
|
|
Graph traversal can be interrupted at any moment. No cleanup function call
|
|
is required and the graph structure can be freed normally.
|
|
|
|
Helper functions can be used to find a link between two given pads, or a pad
|
|
connected to another pad through an enabled link
|
|
:c:func:`media_entity_find_link()` and
|
|
:c:func:`media_entity_remote_pad()`.
|
|
|
|
Use count and power handling
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Due to the wide differences between drivers regarding power management
|
|
needs, the media controller does not implement power management. However,
|
|
the struct :c:type:`media_entity` includes a ``use_count``
|
|
field that media drivers
|
|
can use to track the number of users of every entity for power management
|
|
needs.
|
|
|
|
The :c:type:`media_entity<media_entity>`.\ ``use_count`` field is owned by
|
|
media drivers and must not be
|
|
touched by entity drivers. Access to the field must be protected by the
|
|
:c:type:`media_device`.\ ``graph_mutex`` lock.
|
|
|
|
Links setup
|
|
^^^^^^^^^^^
|
|
|
|
Link properties can be modified at runtime by calling
|
|
:c:func:`media_entity_setup_link()`.
|
|
|
|
Pipelines and media streams
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
When starting streaming, drivers must notify all entities in the pipeline to
|
|
prevent link states from being modified during streaming by calling
|
|
:c:func:`media_pipeline_start()`.
|
|
|
|
The function will mark all entities connected to the given entity through
|
|
enabled links, either directly or indirectly, as streaming.
|
|
|
|
The struct :c:type:`media_pipeline` instance pointed to by
|
|
the pipe argument will be stored in every entity in the pipeline.
|
|
Drivers should embed the struct :c:type:`media_pipeline`
|
|
in higher-level pipeline structures and can then access the
|
|
pipeline through the struct :c:type:`media_entity`
|
|
pipe field.
|
|
|
|
Calls to :c:func:`media_pipeline_start()` can be nested.
|
|
The pipeline pointer must be identical for all nested calls to the function.
|
|
|
|
:c:func:`media_pipeline_start()` may return an error. In that case,
|
|
it will clean up any of the changes it did by itself.
|
|
|
|
When stopping the stream, drivers must notify the entities with
|
|
:c:func:`media_pipeline_stop()`.
|
|
|
|
If multiple calls to :c:func:`media_pipeline_start()` have been
|
|
made the same number of :c:func:`media_pipeline_stop()` calls
|
|
are required to stop streaming.
|
|
The :c:type:`media_entity`.\ ``pipe`` field is reset to ``NULL`` on the last
|
|
nested stop call.
|
|
|
|
Link configuration will fail with ``-EBUSY`` by default if either end of the
|
|
link is a streaming entity. Links that can be modified while streaming must
|
|
be marked with the ``MEDIA_LNK_FL_DYNAMIC`` flag.
|
|
|
|
If other operations need to be disallowed on streaming entities (such as
|
|
changing entities configuration parameters) drivers can explicitly check the
|
|
media_entity stream_count field to find out if an entity is streaming. This
|
|
operation must be done with the media_device graph_mutex held.
|
|
|
|
Link validation
|
|
^^^^^^^^^^^^^^^
|
|
|
|
Link validation is performed by :c:func:`media_pipeline_start()`
|
|
for any entity which has sink pads in the pipeline. The
|
|
:c:type:`media_entity`.\ ``link_validate()`` callback is used for that
|
|
purpose. In ``link_validate()`` callback, entity driver should check
|
|
that the properties of the source pad of the connected entity and its own
|
|
sink pad match. It is up to the type of the entity (and in the end, the
|
|
properties of the hardware) what matching actually means.
|
|
|
|
Subsystems should facilitate link validation by providing subsystem specific
|
|
helper functions to provide easy access for commonly needed information, and
|
|
in the end provide a way to use driver-specific callbacks.
|
|
|
|
.. kernel-doc:: include/media/media-device.h
|
|
|
|
.. kernel-doc:: include/media/media-devnode.h
|
|
|
|
.. kernel-doc:: include/media/media-entity.h
|