The struct media_entity_graph was allocated in the stack, limiting the
number of entities that could be reasonably allocated. Instead, move the
struct to struct media_pipeline which is typically allocated using
kmalloc() instead.
The intent is to keep the enumeration around for later use for the
duration of the streaming. As streaming is eventually stopped, an
unfortunate memory allocation failure would prevent stopping the
streaming. As no memory will need to be allocated, the problem is avoided
altogether.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
This is useful in e.g. knowing whether certain operations have already
been performed for an entity. The users include the framework itself (for
graph walking) and a number of drivers.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
As pointed by Dan, the commit f8fd4c61b5ae ("[media] media-entity:
protect object creation/removal using spin lock")' leads to the
following static checker warning:
drivers/media/media-entity.c:781 media_remove_intf_link()
error: dereferencing freed memory 'link'
drivers/media/media-entity.c
777 void media_remove_intf_link(struct media_link *link)
778 {
779 spin_lock(&link->graph_obj.mdev->lock);
780 __media_remove_intf_link(link);
781 spin_unlock(&link->graph_obj.mdev->lock);
In practice, I didn't see any troubles even with KASAN enabled. I guess
gcc optimizer internally cached the mdev reference, instead of getting
it twice. Yet, it is a very bad idea to rely on such optimization. So,
let's fix the code.
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Changeset f8fd4c61b5ae ("[media] media-entity: protect object
creation/removal using spin lock") changed the object creation/removal
protection to spin lock, as this is what's used on media-device,
keeping the mutex reserved for graph traversal routines. However, it
also changed the link setup, by mistake.
This could cause troubles, as the link setup can affect the graph
traversal, and this is likely the reason for a mutex there.
So, revert media_entity_setup_link() to use mutex.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
If a different entity->pipe in a pipeline was encountered, a warning was
issued but the execution continued as if nothing had happened. Return an
error instead right there.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Instead of flagging an interface link as MEDIA_LNK_FL_INTERFACE_LINK
only when returning to userspace, do it at link creation time.
That would allow using such flag internally, and cleans up a
little bit the code for G_TOPOLOGY ioctl.
[mchehab@osg.samsung.com: folded with a fixup from Dan Carpenter,
replacing & by &&]
Suggested-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Commit 86ee417578a2 ("[media] media: convert links from array to list")
had many changes that were automated using coccinelle but the semantic
patch was not smart enough to rely on operators precedence and avoid
using unnecessary enclosing parenthesis.
This patch removes them since are not needed.
Suggested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Several additional functions are described at media-entity.c.
Moving them to the header file, to make the code cleaner and
to have all such macros at the same place.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Those media_obj_* functions are actually creating/destroying
media graph objects. So, rename them to better represent
what they're actually doing.
No functional changes.
This was created via this small shell script:
for i in $(git grep -l media_gobj_init); do sed s,media_gobj_init,media_gobj_create,g <$i >a && mv a $i; done
for i in $(git grep -l media_gobj_remove); do sed s,media_gobj_remove,media_gobj_destroy,g <$i >a && mv a $i; done
Suggested-by: Sakari Ailus <sakari.ailus@iki.fi>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
> > > + if (rlink != link->reverse) {
> > > + r++;
> >
> > The variable is incremented here but otherwise never used, you can remove it.
Suggested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Move this function to happen earlier, in order to avoid
a uneeded forward declaration.
Suggested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
This function was used in the past to free the links
that were allocated by the media controller core.
However, this is not needed anymore. We should likely
get rid of the funcion on some function, but, for now,
let's just convert into an inlined function and let the
compiler to get rid of it.
Suggested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
With the MC next gen rework, what's left for media_entity_init()
is to just initialize the PADs. However, certain devices, like
a FLASH led/light doesn't have any input or output PAD.
So, there's no reason why calling media_entity_init() would be
mandatory. Also, despite its name, what this function actually
does is to initialize the PADs data. So, rename it to
media_entity_pads_init() in order to reflect that.
The media entity actual init happens during entity register,
at media_device_register_entity(). We should move init of
num_links and num_backlinks to it.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
If an entity is registered with a media device before is initialized
with media_device_register_entity(), the number of pads won't be set
so media_device_register_entity() won't create pad objects and add
it to the media device pads list.
Do this at entity initialization time if the entity was registered
before so the graph is complete and correct regardless of the order
in which the entities are initialized and registered.
Suggested-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
We should not be creating device nodes at IRQ contexts. So,
the only flags we'll be using will be GFP_KERNEL. Let's
remove the gfp_flags, in order to make the interface simpler.
If we ever need it, it would be easy to revert those changes.
While here, remove an extra blank line.
Suggested-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Due to the graph traversal algorithm currently in usage, we
need a copy of all data links. Those backlinks should not be
send to userspace, as otherwise, all links there will be
duplicated.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The logic is testing if num_links==0 at the wrong place. Due to
that, a backlink may be kept without removal, causing KASAN
to complain about usage after free during either entity or
link removal.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The media_entity_cleanup() function only cleans up the entity links list
but this operation is already made in media_device_unregister_entity().
In most cases this should be harmless (besides having duplicated code)
since the links list would be empty so the iteration would not happen
but the links list is initialized in media_device_register_entity() so
if a driver fails to register an entity with a media device and clean up
the entity in the error path, a NULL deference pointer error will happen.
So don't try to empty the links list in media_entity_cleanup() since
is either done already or haven't been initialized yet.
Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com>
Some parts of the media controller are using mutexes while
others are using spin locks in order to protect creation
and removal of elements in the graph. That's wrong!
Also, the V4L2 core can remove graph elements on non-interactive
context:
BUG: sleeping function called from invalid context at include/linux/sched.h:2776
Fix it by always using spin locks for graph element addition/removal,
just like entity creation/removal is protected at media-device.c
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Just like what's done with entities, when the media controller is
unregistered, release any interface and interface links that
might still be there.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Add functions to explicitly unregister all entity links.
This function is called automatically when an entity
link is destroyed.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Every time a graph object is added or removed, the version
of the topology changes. That's a requirement for the new
MEDIA_IOC_G_TOPOLOGY, in order to allow userspace to know
that the topology has changed after a previous call to it.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The MC next gen API sends objects to userspace grouped by
their types.
In the case of pads and links, in order to improve performance
and have a simpler code, the best is to store them also on
separate linked lists at MC.
If we don't do that, we would need this kind of interaction
to send data to userspace (code is in structured english):
for each entity:
for each pad:
store pads
for each entity:
for each link:
store link
for each interface:
for each link:
store link
With would require one nested loop for pads and two nested
loops for links. By using separate linked lists for them,
just one loop would be enough.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Let's control the topology changes inside the graph_object. So, move the
addition and removal of interfaces/entities from the mdev lists to
media_gobj_init() and media_gobj_remove().
The main reason is that mdev should have lists for all object types, as
the new MC api will require to store objects in separate places.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Sometimes, it is important to see if the created pad is
sink or source. Add info to track that.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The media device should list the interface objects, so add a linked list
for those interfaces in struct media_device.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
As we'll be adding other interface types in the future, put the
common interface create code on a separate function.
Suggested-by: Hans Verkuil <hans.verkuil@cisco.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Now that we have a new graph object called "interfaces", we
need to be able to link them to the entities.
Add a linked list to the interfaces to allow them to be
linked to the entities.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Remove entity name from the link as this exists only if the object
type is PAD on both link ends.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The media_entity_add_link() function takes an entity
as an argument just to get the list head.
Make it more generic by changing the function argument
to list_head.
No functional changes.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The entire logic that represent graph links were developed on a
time where there were no needs to dynamic remove links. So,
although links are created/removed one by one via some
functions, they're stored as an array inside the entity struct.
As the array may grow, there's a logic inside the code that
checks if the amount of space is not enough to store
the needed links. If it isn't the core uses krealloc()
to change the size of the link, with is bad, as it
leaves the memory fragmented.
So, convert links into a list.
Also, currently, both source and sink entities need the link
at the graph traversal logic inside media_entity. So there's
a logic duplicating all links. That makes it to spend
twice the memory needed. This is not a big deal for today's
usage, where the number of links are not big.
Yet, if during the MC workshop discussions, it was said that
IIO graphs could have up to 4,000 entities. So, we may
want to remove the duplication on some future. The problem
is that it would require a separate linked list to store
the backlinks inside the entity, or to use a more complex
algorithm to do graph backlink traversal, with is something
that the current graph traversal inside the core can't cope
with. So, let's postpone a such change if/when it is actually
needed.
It should also be noticed that the media_link structure uses
44 bytes on 32-bit architectures and 84 bytes on 64-bit
architecture. It will thus be allocated out of the 64-bytes and
96-bytes pools respectively. That's a 12.5% memory waste on
64-bit architectures and 31.25% on 32-bit architecture.
A linked list is less efficient than an array in this case, but
this could later be optimized if we can get rid of the reverse
links (with would reduce memory allocation by 50%).
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Links are graph objects that represent the links of two already
existing objects in the graph.
While with the current implementation, it is possible to create
the links earlier, It doesn't make any sense to allow linking
two objects when they are not both created.
So, remove the code that would be handling those early-created
links and add a BUG_ON() to ensure that.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Interfaces are different than entities: they represent a
Kernel<->userspace interaction, while entities represent a
piece of hardware/firmware/software that executes a function.
Let's distinguish them by creating a separate structure to
store the interfaces.
Later patches should change the existing drivers and logic
to split the current interface embedded inside the entity
structure (device nodes) into a separate object of the graph.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The struct media_entity has a .parent field that stores a pointer
to the parent struct media_device. But recently a media_gobj was
embedded into the entities and since struct media_gojb already has
a pointer to a struct media_device in the .mdev field, the .parent
field becomes redundant and can be removed.
This patch replaces all the usage of .parent by .graph_obj.mdev so
that field will become unused and can be removed on a later patch.
No functional changes.
The transformation was made using the following coccinelle spatch:
@@
struct media_entity *me;
@@
- me->parent
+ me->graph_obj.mdev
@@
struct media_entity *link;
@@
- link->source->entity->parent
+ link->source->entity->graph_obj.mdev
@@
struct exynos_video_entity *ve;
@@
- ve->vdev.entity.parent
+ ve->vdev.entity.graph_obj.mdev
Suggested-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
With the new API, a link can be either between two PADs or between an interface
and an entity. So, we need to use a better name for the function that create
links between two pads.
So, rename the such function to media_create_pad_link().
No functional changes.
This patch was created via this shell script:
for i in $(find drivers/media -name '*.[ch]' -type f) $(find drivers/staging/media -name '*.[ch]' -type f) $(find include/ -name '*.h' -type f) ; do sed s,media_entity_create_link,media_create_pad_link,g <$i >a && mv a $i; done
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
It helps to check if the media controller is doing the
right thing with the object creation and removal.
No extra code/data will be produced if DEBUG or
CONFIG_DYNAMIC_DEBUG is not enabled.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Just like entities and pads, links also need to have unique
Object IDs along a given media controller.
So, let's add a media_gobj inside it and initialize
the object then a new link is created.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
PADs also need unique object IDs that won't conflict with
the entity object IDs.
The pad objects are currently created via media_entity_init()
and, once created, never change.
While this will likely change in the future in order to
support dynamic changes, for now we'll keep PADs as arrays
and initialize the media_gobj embedded structs when
registering the entity.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
As entities are graph objects, let's embed media_gobj
on it. That ensures an unique ID for entities that can be
global along the entire media controller.
For now, we'll keep the already existing entity ID. Such
field need to be dropped at some point, but for now, let's
not do this, to avoid needing to review all drivers and
the userspace apps.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Due to the MC API proposed changes, we'll need to have an unique
object ID for all graph objects, and have some shared fields
that will be common on all media graph objects.
Right now, the only common object is the object ID, but other
fields will be added later on.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Instead of accessing directly entity.id, let's create a macro,
as this field will be moved into a common struct later on.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Currently, media_entity_init() creates an array with the links,
allocated at init time. It provides a parameter (extra_links)
that would allocate more links than the current needs, but this
is not used by any driver.
As we want to be able to do dynamic link allocation/removal,
we'll need to change the implementation of the links. So,
before doing that, let's first remove that extra unused
parameter, in order to cleanup the interface first.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Fix those sparse warnings:
drivers/media/media-entity.c:238:17: warning: Variable length array is used.
drivers/media/media-entity.c:239:17: warning: Variable length array is used.
That allows sparse and other code check tools to verify if the
function is using more stack than allowed.
It also solves a bad Kernel pratice of using var length arrays
at the stack.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
On the place of the source entity name, the sink entity name was printed.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Patch "[media] media: Print information on failed link validation" cause a
harmless compiler warning in printing a debug message. Fix this.
The type casting is done do ensure the type really is suitable for printing
as %u, as find_first_zero_bit() does return int on some architectures and
unsigned long on others.
Signed-off-by: Sakari Ailus <sakari.ailus@iki.fi>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The Media controller doesn't tell much to the user in cases such as pipeline
startup failure. The link validation is the most common media graph (or in
V4L2's case, format) related reason for the failure. In more complex
pipelines the reason may not always be obvious to the user, so point them to
look at the right direction.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Do not allow streaming if a pad with MEDIA_PAD_FL_MUST_CONNECT flag is not
connected by an active link.
This patch makes it possible to avoid drivers having to check for the most
common case of link state validation: a sink pad that must be connected.
Signed-off-by: Sakari Ailus <sakari.ailus@iki.fi>
Tested-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
The graph traversal API (media_entity_graph_walk_*) doesn't support
cyclic graphs and will fail to correctly walk a graph when circular
links exist. Support circular graph traversal by checking whether an
entity has already been visited before pushing it to the stack.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <m.chehab@samsung.com>
This function allows to remove all media entity's links to other
entities, leaving no references to a media entity's links array
at its remote entities.
Currently, when a driver of some entity is removed it will free its
media entities links[] array, leaving dangling pointers at other
entities that are part of same media graph. This is troublesome when
drivers of a media device entities are in separate kernel modules,
removing only some modules will leave others in an incorrect state.
This function is intended to be used when an entity is being
unregistered from a media device.
With an assumption that normally the media links should be created
between media entities registered to a media device, with the graph
mutex held.
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Reviewed-by: Andrzej Hajda <a.hajda@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@iki.fi>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
