media: atomisp: prevent integer overflow in sh_css_set_black_frame()

[ Upstream commit 3ad290194b ]

The "height" and "width" values come from the user so the "height * width"
multiplication can overflow.

Link: https://lore.kernel.org/r/YxBBCRnm3mmvaiuR@kili

Fixes: a49d25364d ("staging/atomisp: Add support for the Intel IPU v2")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@intel.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
This commit is contained in:
Dan Carpenter 2022-09-01 07:20:09 +02:00 committed by Greg Kroah-Hartman
parent 019076e377
commit a549517e4b

View file

@ -950,8 +950,8 @@ sh_css_set_black_frame(struct ia_css_stream *stream,
params->fpn_config.data = NULL;
}
if (!params->fpn_config.data) {
params->fpn_config.data = kvmalloc(height * width *
sizeof(short), GFP_KERNEL);
params->fpn_config.data = kvmalloc(array3_size(height, width, sizeof(short)),
GFP_KERNEL);
if (!params->fpn_config.data) {
IA_CSS_ERROR("out of memory");
IA_CSS_LEAVE_ERR_PRIVATE(-ENOMEM);