staging: bcm2835-camera: Fix stride on RGB3/BGR3 formats

RGB3/BGR3 end up being 3 bytes per pixel, which meant that
the alignment code ended up trying to align using bitmasking
with a mask of 96.
That doesn't work, so switch to an arithmetic alignment for
those formats.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Stefan Wahren <wahrenst@gmx.net>
Acked-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Acked-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This commit is contained in:
Dave Stevenson 2019-06-29 21:31:40 +02:00 committed by Greg Kroah-Hartman
parent 640e77466e
commit eb3539767a

View file

@ -964,13 +964,27 @@ static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
1, 0);
f->fmt.pix.bytesperline = f->fmt.pix.width * mfmt->ybbp;
if (!mfmt->remove_padding) {
int align_mask = ((32 * mfmt->depth) >> 3) - 1;
/* GPU isn't removing padding, so stride is aligned to 32 */
f->fmt.pix.bytesperline =
(f->fmt.pix.bytesperline + align_mask) & ~align_mask;
if (mfmt->depth == 24) {
/*
* 24bpp is a pain as we can't use simple masking.
* Min stride is width aligned to 16, times 24bpp.
*/
f->fmt.pix.bytesperline =
((f->fmt.pix.width + 15) & ~15) * 3;
} else {
/*
* GPU isn't removing padding, so stride is aligned to
* 32
*/
int align_mask = ((32 * mfmt->depth) >> 3) - 1;
f->fmt.pix.bytesperline =
(f->fmt.pix.bytesperline + align_mask) &
~align_mask;
}
v4l2_dbg(1, bcm2835_v4l2_debug, &dev->v4l2_dev,
"Not removing padding, so bytes/line = %d, (align_mask %d)\n",
f->fmt.pix.bytesperline, align_mask);
"Not removing padding, so bytes/line = %d\n",
f->fmt.pix.bytesperline);
}
/* Image buffer has to be padded to allow for alignment, even though