5.10.83 will not reboot on armv6/7 #4774
Comments
|
A fresh build from that commit reboots for me on a 3B+:
|
|
This is on a RPi 3B+ off an SD card. The only peripherals attached are a tft screen. There is a usb fob in one of the ports. It was connecting by wifi. I have seen this with or without the dtoverlay enabled for the tft screen. I am running 'gpu_mem=16' to boot it headless w/o the vc. I see that you are using start.elf and I think I should be using start_cd.elf... |
|
Hmmm - I notice that the ISP driver is crashing on startup with (and only with) the cutdown firmware: Can you confirm that same crash on startup while I revert to 5.10.81 to see if it goes away? |
|
The ISP isn't present at all with the cutdown firmware. |
|
The problem is that the drvdata is unset until |
|
I'll fix it. |
|
Setting (EDIT) Naush^H^H^H^H^HPhil beat me to it! |
|
:( |
pelwell
added a commit
that referenced
this issue
Dec 16, 2021
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
|
Fixed by 3ec9b52. |
pelwell
added a commit
that referenced
this issue
Dec 16, 2021
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
|
wow! that was a quick fix. I'll re-build and see how it goes. |
|
I did a git pull, verified that the 3ec9b52 commit was applied (from git log) then I rebuilt the kernel and installed it. It advertises as 5.10.83-v7+. It still will not reboot when 'gpu_mem=16'. The console shows that it reached target "Reboot" then a lot of systemd-shutdown lines then the final line |
|
You are not running the correct build - a kernel built for Pi 3 from 3ec9b52 reports itself as |
pelwell
added a commit
that referenced
this issue
Dec 17, 2021
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Dec 17, 2021
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
popcornmix
added a commit
to raspberrypi/firmware
that referenced
this issue
Dec 17, 2021
kernel: drm/vc4: Skip writes to disabled packet RAM See: raspberrypi/linux#4770 kernel: BCM2711 KMS YUV Support See: raspberrypi/linux#4777 kernel: Revert media: bcm2835-codec: Limit video callbacks See: raspberrypi/linux#4773 kernel: staging/bcm2835-isp: Fix cleanup after init fail See: raspberrypi/linux#4774
popcornmix
added a commit
to raspberrypi/rpi-firmware
that referenced
this issue
Dec 17, 2021
kernel: drm/vc4: Skip writes to disabled packet RAM See: raspberrypi/linux#4770 kernel: BCM2711 KMS YUV Support See: raspberrypi/linux#4777 kernel: Revert media: bcm2835-codec: Limit video callbacks See: raspberrypi/linux#4773 kernel: staging/bcm2835-isp: Fix cleanup after init fail See: raspberrypi/linux#4774
|
This seems fixed. I verified it works on armv6 and armv7. |
popcornmix
pushed a commit
that referenced
this issue
Dec 22, 2021
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Dec 22, 2021
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Dec 31, 2021
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Dec 31, 2021
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Jan 5, 2022
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Jan 5, 2022
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Jan 10, 2022
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Jan 11, 2022
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Jan 11, 2022
bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 3, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 10, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 10, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 10, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 10, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 13, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 13, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 17, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 24, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 25, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 25, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Mar 31, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Apr 7, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Apr 7, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Apr 10, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Apr 14, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Apr 14, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Apr 14, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
nmbath
pushed a commit
to victronenergy/linux
that referenced
this issue
Apr 22, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3a4f3eecb168ceb80231c3e8252929892e. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: raspberrypi/linux#4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ torvalds#8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9551c1 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Apr 24, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Apr 24, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Apr 24, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Apr 28, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
Apr 28, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
May 6, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
May 6, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
May 14, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
May 14, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
May 20, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
popcornmix
pushed a commit
that referenced
this issue
May 20, 2025
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL component to program the ISP hardware through the VC firmware. The ISP component can produce two video stream outputs, and Bayer image statistics. This can't be encompassed in a simple V4L2 M2M device, so create a new device that registers 4 video nodes. This patch squashes all the development patches from the earlier rpi-5.4.y branch into one Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Add the unpacked (16bpp) raw formats Now that the firmware supports the unpacked (16bpp) variants of the MIPI raw formats, add the mappings. Signed-off-by: Dave Stevenson <[email protected]> staging/bcm2835-isp: Log the number of excess supported formats When logging that the firmware has provided more supported formats than we had allocated storage for, log the number allocated and returned. Signed-off-by: Dave Stevenson <[email protected]> staging: vc04_services: ISP: Add colour denoise control Add colour denoise control to the bcm2835 driver through a new v4l2 control: V4L2_CID_USER_BCM2835_ISP_CDN. Add the accompanying MMAL configuration structure definitions as well. Signed-off-by: Naushir Patuck <[email protected]> bcm2835-isp: Allow formats with different colour spaces. Each supported format now includes a mask showing the allowed colour spaces, as well as a default colour space for when one was not specified. Additionally we translate the colour space to mmal format and pass it over to the VideoCore. Signed-off-by: David Plowman <[email protected]> media: i2c: add ov9281 driver. Change-Id: I7b77250bbc56d2f861450cf77271ad15f9b88ab1 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: fix mclk issue when probe multiple camera. Takes the ov9281 part only from the Rockchip's patch. Change-Id: I30e833baf2c1bb07d6d87ddb3b00759ab45a90e4 Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: add enum_frame_interval function for iq tool 2.2 and hal3 Adds the ov9281 parts of the Rockchip patch adding enum_frame_interval to a large number of drivers. Change-Id: I03344cd6cf278dd7c18fce8e97479089ef185a5c Signed-off-by: Zefa Chen <[email protected]> media: i2c: ov9281: Fixup for recent kernel releases, and remove custom code The Rockchip driver was based on a 4.4 kernel, and had several custom Rockchip parts. Update to 5.4 kernel APIs, with the relevant controls required by libcamera, and remove custom Rockchip parts. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Read chip ID via 2 reads Vision Components have made an OV9281 module which blocks reading back the majority of registers to comply with NDAs, and in doing so doesn't allow auto-increment register reading as used when reading the chip ID. Use two reads and manually combine the results. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Add support for 8 bit readout The sensor supports 8 bit mode as well as 10bit, so add the relevant code to allow selection of this. Signed-off-by: Dave Stevenson <[email protected]> media: ov9281: Add 1280x720 and 640x480 modes Breaks out common register set and adds the different registers for 1280x720 (cropped) and 640x480 (skipped) modes Signed-off-by: Dave Stevenson <[email protected]> Fixed picture line bug in all ov9281 modes Signed-off-by: Mathias Anhalt <[email protected]> Added hflip and vflip controls to ov9281 Signed-off-by: Mathias Anhalt <[email protected]> media: i2c: ov9281: Remove override of subdev name From the original Rockchip driver, the subdev was renamed from the default to being "mov9281 <dev_name>" whereas the default would have been "ov9281 <dev_name>". Remove the override to drop back to the default rather than a vendor custom string. Signed-off-by: Dave Stevenson <[email protected]> media: v4l2-subdev: add subdev-wide state struct Signed-off-by: Dom Cobley <[email protected]> media: i2c: ov9281: Add fwnode properties controls Add call to v4l2_ctrl_new_fwnode_properties to read and create the fwnode based controls. Signed-off-by: Dave Stevenson <[email protected]> media: i2c: ov9281: Sensor should report RAW color space Tested on Raspberry Pi running libcamera. Signed-off-by: David Plowman <[email protected]> Partial revert "media: i2c: add ov9281 driver." This partially reverts commit 84e98e3. The commit had merged some changes to other drivers with adding the ov9281 driver. Only the ov9281 parts have been reverted. staging/bcm2835-isp: Fix compiler warning The result of dividing a u32 by a size_t is an unsigned int on arm32 and a long unsigned int on arm64. Use "%zu" (the size_t format) to remove the build warning for 64-bit builds. Signed-off-by: Phil Elwell <[email protected]> staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes The bcm2835 ISP requires the base address of all input/output planes to have 32 byte alignment. Using a Y stride of 32 bytes would not guarantee that the V plane would fulfil this, e.g. a height of 650 lines would mean the V plane buffer is not 32 byte aligned for YUV420 formats. Having a Y stride of 64 bytes would ensure both U and V planes have a 32 byte alignment, as the luma height will always be an even number of lines. Signed-off-by: Naushir Patuck <[email protected]> vc04_services: isp: Report input node as wanting full range RAW color space RAW color spaces are more usually reported as having full range quantization. Tested using libcamera. Signed-off-by: David Plowman <[email protected]> drivers: bcm2835_isp: Allow multiple users for the ISP driver. Add a second (identical) set of device nodes to allow concurrent use of the ISP hardware by another user. This change effectively creates a second state structure (struct bcm2835_isp_dev) to maintain independent state for the second user. Node and media entity names are appened with the instance index appropriately. Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES define. Signed-off-by: Naushir Patuck <[email protected]> drivers: bcm2835_isp: Fix div by 0 bug. Fix a possible division by 0 bug when setting up the mmal port for the stats port. Signed-off-by: Naushir Patuck <[email protected]> staging/bcm2835-isp: Fix cleanup after init fail bcm2835_isp_remove is called on an initialisation failure, but at that point the drvdata hasn't been set. This causes a crash when e.g. using the cutdown firmware (gpu_mem=16). Move platform_set_drvdata before the instance probing loop to avoid the problem. See: #4774 Signed-off-by: Phil Elwell <[email protected]> bcm2835-v4l2-isp: Add missing lock initialization ISP device allocation is dynamic hence the locks too. struct mutex queue_lock is not initialized which result in bug. Fixing same by initializing it. [ 29.847138] INFO: trying to register non-static key. [ 29.847156] The code is fine but needs lockdep annotation, or maybe [ 29.847159] you didn't initialize this object before use? [ 29.847161] turning off the locking correctness validator. [ 29.847167] CPU: 1 PID: 343 Comm: v4l_id Tainted: G C 5.15.11-rt24-v8+ #8 [ 29.847187] Hardware name: Raspberry Pi 4 Model B Rev 1.4 (DT) [ 29.847194] Call trace: [ 29.847197] dump_backtrace+0x0/0x1b8 [ 29.847227] show_stack+0x20/0x30 [ 29.847240] dump_stack_lvl+0x8c/0xb8 [ 29.847254] dump_stack+0x18/0x34 [ 29.847263] register_lock_class+0x494/0x4a0 [ 29.847278] __lock_acquire+0x80/0x1680 [ 29.847289] lock_acquire+0x214/0x3a0 [ 29.847300] mutex_lock_nested+0x70/0xc8 [ 29.847312] _vb2_fop_release+0x3c/0xa8 [videobuf2_v4l2] [ 29.847346] vb2_fop_release+0x34/0x60 [videobuf2_v4l2] [ 29.847367] v4l2_release+0xc8/0x108 [videodev] [ 29.847453] __fput+0x8c/0x258 [ 29.847476] ____fput+0x18/0x28 [ 29.847487] task_work_run+0x98/0x180 [ 29.847502] do_notify_resume+0x228/0x3f8 [ 29.847515] el0_svc+0xec/0xf0 [ 29.847523] el0t_64_sync_handler+0x90/0xb8 [ 29.847531] el0t_64_sync+0x180/0x184 Signed-off-by: Padmanabha Srinivasaiah <[email protected]> staging: vc04_services: isp: Permit all sRGB colour spaces on ISP outputs ISP outputs actually support all colour spaces that are fundamentally sRGB underneath, regardless of whether an RGB or YUV output format is actually requested. Signed-off-by: David Plowman <[email protected]> drivers: staging: bcm2835-isp: Do not cleanup mmal vcsm buffer on stop_streaming On stop_streaming() the vcsm buffer handle gets released by the buffer cleanup code. This will subsequently cause and error if userland re-queues the same buffer on the next start_streaming() call. Remove this cleanup code and rely on the vb2_ops->buf_cleanup() call to do the cleanups instead. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Clear LS table handle in the firmware When all nodes have stopped streaming, ensure the firmware has released its handle on the LS table dmabuf. This is done by passing a null handle in the LS params. Signed-off-by: Naushir Patuck <[email protected]> drivers: staging: bcm2835-isp: Respect caller's stride value The stride value reported for output image buffers should be at least as large as any value that was passed in by the caller (subject to correct alignment for the pixel format). If the value is zero (meaning no value was passed), or is too small, the minimum acceptable value will be substituted. Signed-off-by: David Plowman <[email protected]> staging: vc04_services: bcm2835-isp: Drop include Makefile directive Drop the include directive. They can break the build, when one only wants to build a subdirectory. Replace with "../" for the includes in the bcm2835-isp instead. The fix is equivalent to the four patches between 29d49a7 ("staging: vc04_services: bcm2835-audio: Drop include Makefile directive")...2529ca2 ("staging: vc04_services: interface: Drop include Makefile directive") Fixes: c8f89c9 ("staging: vc04_services: ISP: Add a more complex ISP processing component") Suggested-by: Greg Kroah-Hartman <[email protected]> Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Register with vchiq_bus_type Register the bcm2835-v4l2-isp driver with the vchiq_bus_type instead of using the platform driver/device. Signed-off-by: Kieran Bingham <[email protected]> staging: vc04_services: bcm2835-v4l2-isp: Explicitly set DMA mask The platform model originally handled the DMA mask. Now that we are on the vchiq_bus we need to explicitly set this. Signed-off-by: Kieran Bingham <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The 'shipping' kernel 5.10.63+ does reboot. When the 'bleeding edge' kernel is built, it fails to reboot. I believe this bug was introduced at 5.10.80 or 5.10.81. The kernel I built last night reported itself as 5.10.83-v7+ and was built from commit: 20f6415.
What appears to happen is that the Rpi does not shut down and therefore never reboots. This is after issuing the 'sudo reboot' command from the command line. This is a serious issue with headless devices.
The text was updated successfully, but these errors were encountered: