Exposure Control for other UVC Webcams

Question

Exposure Control for other UVC Webcams

ftc19743 opened this issue a month ago · 7 comments

ftc19743 commented a month ago

We are using an ArduCam B0454 UVC compliant webcam this season and have enjoyed the enhanced VisionPortal and CameraControls (thank you!).

We are setting the exposure manually and noticing that there does not seem to be a clear mapping from the numbers we are passing in via exposureControl.setExposure(exposure, TimeUnit.MILLISECONDS) and the camera's behavior. For example, a value of 1 for "exposure" in the previous call results in a fairly typically lighted image while a value of 2 reduces the brightness considerably (implying a shorter exposure time). This trend continues as the exposure value increases. getMinExposure(TimeUnit.MILLISECONDS) and getMaxExposure(TimeUnit.MILLISECONDS) return a range of 1-204.

We have this data from the manufacturer: https://docs.arducam.com/UVC-Camera/Adjust-the-minimum-exposure-time/ Its tempting to think that the "exposure" value used in setExposure() is being passed directly to the CAP_PROP_EXPOSURE property but with the sign flipped. However, we are fairly confident that the actual exposure time with an exposure value of "1" is considerably less than the 500ms the linked document suggests.

Is there a way to address this, or perhaps a way via the SDK to bypass ExposureControl and set UVC properties on the webcam directly?

Answer 1 · 2024-11-20T23:07:01.000Z

The issue likely stems from how setExposure() maps values to UVC properties, you could try,

Map Exposure Values: Test a range of values (e.g., 1, 5, 10, …, 200) to observe their effect on brightness and deduce the relationship. The range 1–204 may not correspond directly to milliseconds.
Use OpenCV for Direct Control: Bypass ExposureControl and set the UVC exposure directly with OpenCV:

VideoCapture capture = new VideoCapture(0); // Adjust index for your camera
capture.set(Videoio.CAP_PROP_EXPOSURE, -1 * desiredExposure); // Negate the value as needed

Use ArduCam SDK: Check if ArduCam’s SDK allows direct UVC property manipulation. This might bypass VisionPortal’s abstractions and let you control exposure time precisely.

Answer 2 · 2024-11-21T02:28:24.000Z

Hi there - author of EasyOpenCV here - a couple notes:

Using OpenCV to open a video capture device and set the exposure will not work. OpenCV has absolutely no support for external webcams on Android, and the FTC SDK Vision Portal relies entirely upon EasyOpenCV to bridge that gap. EasyOpenCV in turn relies on the FTC SDK's heavily modified copy of libuvc to implement a user-mode UVC driver to communicate with the camera within the constraints imposed upon USB device access by Android.

I couldn't seem to find the actual source code for the ArduCam SDK that @torinriley mentioned (I only found .deb binaries in the GitHub repo), but even if you could find the source, I can say with a high degree of confidence that it would not work on Android without extensive modifications.

Without having hardware in hand, it would be extremely difficult to try to debug this, as cameras tend to be the wild west and just because a camera is "UVC compatible" does not necessarily mean it is "well behaved".

@ftc19743 You mentioned wanting to try to bypass the ExposureControl class and set UVC properties directly. I don't know of a super easy way to do that off the top of my head, but if you want to trace the callstack in a debugger and try to see if that yields any clues, here's the general way we get from calling for example getMinExposure() down to initiating a low-level USB control transfer request:

You call getMinExposure() on an ExposureControl interface returned by the VisionPortal
The object behind that interface is a CachingExposureControl, which when you call getMinExposure() returns a cached value obtained from querying getMinExposure() on the delegated UvcApiExposureControl object.
The UvcApiExposureControl then delegates to getMinExposure() of UvcDeviceHandle
UvcDeviceHandle then calls into C++ code with nativeGetMinExposure()
Now inside C++ side of getMinExposure(), the call is passed along to the libuvc library with the call to uvc_get_exposure_abs().
Finally, the libuvc uvc_get_exposure_abs() function calls libusb_control_transfer() which ultimately ends up submitting the ioctl to the linux kernel.

One other thing to keep in mind is that the UVC standard also supports relative exposure (see for instance uvc_set_exposure_rel() in ctrl-gen.cpp) in addition to absolute exposure, but the FTC SDK does not expose this functionality.

Answer 3 · 2024-11-22T18:17:55.000Z

Thanks as always for the insights! We can see in our log a bit of the call stack you referenced above.

getMinExposure(NANOSECONDS)...
2024-11-22 10:40:21.524  1687-1794  Uvc                     com.qualcomm.ftcrobotcontroller      D  [jni_devicehandle.cpp:654] Java_org_firstinspires_ftc_robotcore_internal_camera_libuvc_nativeobject_UvcDeviceHandle_nativeGetMinExposure()...
2024-11-22 10:40:21.529  1687-1794  Uvc                     com.qualcomm.ftcrobotcontroller      D  [jni_devicehandle.cpp:654] ...Java_org_firstinspires_ftc_robotcore_internal_camera_libuvc_nativeobject_UvcDeviceHandle_nativeGetMinExposure()
2024-11-22 10:40:21.529  1687-1794  UvcApiExposureControl   com.qualcomm.ftcrobotcontroller      D  ...getMinExposure(NANOSECONDS): 300000

This led us to think that maybe we could poke values into the camera registers with setExposure() in the SDK and then read the exposure value back out of the camera to see what is actually stored there.

if (exposureControl.setExposure(exposure, TimeUnit.MILLISECONDS)) {
                teamUtil.log("Set WebCam Exposure to "+ exposure);
                teamUtil.log("WebCam Exposure now "+ exposureControl.getExposure(TimeUnit.MILLISECONDS));

We tried it and realized that the SDK is caching the exposure value somewhere and just returns that:

2024-11-22 10:40:21.987  1687-1855  UvcApiExposureControl   com.qualcomm.ftcrobotcontroller      D  setExposure(4 MILLISECONDS)...
2024-11-22 10:40:21.994  1687-1855  UvcApiExposureControl   com.qualcomm.ftcrobotcontroller      D  ...setExposure(4 MILLISECONDS): true
2024-11-22 10:40:21.995  1687-1855  RobotCore               com.qualcomm.ftcrobotcontroller      D  19743LOG:T173 configureCam: Set WebCam Exposure to 4
2024-11-22 10:40:22.004  1687-1855  RobotCore               com.qualcomm.ftcrobotcontroller      D  19743LOG:T173 configureCam: WebCam Exposure now 4

So the follow on question: Is there perhaps a simple way to trick the SDK into calling down through the libuvc library to get the camera's currently stored value for the exposure property and returning/logging that?

Answer 4 · 2024-11-25T01:00:13.000Z

@ftc19743

Here's an OpMode that uses reflection to access the UvcDeviceHandle object. Please note that this is absolutely not safe to use on a competition robot, do not use this code for any purposes beyond debugging attempts.

package org.firstinspires.ftc.teamcode;

import com.qualcomm.robotcore.eventloop.opmode.LinearOpMode;
import com.qualcomm.robotcore.eventloop.opmode.TeleOp;

import org.firstinspires.ftc.robotcore.external.hardware.camera.WebcamName;
import org.firstinspires.ftc.robotcore.external.hardware.camera.controls.ExposureControl;
import org.firstinspires.ftc.robotcore.internal.camera.delegating.CachingExposureControl;
import org.firstinspires.ftc.robotcore.internal.camera.libuvc.api.UvcApiExposureControl;
import org.firstinspires.ftc.robotcore.internal.camera.libuvc.nativeobject.UvcDeviceHandle;
import org.firstinspires.ftc.vision.VisionPortal;

import java.lang.reflect.Field;

@TeleOp
public class UvcControlDirectAccess extends LinearOpMode
{
    public void runOpMode()
    {
        VisionPortal portal = VisionPortal.easyCreateWithDefaults(
                hardwareMap.get(WebcamName.class, "Webcam 1"));

        UvcDeviceHandle uvcDeviceHandle = null;

        while (!isStopRequested())
        {
            if (portal.getCameraState() == VisionPortal.CameraState.STREAMING)
            {
                CachingExposureControl cachingExposureControl = (CachingExposureControl) portal.getCameraControl(ExposureControl.class);
                if (cachingExposureControl == null)
                {
                    throw new RuntimeException("Failed to get exposure control");
                }

                try
                {
                    Field f = CachingExposureControl.class.getDeclaredField("delegatedExposureControl");
                    f.setAccessible(true);
                    UvcApiExposureControl uvcApiExposureControl = (UvcApiExposureControl) f.get(cachingExposureControl);

                    Field f2 = UvcApiExposureControl.class.getDeclaredField("uvcDeviceHandle");
                    f2.setAccessible(true);
                    uvcDeviceHandle = (UvcDeviceHandle) f2.get(uvcApiExposureControl);
                }
                catch (Exception e)
                {
                    throw new RuntimeException("Failed to reflect");
                }

                uvcDeviceHandle.setExposureMode(ExposureControl.Mode.Manual);
                uvcDeviceHandle.setAePriority(false);

                break;
            }
            telemetry.addData("Camera State", portal.getCameraState());
            telemetry.update();
        }

        while (!isStopRequested())
        {
            long requestedExposure = (uvcDeviceHandle.getMinExposure() + uvcDeviceHandle.getMaxExposure()) / 2;

            uvcDeviceHandle.setExposure(requestedExposure);

            telemetry.addData("Min exposure (ns)", uvcDeviceHandle.getMinExposure());
            telemetry.addData("Max exposure (ns)", uvcDeviceHandle.getMaxExposure());
            telemetry.addData("Current exposure (ns)", uvcDeviceHandle.getExposure());
            telemetry.addData("Requested Exposure (ns)", requestedExposure);
            telemetry.update();
        }
    }
}

Answer 5 · 2024-11-27T23:35:46.000Z

Thanks, very interesting to see how to get down to the lower levels. This confirmed that the values passed to setExposure are indeed finding their way down to the camera so that uvcDeviceHandle.getExposure is returning them (translated to ns). Of course it does not solve our mystery of why "increasing" the exposure time on the camera results in a darker image. In any event, we were able to find an exposure time that results in solid CV with negligible blur even at high speeds. So all is well for now. As you pointed out, these UVC web cameras are under documented. :)

Answer 6 · 2024-11-28T00:55:04.000Z

Of course it does not solve our mystery of why "increasing" the exposure time on the camera results in a darker image.

If it's any comfort, it's not just your mystery!

Here are a couple of points from the long tutorial on Camera Controls at ftc-docs:

Logitech C270 preview becomes lighter up to exposure 655, then rolls over to dark at 656. This webcam’s Min is 0, Max is 1000.
Logitech V-UAX16 preview looks normal at exposure = 0, becomes darker up to 30-40.

Answer 7 · 2024-11-30T15:35:21.000Z

Just stating the obvious here, possibly for others...... the Sample opmode ConceptAprilTagOptimizeExposure lets you interactively play with gain and exposure on a camera to observe the effects in real-time... (connect to the RC with ScrCpy or direct HDMI)