CapSoftware/Cap

Make camera overlay window more stable (macOS)

richiemcilroy opened this issue · 17 comments

richiemcilroy commented

Issue
It's possible to break the camera window (window turns black) when the app is first launched. Seems to happen if the app has launched in the background, then you open it after the camera preview has already initialised.

Screenshot 2024-04-11 at 14 00 51

You can use this guide to help you get started developing locally as quickly as possible.

richiemcilroy commented

/bounty $25

algora-pbc commented

💎 $100 bounty • Cap

Steps to solve:

  1. Start working: Comment /attempt #25 with your implementation plan
  2. Submit work: Create a pull request including /claim #25 in the PR body to claim the bounty
  3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to CapSoftware/Cap!

Add a bounty • Share on socials

Attempt Started (GMT+0) Solution
🔴 @feliciien Apr 12, 2024, 6:21:11 PM WIP
🔴 @Anshgrover23 Jun 27, 2024, 3:29:57 PM WIP
feliciien commented

/attempt #25

Algora profile Completed bounties Tech Active attempts Options
@feliciien 5 bounties from 1 project
MDX, Rust,
JavaScript & more
Cancel attempt
richiemcilroy commented

Hey @feliciien are you working on this?

feliciien commented
richiemcilroy commented

/bounty $100

richiemcilroy commented

Upped the bounty here, would love to get this one fixed

  • 👍1
Vijaykv5 commented

Giving it a try @richiemcilroy

/attempt #25

algora-pbc commented

Here are some steps and pointers to help you get started on resolving this issue:

Steps to Address the Issue

  1. Reproduce the Issue:

    • Follow the steps to reproduce the issue: Launch the app in the background and then open it after the camera preview has already initialized.

  2. Investigate Initialization Logic:

    • The issue likely stems from the camera initialization logic in the Camera.tsx component. Specifically, the useEffect hook that initializes the camera might not handle the app being in the background properly.

  3. Modify Initialization Logic:

    • Ensure that the camera initialization logic checks if the app is in the foreground before attempting to initialize the camera. You can use the document.hidden property to check if the app is in the background.

  4. Handle App Visibility Changes:

    • Add an event listener for the visibilitychange event to handle cases where the app moves from the background to the foreground. Reinitialize the camera if necessary.

Relevant Files and Code

  • Camera Component: The main logic for the camera overlay is in the Camera.tsx file.

    • Path: /apps/desktop/src/components/windows/Camera.tsx
    • Key section to modify: 
      useEffect(() => {
  if (!videoRef.current || !selectedVideoDevice) return;
  const video = videoRef.current;
  const constraints = {
    video: {
      deviceId: selectedVideoDevice.id,
    },
  };

  if (typeof navigator === "undefined") return;

  const initializeCamera = () => {
    navigator.mediaDevices
      .getUserMedia(constraints)
      .then((stream) => {
        video.srcObject = stream;
        video.play();
        setIsLoading(false);
      })
      .catch((err) => {
        console.error(err);
      });
  };

  if (!document.hidden) {
    initializeCamera();
  }

  const handleVisibilityChange = () => {
    if (!document.hidden) {
      initializeCamera();
    }
  };

  document.addEventListener("visibilitychange", handleVisibilityChange);

  return () => {
    document.removeEventListener("visibilitychange", handleVisibilityChange);
    if (video.srcObject) {
      const stream = video.srcObject as MediaStream;
      stream.getTracks().forEach((track) => {
        track.stop();
      });
    }
  };
}, [selectedVideoDevice]);

  • App Initialization: Ensure the app's main page handles the camera window state correctly.

    • Path: /apps/desktop/src/app/page.tsx
    • Ensure that the camera window is properly initialized and managed when the app state changes.

Potential Implications

  • Security: Ensure that the camera permissions are handled securely and that the camera is not accessed without user consent.
  • Stability: Properly managing the camera initialization based on app visibility will improve the stability of the camera overlay.
  • Bugs: Be cautious of potential race conditions or state inconsistencies when handling visibility changes.

Testing

  • Test the changes thoroughly by launching the app in various states (foreground, background) and ensuring the camera overlay initializes correctly.
  • Verify that the camera permissions are requested and handled appropriately.
ItsEeleeya commented

Is the previous message automated?
I cannot reproduce the issue even when I set the app to be unfocused and minimized on start. It happens at random when I change the input video quickly between 3 devices.

Also, I keep getting the update available message even though my fork is synced with the main branch. (Apparently release is newer than main for now.)

ItsEeleeya commented

Hey @richiemcilroy! Could you please try out this branch and check if it's solved the issue?
If not I'd greatly appreciate it if you could provide a little more info on how to reproduce this.

Also, I came across a few other issues trying this. It seems like switching between cameras doesn't work properly and it gets stuck on the same input the app started with (Same commit, and the latest release).

All Tao windows come into focus by default unless specified. The camera overlay is set to always be on top. After forcing them to go behind everything, the video gets hidden and then starts playing again once the window is visible and this is without any changes required it seems.
Has anyone else reported this same issue?

If the provided fix works, then I'm going to cleanup and re-do some parts of Camera.tsx and MediaDeviceContext.tsx (And probably a few more…)

Vijaykv5 commented

Hey @ItsEeleeya
I'm currently working on fixing them and would raise a PR soon

  • 👀1
ItsEeleeya commented

Hey @ItsEeleeya I'm currently working on fixing them and would raise a PR soon

Awesome!
It'd be great if you could also fix the device changing issue

its time to go back working on the tauri v2 transition for me 👀

  • 👍1
richiemcilroy commented

hey @ItsEeleeya - you want to create your branch as a PR so I can take a look?

  • 👍1
Vijaykv5 commented

Can we close this issue @richiemcilroy if its fixed!