RuntimeError: color color_image must be of color_format K4A_IMAGE_FORMAT_COLOR_BGRA32 for transformation_color_image_to_depth_camera

[GuardKing]

Hello author!

I have explored the project for a few days and some issues occured. In the beginning, I used the command pip install ak-frame-extractor to quickly install the package. But it was interrupted when installing the pyk4a1.3.0. So I couldn't install the package and run it directly through python -m ak_frame_extractor.

Here is my solution. I cloned the source code using the Git tool. Then I modifed the requirements_windows.txt by removing version restrictions for pyk4a, which successfully ran the command pip install -r requirements_windows.txt in the virtual environment I created. Then I solved some errors about importing modules. At last I ran the command python ak_frame_extractor_main.py under the ak_frame_extractor directory to try to start the main program. Fortunately it worked and the GUI appeared.

The program can extract the RGB, Depth and IR data using the supplied video samples. But when I try to extract the cloud points by checking the checkbox of cloud points, some errors occur below. As a result, As a result, it extracts the RGB, Depth and IR data excluding the cloud points under the customed Dataset directory.

F:/Download-edge/program/Extracter_test\dataset
frames_written= 0
frames_written= 1
frames_written= 2
a_matroska_file --> E:/OneDrive - tongji.edu.cn/20210927_192424_k_r2_e_000_150_138.mkv
start_offset --> 0
number_of_frames --> 3
self._frame_video_config.path_images_ouput --> F:/Download-edge/program/Extracter_test\dataset\preprocessed_data\images
self._frame_video_config.path_mesh_output --> F:/Download-edge/program/Extracter_test\dataset\preprocessed_data\cloud_points
frames_written= 0
Exception in Tkinter callback
Traceback (most recent call last):
  File "F:\software\Python\lib\tkinter\__init__.py", line 1705, in __call__
    return self.func(*args)
  File "F:\Download-edge\program\ak_frame_extractor\src\gui_frame_ext\gui_frame_ext_console.py", line 620, in extract_file_data
    an_offset, a_number_of_frames)
  File "F:\Download-edge\program\ak_frame_extractor\src\video_extraction_management\frame_extraction.py", line 412, in export_frames_to_colorized_mesh
    color_data = capture.transformed_color[..., (2, 1, 0)].reshape((-1, 3))
  File "F:\myenv\test_venv\lib\site-packages\pyk4a\capture.py", line 178, in transformed_color
    "color color_image must be of color_format K4A_IMAGE_FORMAT_COLOR_BGRA32 for "
RuntimeError: color color_image must be of color_format K4A_IMAGE_FORMAT_COLOR_BGRA32 for transformation_color_image_to_depth_camera

So my question is how to solve the error and extract the cloud points successfully? Is something wrong that the way I run the project?

I would be grateful if I could get a reply from you! Becasue I have focused on this for a long time and i really wish it can be fixed. And my OS is windows10.

[juancarlosmiranda]

Dear Guardking, thank you very much for your comments.
Yes, it is true that the program presented an error to extract the cloud points. It is because, the videos must be in BGRA MODE, this is a mode that you can select from the camera.
I will check the error following your suggestions.

[juancarlosmiranda]

Dear Guardking, a new version was tested (https://pypi.org/project/ak-frame-extractor/), I added videos in BGRA32 mode, to test the software. You can download it from here (https://zenodo.org/record/8232445). The tests I performed were based on the video 1080_07072022180929.mkv.
According to the manufacturer's site, the best way to extract point clouds is BGRA32. The error was due to the fact that the other videos are in MJPG format.
Dependencies have been updated.
Steps followed for install:
Setting Python environment

Installing package

Using extracted data in CloudCompare

[juancarlosmiranda]

Thank you for your report!

[juancarlosmiranda]

👍 :)

[GuardKing]

@juancarlosmiranda Thanks for your sweet reply!

I received your reply on time a month ago, but I'm sorry that I couldn't respond immediately. You solved the problem I asked about perfectly. However, due to the long delay in your response, I ended up solving the issue by writing some code to extract the point cloud data by my way. The key point indeed was the BGRA32 format. Although I don't have the opportunity to use your project now, I still want to sincerely express my gratitude for your work and responsible reply.

Wishing you a good day!