terryky/tflite_gles_app

Adding support for UYVY colorformat usb camer

palcode opened this issue · 2 comments

Hi ,

The usb camera which we use all are having UYVY color format

Video input : 0 (Camera 1: ok) Format Video Capture: Width/Height : 1280/720 Pixel Format : 'UYVY' (UYVY 4:2:2)
The by default support YUYV as seen from copy_yuyv_image function .
Any pointers on how to add support for UYVY, which is not compute intensive.
Regards
Pallab Sarkar

Hi,
It requires OpenGLES Shader code which runs on GPU.
currently, shader code for YUYV format is ready as below.
I think you can easily modify it to support UYVY. (just change the order.)

/* ------------------------------------------------------ *
* shader for YUYV Texture
* +--+--+--+--+
* | R| G| B| A|
* +--+--+--+--+
* |Y0| U|Y1| V|
* +--+--+--+--+
* ------------------------------------------------------ */
static char vs_tex_yuyv[] = " \n\
attribute vec4 a_Vertex; \n\
attribute vec2 a_TexCoord; \n\
varying vec2 v_TexCoord; \n\
varying vec2 v_TexCoordPix; \n\
uniform mat4 u_PMVMatrix; \n\
uniform vec2 u_TexDim; \n\
\n\
void main (void) \n\
{ \n\
gl_Position = u_PMVMatrix * a_Vertex; \n\
v_TexCoord = a_TexCoord; \n\
v_TexCoordPix = a_TexCoord * u_TexDim; \n\
} \n";
static char fs_tex_yuyv[] = " \n\
precision mediump float; \n\
varying vec2 v_TexCoord; \n\
varying vec2 v_TexCoordPix; \n\
uniform sampler2D u_sampler; \n\
uniform vec4 u_Color; \n\
\n\
void main (void) \n\
{ \n\
vec2 evenodd = mod(v_TexCoordPix, 2.0); \n\
vec3 yuv, rgb; \n\
vec4 texcol = texture2D (u_sampler, v_TexCoord); \n\
if (evenodd.x < 1.0) \n\
{ \n\
yuv.r = texcol.r; /* Y */ \n\
yuv.g = texcol.g - 0.5; /* U */ \n\
yuv.b = texcol.a - 0.5; /* V */ \n\
} \n\
else \n\
{ \n\
yuv.r = texcol.b; /* Y */ \n\
yuv.g = texcol.g - 0.5; /* U */ \n\
yuv.b = texcol.a - 0.5; /* V */ \n\
} \n\
\n\
rgb = mat3 ( 1, 1, 1, \n\
0, -0.34413, 1.772, \n\
1.402, -0.71414, 0) * yuv; \n\
gl_FragColor = vec4(rgb, 1.0); \n\
gl_FragColor *= u_Color; \n\
} \n";

I've implemented to support UYVY color format.
Could you try gl2blazeface ?
(Since I don't have UYVY camera, I can't check it.)