Multitouch support ?
Closed this issue · 4 comments
maleselo commented
Hi,
I am trying to run a simple Qt application (a simple button that changes a rectangle color) on my system (Buildroot, iMX6 and ILI251x touchscreen). I would like to remotely access this application from a VNC client (visualize and interact with).
I tried framebuffer-vncserver and vncviewer and I successfully visualize my application on the client. But I could not remotely push my button.
Input events from vncwiever generate ABS_X, ABS_Y and BTN_TOUCH events on the server side but they are not recognized and my button is not pushed as it should be.
root@target:# evtest /dev/input/event1 &
root@target:# ./framebuffer-vncserver -f /dev/fb1 -t /dev/input/ev
ent1 -v &
root@target:# Initializing framebuffer device /dev/fb1...
xres=800, yres=600, xresv=800, yresv=600, xoffs=0, yoffs=0, bpp=32
offset:length red=16:8 green=8:8 blue=0:8
No keyboard device
Initializing touch device /dev/input/event1 ...
x:(0 14800) y:(0 9800)
Initializing VNC server:
width: 800
height: 600
bpp: 32
port: 5900
rotate: 0
Initializing server...
01/01/2000 03:11:27 Listening for VNC connections on TCP port 5900
01/01/2000 03:11:27 Listening for VNC connections on TCP6 port 5900
01/01/2000 03:11:32 Got connection from client 192.168.214.248
01/01/2000 03:11:32 other clients:
01/01/2000 03:11:32 Normal socket connection
01/01/2000 03:11:32 Client Protocol Version 3.8
01/01/2000 03:11:32 Protocol version sent 3.8, using 3.8
fps: 0.000000
Dirty page: 801x600+0+0...
01/01/2000 03:11:32 rfbProcessClientSecurityType: executing handler for type 1
01/01/2000 03:11:32 rfbProcessClientSecurityType: returning securityResult for client rfb version >= 3.8
Got ptrevent: 0000 (x=400, y=300)
01/01/2000 03:11:32 Pixel format for client 192.168.214.248:
01/01/2000 03:11:32 8 bpp, depth 6
01/01/2000 03:11:32 true colour: max r 3 g 3 b 3, shift r 4 g 2 b 0
01/01/2000 03:11:32 Enabling full-color cursor updates for client 192.168.214.248
01/01/2000 03:11:32 Enabling NewFBSize protocol extension for client 192.168.214.248
01/01/2000 03:11:32 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0xFFFFFECC)
01/01/2000 03:11:32 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0xFFFFFECD)
01/01/2000 03:11:32 Enabling LastRect protocol extension for client 192.168.214.248
01/01/2000 03:11:32 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0xFFFFFEC7)
01/01/2000 03:11:32 rfbProcessClientNormalMessage: ignoring unsupported encoding type Enc(0xFFFFFEC8)
01/01/2000 03:11:32 Using compression level 1 for client 192.168.214.248
01/01/2000 03:11:32 Using image quality level 0 for client 192.168.214.248
01/01/2000 03:11:32 Using JPEG subsampling 1, Q15 for client 192.168.214.248
01/01/2000 03:11:32 Using raw encoding for client 192.168.214.248
...
Got ptrevent: 0001 (x=128, y=82)
Event: time 946696300.137120, type 1 (EV_KEY), code 330 (BTN_TOUCH), value 1
Event: time 946696300.137120, type 3 (EV_ABS), code 0 (ABS_X), value 2368
Event: time 946696300.137120, type 3 (EV_ABS), code 1 (ABS_Y), value 1339
Event: time 946696300.137120, -------------- SYN_REPORT ------------
injectTouchEvent (screen(128,82) -> touch(2368,1339), down=1)
Got ptrevent: 0000 (x=128, y=82)
injectTouchEvent (screen(128,82) -> touch(2368,1339), down=0)
Event: time 946696300.170415, type 1 (EV_KEY), code 330 (BTN_TOUCH), value 0
When I press the button directly with the touchscreen (not with the VNC client), I have got these logs:
root@target:# Event: time 946696796.496230, type 3 (EV_ABS), code 47 (ABS_MT_SLOT), value 0
Event: time 946696796.496230, type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value 102
Event: time 946696796.496230, type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 1109
Event: time 946696796.496230, type 3 (EV_ABS), code 54 (ABS_MT_POSITION_Y), value 660
Event: time 946696796.496230, type 3 (EV_ABS), code 47 (ABS_MT_SLOT), value 5
Event: time 946696796.496230, type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value -1
Event: time 946696796.496230, type 1 (EV_KEY), code 330 (BTN_TOUCH), value 1
Event: time 946696796.496230, type 3 (EV_ABS), code 0 (ABS_X), value 1109
Event: time 946696796.496230, type 3 (EV_ABS), code 1 (ABS_Y), value 660
Event: time 946696796.496230, -------------- SYN_REPORT ------------
Event: time 946696796.585903, type 3 (EV_ABS), code 47 (ABS_MT_SLOT), value 0
Event: time 946696796.585903, type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value -1
Event: time 946696796.585903, type 1 (EV_KEY), code 330 (BTN_TOUCH), value 0
Event: time 946696796.585903, -------------- SYN_REPORT ------------
It looks like framebuffer-vncserver does not generate all the required events to actually push my button. I did some modifications on the touch.c file in order to generate these events (ABS_MT_TRACKING_ID, ABS_MT_POSITION_X and ABS_MT_POSITION_Y):
diff --git a/framebuffer-vncserver.pro b/framebuffer-vncserver.pro
index fe49b51..48f778b 100644
--- a/src/touch.c
+++ b/src/touch.c
@@ -28,6 +28,7 @@ static int touchfd = -1;
static int xmin, xmax;
static int ymin, ymax;
static int rotate;
+static int trkg_id = -1;
int init_touch(const char *touch_device, int vnc_rotate)
{
@@ -101,6 +102,36 @@ void injectTouchEvent(int down, int x, int y, struct fb_var_screeninfo *scrinfo)
if (down >= 0)
{
+ // Then send a ABS_MT_TRACKING_ID
+ gettimeofday(&ev.time, 0);
+ ev.type = EV_ABS;
+ ev.code = ABS_MT_TRACKING_ID;
+ ev.value = ++trkg_id;
+ if (write(touchfd, &ev, sizeof(ev)) < 0)
+ {
+ error_print("write event failed, %s\n", strerror(errno));
+ }
+
+ // Then send a ABS_MT_POSITION_X
+ gettimeofday(&ev.time, 0);
+ ev.type = EV_ABS;
+ ev.code = ABS_MT_POSITION_X;
+ ev.value = xin;
+ if (write(touchfd, &ev, sizeof(ev)) < 0)
+ {
+ error_print("write event failed, %s\n", strerror(errno));
+ }
+
+ // Then send a ABS_MT_POSITION_Y
+ gettimeofday(&ev.time, 0);
+ ev.type = EV_ABS;
+ ev.code = ABS_MT_POSITION_Y;
+ ev.value = yin;
+ if (write(touchfd, &ev, sizeof(ev)) < 0)
+ {
+ error_print("write event failed, %s\n", strerror(errno));
+ }
+
// Then send a BTN_TOUCH
gettimeofday(&ev.time, 0);
ev.type = EV_KEY;
@@ -110,6 +141,16 @@ void injectTouchEvent(int down, int x, int y, struct fb_var_screeninfo *scrinfo)
{
error_print("write event failed, %s\n", strerror(errno));
}
+ } else {
+ // ABS_MT_TRACKING_ID
+ gettimeofday(&ev.time, 0);
+ ev.type = EV_ABS;
+ ev.code = ABS_MT_TRACKING_ID;
+ ev.value = -1;
+ if (write(touchfd, &ev, sizeof(ev)) < 0)
+ {
+ error_print("write event failed, %s\n", strerror(errno));
+ }
}
// Then send the X
With these modifications, when I push the button from the VNC client, the push is well recognized by my Qt application (I do have some multi touch x and y position issues though).
I am pretty unaware of how input devices works on Linux as well as VNC, so any suggestion will be appreciated. I would like to know if you plan to support multi touchscreen events like ABS_MT_TRACKING_ID, ABS_MT_POSITION_X and ABS_MT_POSITION_Y ?
Best regards,
Maleselo
ponty commented
I added your patch, then fixed mouse grab and mouse release after reading the doc:
https://www.kernel.org/doc/Documentation/input/multi-touch-protocol.txt
I tested it with single-touch and multi-touch devices on iMX6 (QT demo CinematicExperience, Mouse press, release, grab.) Can you test if it works on your hardware?
maleselo commented
Hi @ponty,
Thanks for your very quick answer.
I still have some issues. The ABS_MT_POSITION_X and ABS_MT_POSITION_Y events generated by the framebuffer-vncserver have different values from ABS_X and ABS_Y:
Got ptrevent: 0000 (x=148, y=118)
Got ptrevent: 0001 (x=148, y=118)
Event: time 946687240.090812, type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value 14
Event: time 946687240.090812, type 1 (EV_KEY), code 330 (BTN_TOUCH), value 1
injectTouchEvent (screen(148,118) -> touch(2738,1927), mouse=1)
Event: time 946687240.090812, type 3 (EV_ABS), code 53 (ABS_MT_POSITION_X), value 148
Event: time 946687240.090812, type 3 (EV_ABS), code 54 (ABS_MT_POSITION_Y), value 118
Event: time 946687240.090812, type 3 (EV_ABS), code 0 (ABS_X), value 2738
Event: time 946687240.090812, type 3 (EV_ABS), code 1 (ABS_Y), value 1927
Event: time 946687240.090812, -------------- SYN_REPORT ------------
Got ptrevent: 0000 (x=148, y=118)
Event: time 946687240.213149, type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value -1
injectTouchEvent (screen(148,118) -> touch(2738,1927), mouse=0)
Event: time 946687240.213149, type 1 (EV_KEY), code 330 (BTN_TOUCH), value 0
Event: time 946687240.213149, -------------- SYN_REPORT ------------
Got ptrevent: 0000 (x=148, y=118)
I tried this and it works well for me. What do you think about it ?
diff --git a/src/touch.c b/src/touch.c
index 14716f8..862af5d 100644
--- a/src/touch.c
+++ b/src/touch.c
@@ -156,7 +156,7 @@ void injectTouchEvent(enum MouseAction mouseAction, int x, int y, struct fb_var_
gettimeofday(&ev.time, 0);
ev.type = EV_ABS;
ev.code = ABS_MT_POSITION_X;
- ev.value = xin;
+ ev.value = x;
if (write(touchfd, &ev, sizeof(ev)) < 0)
{
error_print("write event failed, %s\n", strerror(errno));
@@ -166,7 +166,7 @@ void injectTouchEvent(enum MouseAction mouseAction, int x, int y, struct fb_var_
gettimeofday(&ev.time, 0);
ev.type = EV_ABS;
ev.code = ABS_MT_POSITION_Y;
- ev.value = yin;
+ ev.value = y;
if (write(touchfd, &ev, sizeof(ev)) < 0)
{
error_print("write event failed, %s\n", strerror(errno));
Here is my evtest output just in case:
evtest /dev/input/event1
Input driver version is 1.0.1
Input device ID: bus 0x18 vendor 0x0 product 0x0 version 0x0
Input device name: "ili251x Touchscreen"
Supported events:
Event type 0 (EV_SYN)
Event type 1 (EV_KEY)
Event code 59 (KEY_F1)
Event code 60 (KEY_F2)
Event code 61 (KEY_F3)
Event code 62 (KEY_F4)
Event code 330 (BTN_TOUCH)
Event type 3 (EV_ABS)
Event code 0 (ABS_X)
Value 3829
Min 0
Max 14800
Event code 1 (ABS_Y)
Value 2156
Min 0
Max 9800
Event code 47 (ABS_MT_SLOT)
Value 5
Min 0
Max 5
Event code 53 (ABS_MT_POSITION_X)
Value 0
Min 0
Max 14800
Event code 54 (ABS_MT_POSITION_Y)
Value 0
Min 0
Max 9800
Event code 57 (ABS_MT_TRACKING_ID)
Value 0
Min 0
Max 65535
Properties:
Testing ... (interrupt to exit)
ponty commented
This seems to be correct.
maleselo commented
Thank you very much