linux-surface/iptsd

Add motion prediction

danielzgtg opened this issue · 2 comments

danielzgtg commented

I just saw Android 14 DP2 is out. https://android-developers.googleblog.com/2023/03/android-14-developer-preview-2.html . It has motion prediction https://developer.android.com/jetpack/androidx/releases/input . The library is open-source so it is possible to port it.

One would think if because Android released a whole library for this, it must be useful and important. We should investigate this motion prediction to see if we want to add it.

StollD commented

This definitely looks interesting. But I am not sure if this can help us much to be honest.

I only skimmed over the code briefly, but I got the impression that google uses this to reduce the required sampling rate on phones with high refresh rate displays. e.g. instead of sampling at 120hz, the touchscreen could sample at 30hz and Android would interpolate the missing events, making the motion more fluid.

On IPTS this is less of an issue, because at least on my SB2, IPTS already samples at 100hz, so we don't need to interpolate anything to make it smooth. The biggest problem right now is latency on the polling based IPTS driver (linux-surface/linux-surface#1061).

And I don't really think that made up events are a good idea if the true events are already coming in delayed.

StollD commented

So I went back to this and took a closer look at the code. Also read up a bit on Kalman filters.

Turns out I was completely wrong. This is in theory exactly what we need.

But there are some issues. Google claims it can make stable predictions for as much as 25ms. To get to windows levels of latency we need at least 50ms. And even with something below that threshold, the events were very jittery for me.

There are comments in the code talking about pens, but maybe this is really aimed at finger touch, where jitter isn't as noticable as with a stylus.

I pushed my code here, if you want to take a look: https://github.com/linux-surface/iptsd/tree/feature/prediction

My first idea to get rid of the jitter would be a simple low pass filter, but I don't know if that won't negate the prediction.