Add points as they get visible throughout the video
Closed this issue · 5 comments
Hi,
I wish to add points as the video progresses at inference, at the frame when they get visible. In other words, I wish to initiate the tracking of a set of points where the first visible frame is different for every point. One way to handle this is to restart the model prediction for every new point that gets visible, but then I might loose some temporal information. Is there a good way to handle this?
Thanks for your help!
Hi @Anderstask1, the model supports this type of tracking. Just choose the corresponding query frame for each point and discard all the predictions before this frame. The model will start tracking from the query frame. Please see section "Tracking manually selected points" in the Colab demo and let me know if you have other questions.
Make sense! I did not discard the prediction before the query frame, hence the poor performance.
@Anderstask1 I am working on the same thing, do you by chance have a code snippet that I can look at? I am using the online model and I restart predictions by setting is_first_step to true every time there is a new point.
@aetaffe I set the queries to be a 3-dimensional as stated in the documentation (# - queries. Queried points of shape (1, N, 3) in format (t, x, y) for frame index and pixel coordinates.). I my case, the number of points is fixed, so this step is easy to implement. Here, t is the index of the first visible frame of the respective point, and xy are coordinate values.
Got it! Thanks @Anderstask1