Question about evaluation

[chwang0721]

Hi Yushao,

Thanks for your work and code, but it seems that the evaluation code is missing in the repository. I have a question about the evaluation of the generated trajectories.

In section 5.2 you mention that "Specifically, we randomly generate 3,000 trajectories from each generative method and then compute all metrics".

I wonder how these 3,000 trajectories are selected. Is it by dividing the dataset into a training set and a test set, then training on the training set and generating from 3000 selected conditional information from the test dataset, or just selecting from the training set?

Could you please provide details on the evaluation process? Thank you very much.

Best,
Chenhao

[Yasoz]

I have used both methods separately and obtained very similar results.

[chwang0721]

Thank you for your response! If generated by a test set, does this mean that [depature, total_dis, total_time, total_len, avg_dis, avg_speed, sid, eid] is known before the trajectory is generated? May I ask if this is a kind of information leakage and seems difficult to do in practice?

[Yasoz]

I think this information in is easy to access, like when we use taxi-riding or navigation apps.

[chwang0721]

您好，您在上面提到了可以在训练集上训练，然后在测试集上评估。我的疑问在于训练完成后，在测试集上评估生成效果时，需要用到[depature, total_dis, total_time, total_len, avg_dis, avg_speed, sid, eid]这些条件信息，可是此时我们并不能获得测试集的数据，这些条件信息是怎么得到的呢？如果用导航软件获得这些信息，那也需要给出一定的条件（至少有起始点和出发时间），并且按照文中的假设，每条轨迹的出发时间不同，那是否需要在每天对应的时候去导航软件上获得对应轨迹的相关信息呢，这样做我认为也不是easy to access。

[Yasoz]

From a diffusion models perspective, this information can be regarded as the conditional information that guides the model generation. From a practical application point of view, departure time, length, and distance are the basic attributes.