Performance on OC20 S2EF 2M

[TommyDzh]

Hi, thank you for your fantastic work and codebase!

I'm currently testing EquiformerV2 on the OC20 S2EF 2M dataset, and I noticed a potentially unusual performance. Specifically, the validation force has already reached 16.51 meV/Å after only 3 epochs, which is lower than the reported value of 19.4 meV/Å in your paper. I’m curious if this level of performance is expected or if there might be an issue with my data processing or model implementation.

Below is the validation force loss curve for reference.

Model: equiformer_v2_N@8_L@4_M@2_31M
Model Configuration:
layer = 8, L_max = 4, batch_size (per gpu) = 4, lr = 2e-4

Hardware: 1 node, 4x A100 (40GB each)
Details:

Thanks in advance for any insights or advice on this!

[yilunliao]

Hi @TommyDzh

I think the curve you showed is evaluated on only validation ID while the paper reports the results averaged over validation ID, OOD-Ads, OOD-Cat and OOD-Both.