Reproduction on the Tank and Temple Dataset

[MickShen7558]

Hi,

First, I'd like to thank you for the nice paper and the codes!

I ran your code with default parameters on the scene Barn in Tank and Temple Dataset. Compared to the F1-score of 0.36 in the paper Table 2, I ended up with 0.28. May I know if there were any specific settings of the parameters when you extracted the mesh on the Tank and Temple Dataset? Such as the voxel_size and depth_trunc?

Thank you!

[hbb1]

Hi, we will investigate and attempt to replicate the TNT result in this new repository. We will provide you with a response within two days.

[hbb1]

Hi, we reproduce the result for the Ban by this.
First, simply modify the lambda_dist from 1000 to 100 in the DTU evaluation script. We then evaluate the score using TNT official script, you can find related script in https://github.com/autonomousvision/gaussian-opacity-fields/tree/main/eval_tn.

if not args.skip_training:
    common_args = " --quiet --test_iterations -1 --depth_ratio 1.0 -r 2 --lambda_dist 100"
    for scene in dtu_scenes:
        source = args.TNT_data + "/" + scene
        print("python train.py -s " + source + " -m " + args.output_path + "/" + scene + common_args)
        os.system("python train.py -s " + source + " -m " + args.output_path + "/" + scene + common_args)


if not args.skip_rendering:
    all_sources = []
    common_args = " --quiet --skip_train --depth_ratio 1.0 --num_cluster 1 --voxel_size 0.004 --sdf_trunc 0.016 --depth_trunc 3.0"
    for scene in dtu_scenes:
        source = args.TNT_data + "/" + scene
        print("python render.py --iteration 30000 -s " + source + " -m" + args.output_path + "/" + scene + common_args)
        os.system("python render.py --iteration 30000 -s " + source + " -m" + args.output_path + "/" + scene + common_args)

The F1 score for the Barn scene is 0.41.

The number is ok I think. We still need some time to check for other scenes, and once we have done, we will upload the full evaluation script. Thanks.

[MickShen7558]

Hi,

Thank you for uploading the codes and the results. The Barn is 0.38 on my end. However, it's already above what is reported in the paper.