LiheYoung/Depth-Anything

[Uploaded model] Depth-Anything-Small metric depth model

lukasz-staniszewski opened this issue · 4 comments

Hi, just letting you know I've trained Depth-Anything-Small for KITTI metric prediction (10 epochs).

Feel free to re-use/finetune it or put it in the repo:

Wandb run
HuggingFace checkpoint

Metrics:

Method $\delta_1 \uparrow$ $\delta_2 \uparrow$ $\delta_3 \uparrow$ AbsRel $\downarrow$ RMSE $\downarrow$ log10 $\downarrow$
ZoeDepth 0.971 0.996 0.999 0.054 2.281 0.082
Depth Anything Large 0.982 0.998 1.000 0.046 1.896 0.069
Depth Anything Small 0.971 0.996 0.999 0.053 2.267 0.023

Model size:

Method model size (MB) $\downarrow$
Depth Anything Large 1 344.4
Depth Anything Small 101.6

怎么推理

I want to use a model you've created,
but in the new model you made, there are additional parameters like

"conv2.weight", "conv2.bias", "seed_bin_regressor._net.0.weight", "seed_bin_regressor._net.0.bias", "seed_bin_regressor._net.2.weight", ..... 

and so on. Because of this, I'm unsure how to properly load the model. When using DepthAnythingV2 and attempting to use load_state_dict, I encounter the following error:

RuntimeError: Error(s) in loading state_dict for DepthAnythingV2:
	Unexpected key(s) in state_dict: "conv2.weight", "conv2.bias" ......

How can I resolve this issue?

For metric depth, we need to load the weights using the metric_depth code. You can follow this file to use the weights.

Here is how I use it in my project, where the process_image function returns numpy array of metric depths: LINK.

@lukasz-staniszewski Hi, thanks for sharing the trained model! Can you share the config_zoedetph.json file you used to train it? Thanks a lot!

@lukasz-staniszewski Hi, thanks for sharing the trained model! Can you share the config_zoedetph.json file you used to train it? Thanks a lot!

I'm pretty sure I didn't change the file, and it's the same as this one. However, there is also config_zoedepth_kitti.json, and I'm not sure which one is the exact one used under the hood.