Scale-Invariant Monocular Depth Estimation via SSI Depth

S. Mahdi H. Miangoleh, Mahesh Reddy, Yağız Aksoy. Main pdf, Supplementary pdf, Project Page.

Proc. SIGGRAPH, 2024

Existing methods for scale-invariant monocular depth estimation (SI MDE) often struggle due to the complexity of the task, and limited and non-diverse datasets, hindering generalizability in real-world scenarios. This is while shift-and-scale-invariant (SSI) depth estimation, simplifying the task and enabling training with abundant stereo datasets achieves high performance. We present a novel approach that leverages SSI inputs to enhance SI depth estimation, streamlining the network's role and facilitating in-the-wild generalization for SI depth estimation while only using a synthetic dataset for training. Emphasizing the generation of high-resolution details, we introduce a novel sparse ordinal loss that substantially improves detail generation in SSI MDE, addressing critical limitations in existing approaches. Through in-the-wild qualitative examples and zero-shot evaluation we substantiate the practical utility of our approach in computational photography applications, showcasing its ability to generate highly detailed SI depth maps and achieve generalization in diverse scenarios.

Try our model easily on Colab :

Inference

1. Create a python conda environment as following:

conda create -n ENVNAME python=3.8
pip install -r requirements.txt

conda activate ENVNAME

2. Download our model weights from here and place them inside "./weights/" folder.

3. Run the code.

python eval.py -i PATH-TO-INPUT -o PATH-TO-OUTPUT -m SI --colorize
python eval.py -i PATH-TO-INPUT -o PATH-TO-OUTPUT -m SSI --colorize

** Our implementation boosts our SSI depth using BoostingMonocularDepth by default. If you need the base estimations at the lower receptieve field resolution run the following:

python eval.py -i PATH-TO-INPUT -o PATH-TO-OUTPUT -m SSIBase --colorize

Citation

This implementation is provided for academic use only. Please cite our paper if you use this code or any of the models.

@INPROCEEDINGS{miangolehSIDepth,
author={S. Mahdi H. Miangoleh and Mahesh Reddy and Ya\u{g}{\i}z Aksoy},
title={Scale-Invariant Monocular Depth Estimation via SSI Depth},
booktitle={Proc. SIGGRAPH},
year={2024},
}

Credits

"./model/bmd/" is adapted from BoostingMonocularDepth for their Boosting framework implementation.

"./model/bmd/pix2pix" folder was adapted from the pytorch-CycleGAN-and-pix2pix repository.

"./model/MiDaS" is adapted from MiDaS for their EfficientNet implementation.