Multi-frame self-supervised monocular depth estimation with multi-scale feature enhancement

Qiqi KOU; Weichen WANG; Chenggong HAN; Chen LÜ; Deqiang CHENG; Yucheng JI

doi:10.37188/OPE.20243224.3603

Journals >Optics and Precision Engineering >Volume 32 >Issue 24 >Page 3603 > Article

Optics and Precision Engineering
Vol. 32, Issue 24, 3603 (2024)

Multi-frame self-supervised monocular depth estimation with multi-scale feature enhancement

Qiqi KOU¹, Weichen WANG², Chenggong HAN², Chen LÜ², Deqiang CHENG², and Yucheng JI^3、*

Author Affiliations

¹School of Computer Science and Technology，China University of Mining and Technology， Xuzhou226，China

²School of Information and Control Engineering，China University of Mining and Technology， Xuzhou1116，China

³Department Big Data Center，Ministry of Emergency Management， Beijing10001， China

show less

DOI: 10.37188/OPE.20243224.3603 Cite this Article

Qiqi KOU, Weichen WANG, Chenggong HAN, Chen LÜ, Deqiang CHENG, Yucheng JI. Multi-frame self-supervised monocular depth estimation with multi-scale feature enhancement[J]. Optics and Precision Engineering, 2024, 32(24): 3603 Copy Citation Text

show less

Fig. 1. Process of self-supervised depth estimation algorithm

Download full size | View in the Article

Fig. 2. Cost volume construction of multi-frame depth estimation network

Download full size | View in the Article

Fig. 3. Our depth estimation network architecture

Download full size | View in the Article

Fig. 4. Activation Module Based onVision Attention（Act-VAN）

Download full size | View in the Article

Fig. 5. Large kernel attention

Download full size | View in the Article

Fig. 6. Structure enhancement module

Download full size | View in the Article

Fig. 7. Structure of dynamic upsampling

Download full size | View in the Article

Fig. 8. Comparison of visualization results on the KITTI dataset

Download full size | View in the Article

Fig. 9. Comparison of visualization results on the CityScapes dataset

Download full size | View in the Article

Fig. 10. Comparison of results for module design on SqRel

Download full size | View in the Article

Method （M）	Resolution	Indicator of Error （Lower is better）				Accuracy of Prediction （Higher is better）
Method （M）	Resolution	AbsRel	SqRel	RMSE	RMSElog	$δ < 1.25$	$δ < 1 . 25^{2}$	$δ < 1 . 25^{3}$
Monodepth2^［8］	640×192	0.115	0.903	4.863	0.193	0.877	0.959	0.981
Johnston et al.^［26］	640×192	0.106	0.861	4.699	0.185	0.889	0.962	0.982
Packnet-SFM^［27］	640×192	0.111	0.785	4.601	0.189	0.878	0.960	0.982
VA-Depth^［28］	640×192	0.112	0.864	4.804	0.190	0.877	0.959	0.982
Zeeshan et al.^［29］	640×192	0.113	0.903	4.863	0.193	0.877	0.959	0.981
STDepthFormer^［30］	640×192	0.110	0.805	4.678	0.187	0.878	0.961	0.983
Patil et al.^［31］	640×192	0.111	0.821	4.650	0.187	0.883	0.961	0.982
Suanders et al.^［32］	640×192	0.100	0.747	4.455	0.177	0.895	0.966	0.984
Wang et al.^［33］	640×192	0.106	0.799	4.662	0.187	0.889	0.961	0.983
Manydepth^［13］	640×192	0.098	0.770	4.459	0.176	0.900	0.965	0.983
Ours	640×192	0.095	0.743	4.374	0.175	0.903	0.965	0.983
Monodepth2^［8］	1 024×320	0.115	0.882	4.701	0.190	0.879	0.961	0.982
Packnet-SFM^［27］	1 280×384	0.107	0.802	4.538	0.186	0.889	0.962	0.981
Shu et al.^［34］	1 024×320	0.104	0.729	4.481	0.179	0 893	0.965	0.984
Wang et al.^［33］	1 024×320	0.106	0.773	4.491	0.185	0.890	0.962	0.982
Manydpeth^［13］	1 024×320	0.093	0.715	4.245	0.172	0.909	0.966	0.983
Ours	1 024×320	0.090	0.703	4.213	0.170	0.913	0.966	0.983

Table 1. Test results on the KITTI dataset

View in the Article

Method	AbsRel	SqRel	RMSE	RMSElog
Monodepth2^［8］	0.129	1.569	6.876	0.187
Li et al.^［35］	0.119	1.290	6.980	0.190
Manydepth^［13］	0.114	1.193	6.223	0.170
Ours	0.111	1.165	6.196	0.168

Table 2. Test results on the CityScapes dataset

View in the Article

Model	Params/M	FLOPs/G
Monodepth2^［8］	14.3	8.0
Manydepth^［13］	14.4	10.2
Ours	22.0	16.3

Table 3. Model parametric and computational complexity

View in the Article

Experiment	Act-VAN	SEM	DySample	Indicator of Error （Lower is better）				Accuracy of Prediction （Higher is better）
Experiment	Act-VAN	SEM	DySample	AbsRel	SqRel	RMSE	RMSElog	$δ < 1.25$	$δ < 1 . 25^{2}$	$δ < 1 . 25^{3}$
1	×	×	×	0.098	0.770	4.459	0.176	0.900	0.965	0.983
2	×	×	√	0.098	0.765	4.441	0.176	0.900	0.965	0.983
3	×	×	√	0.096	0.753	4.379	0.175	0.902	0.965	0.983
4	×	√	√	0.096	0.755	4.436	0.176	0.901	0.965	0.983
5	√	√	√	0.095	0.743	4.374	0.175	0.903	0.965	0.983

Table 4. Ablation experiment of modules

View in the Article

Experiment	Method	Indicator of Error （Lower is better）				Accuracy of Prediction （Higher is better）
Experiment	Method	AbsRel	SqRel	RMSE	RMSElog	$δ < 1.25$	$δ < 1 . 25^{2}$	$δ < 1 . 25^{3}$
1	Baseline	0.098	0.770	4.459	0.176	0.900	0.965	0.983
2	VAN	0.097	0.765	4.495	0.176	0.901	0.965	0.983
3	Act-VAN	0.096	0.756	4.392	0.176	0.902	0.965	0.983
4	SEM（w/o）（LKA）	0.096	0.761	4.440	0.176	0.901	0.965	0.983
5	SEM	0.096	0.759	4.437	0.176	0.901	0.965	0.983

Table 5. Ablation experiment of modules design

Download Citation

Save the article for my favorites

Paper Information