Researching | Cross-Modal Hash Method Based on Multi-Scale Fusion and Projection Matching Constraint

Journals >Laser & Optoelectronics Progress >Volume 59 >Issue 24 >Page 2410006 > Article

Laser & Optoelectronics Progress
Vol. 59, Issue 24, 2410006 (2022)

Cross-Modal Hash Method Based on Multi-Scale Fusion and Projection Matching Constraint

Wanyu Deng, Yina Zhao^*, Wanzhen Yang, Bo Zhang..., Hao Li and Shuqi Ye|Show fewer author(s)

Author Affiliations

School of Computer Science & Technology, Xi'an University of Posts & Telecommunications, Xi'an 710121, Shaanxi , China

show less

DOI: 10.3788/LOP202259.2410006 Cite this Article Set citation alerts

Wanyu Deng, Yina Zhao, Wanzhen Yang, Bo Zhang, Hao Li, Shuqi Ye. Cross-Modal Hash Method Based on Multi-Scale Fusion and Projection Matching Constraint[J]. Laser & Optoelectronics Progress, 2022, 59(24): 2410006 Copy Citation Text

show less

Framework of proposed MFPMC

Fig. 1. Framework of proposed MFPMC

Download full size

P-R curves of Image2Text on MIRFlickr-25K dataset when Hash code length is 16 bit

Fig. 2. P-R curves of Image2Text on MIRFlickr-25K dataset when Hash code length is 16 bit

Download full size

Fig. 3. P-R curves of Text2Image on MIRFlickr-25K dataset when Hash code length is 16 bit

Download full size

Fig. 4. P-R curves of Image2Text on NUS-WIDE dataset when Hash code length is 16 bit

Download full size

Fig. 5. P-R curves of Text2Image on NUS-WIDE dataset when Hash code length is 16 bit

Download full size

Fig. 6. Influence of hyper-parameter

ξ

on mAP on MIRFlickr-25K dataset

Download full size

Fig. 7. Influence of hyper-parameter

τ

on mAP on MIRFlickr-25K dataset

Download full size

Fig. 8. Influence of hyper-parameter

γ

on mAP on MIRFlickr-25K dataset

Download full size

Fig. 9. Influence of hyper-parameter

η

on mAP on MIRFlickr-25K dataset

Download full size

Fig. 10. Influence of hyper-parameter

μ

on mAP on MIRFlickr-25K dataset

Download full size

Input	Layer	Kernel size	Stride	Output
Original image	Average pooling 1	5×5	5×5	Ipool 1
Ipool 1	1×1Conv	1×1	1×1	IMs-feture 1
Original image	Average pooling 2	10×10	10×10	Ipool 2
Ipool 2	1×1Conv	1×1	1×1	IMs-feature 2
Original image	Average pooling 3	15×15	15×15	Ipool 3
Ipool 3	1×1Conv	1×1	1×1	IMs-feature 3

Table 1. Detailed parameter settings for IMFM

Input	Layer	Kernel size	Stride	Output
Bow vector	Average pooling 1	1×50	1×50	Tpool 1
Tpool 1	1×1Conv	1×1	1×1	TMs-feture 1
Bow vector	Average pooling 2	1×30	1×30	Tpool 2
Tpool 2	1×1Conv	1×1	1×1	TMs-feature 2
Bow vector	Average pooling 3	1×15	1×15	Tpool 3
Tpool 3	1×1Conv	1×1	1×1	TMs-feature 3
Bow vector	Average pooling 4	1×10	1×10	Tpool 4
Tpool 4	1×1Conv	1×1	1×1	TMs-feature 4
Bow vector	Average pooling 5	1×5	1×5	Tpool 5
Tpool 5	1×1Conv	1×1	1×1	TMs-feature 5

Table 2. Detailed parameter settings for TMFM

Method	Image2Text			Text2Image
Method	16 bit	32 bit	64 bit	16 bit	32 bit	64 bit
SCM	0.6157	0.6213	0.6268	0.6102	0.6284	0.6292
SePH	0.6481	0.6453	0.6596	0.6457	0.6476	0.6508
STMH	0.5877	0.5901	0.6001	0.5863	0.5877	0.5879
CMFH	0.5780	0.5827	0.5861	0.5784	0.5878	0.5889
DCMH	0.7219	0.7332	0.7450	0.7526	0.7576	0.7704
PRDH	0.7052	0.7125	0.7208	0.7607	0.7739	0.7784
CMHH	0.7302	0.7387	0.7444	0.7320	0.7283	0.7301
MFPMC	0.7501	0.7608	0.7687	0.7764	0.7895	0.7898

Table 3. Comparison of mAP values of different methods on MIRFlickr-25K dataset

Method	Image2Text			Text2Image
Method	16 bit	32 bit	64 bit	16 bit	32 bit	64 bit
SCM	0.4905	0.4946	0.4995	0.4598	0.4660	0.4701
SePH	0.5324	0.5350	0.5529	0.5078	0.5095	0.5177
STMH	0.4354	0.4471	0.4544	0.3895	0.4098	0.4187
CMFH	0.3925	0.3958	0.3990	0.3956	0.3955	0.3978
DCMH	0.5257	0.5375	0.5458	0.5792	0.5875	0.5944
PRDH	0.5919	0.6058	0.6116	0.6155	0.6287	0.6349
CMHH	0.5530	0.5697	0.5559	0.5739	0.5786	0.5639
MFPMC	0.6042	0.6196	0.6256	0.6246	0.6375	0.6437

Table 4. Comparison of mAP values of different methods on NUS-WIDE dataset

Task	Method	MIRFlickr-25K	NUS-WIDE
Image2Text	Base	0.7250	0.5630
	Base+IMFM	0.7312	0.5734
	Base+TMFM	0.7324	0.5727
	Base+IMFM+TMFM	0.7401	0.5833
	Base+LFPMC	0.7568	0.5974
	Base+IMFM+TMFM+LFPMC	0.7687	0.6256
Text2Image	Base	0.7341	0.5605
	Base+IMFM	0.7398	0.5769
	Base+TMFM	0.7433	0.5834
	Base+IMFM+TMFM	0.7522	0.6008
	Base+LFPMC	0.7698	0.6294
	Base+IMFM+TMFM+LFPMC	0.7898	0.6437

Table 5. Comparison of mAP values of ablation experiments

Get PDF(in Chinese)

Figures&Tables (15)

References (28)

Copy Citation Text

Wanyu Deng, Yina Zhao, Wanzhen Yang, Bo Zhang, Hao Li, Shuqi Ye. Cross-Modal Hash Method Based on Multi-Scale Fusion and Projection Matching Constraint[J]. Laser & Optoelectronics Progress, 2022, 59(24): 2410006

Download Citation

Set citation alerts for the article

Set citation alerts for the article

Save the article for my favorites

Paper Information

Recommended Topics

laser devices and laser physics

Lasers and Laser Optics

laser manufacturing

Instrumentation, Measurement and Metrology

Set citation alerts for the article

Please enter your email address