LoQI-VPR

Distillation Improves Visual Place Recognition for Low Quality Images

Anbang Yang*, Ge Jin*, Junjie Huang, Yao Wang, John-Ross Rizzo, Chen Feng

New York University

Snapshot

A knowledge-distillation approach enhances visual place recognition accuracy under low-quality images by learning from high-quality data, achieving significant recall rate improvements across diverse VPR methods and datasets.

Method

Distillation Architecture: Our goal was to enhance existing VPR methods to extract more representative global descriptors from lower-quality images. Thus, we apply knowledge distillation techniques at the descriptor extraction level of VPR. For any VPR method, we aim to allow a student descriptor extractor handling low-quality images to approximate the output of its teacher counterpart, handling high-quality images.

Performance Improvements after Distillation

For low-quality images, the loss combination producing the highest recall rate is compared against the baseline of fine-tuning. For both distilled and fine-tuned weights, the change in VPR recall is represented as a delta relative to the performance of each method using pretrained weights. Within each dataset and method, green text indicates the greatest improvement for every R@N, whereas any decrease relative to pretrained performance is marked as red. The recall rates using pretrained weights on unmodified high-quality images (\( I^h \)) are provided as a reference.

VPR Methods	Configuration	Mapillary SLS				Nordland				Tokyo 24/7
VPR Methods	Configuration	R@1	R@2	R@5	R@10	R@1	R@2	R@5	R@10	R@1	R@2	R@5	R@10
MixVPR	pretrained (\( I^h \))	82.73	86.67	89.73	91.65	57.79	64.13	71.49	76.41	87.30	89.52	92.06	93.65
	pretrained (\( I^l \))	71.87	76.61	81.22	84.19	31.05	36.12	44.13	51.23	66.03	73.33	78.73	82.22
	finetuned	+4.43	+3.86	+3.60	+2.99	+13.37	+14.28	+14.64	+14.46	+9.52	+7.62	+6.03	+5.40
	ICKD	+4.33	+3.69	+3.20	+2.93	+15.00	+16.59	+16.78	+15.94	+8.25	+6.03	+6.35	+6.67
CricaVPR	pretrained (\( I^h \))	74.74	80.92	86.09	88.75	87.64	90.65	94.24	95.80	90.16	92.38	95.56	96.19
	pretrained (\( I^l \))	68.14	74.41	80.24	83.19	63.51	69.71	77.46	82.64	74.29	79.68	84.76	87.94
	finetuned	-0.12	-0.64	-0.77	-0.23	-2.97	-2.93	-2.97	-2.79	+5.08	+4.44	+2.54	+1.90
	ICKD	+0.89	+1.39	+1.11	+1.14	+11.63	+10.80	+8.95	+7.03	+6.98	+5.08	+6.03	+4.76
DINOv2 SALAD	pretrained (\( I^h \))	89.20	92.40	94.70	95.84	88.08	90.94	94.13	95.98	97.14	97.46	98.73	99.05
	pretrained (\( I^l \))	84.60	88.85	91.89	93.68	67.90	73.88	80.40	84.24	89.21	92.06	95.87	96.51
	finetuned	-0.22	-0.57	-0.60	-0.76	+0.04	-0.40	-0.76	-0.04	-0.63	0.00	-0.95	0.00
	ICKD	+0.52	+0.45	+0.20	+0.30	+1.56	+1.81	+1.09	+1.20	+1.59	+1.27	+0.32	0.00
NetVLAD	pretrained (\( I^h \))	49.29	55.24	62.07	67.29	5.51	6.67	8.62	11.38	60.63	63.81	69.21	74.29
	pretrained (\( I^l \))	32.60	37.67	44.92	50.13	1.99	3.01	4.93	7.07	27.94	33.02	41.90	47.94
	finetuned	+0.03	+0.01	-0.02	-0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
	MSE	+4.46	+5.32	+5.09	+5.01	+0.54	+0.43	-0.18	-0.72	+8.25	+6.35	+6.03	+5.08
AnyLoc	pretrained (\( I^h \))	56.51	62.31	68.28	72.89	12.90	16.23	20.18	24.28	88.25	91.11	94.92	96.83
	pretrained (\( I^l \))	48.04	56.45	63.84	69.17	10.11	12.97	17.86	21.92	83.49	87.30	91.75	95.87
	ICKD + Triplet	+0.99	+1.54	+2.68	+2.44	+1.74	+1.88	+1.52	+2.54	-5.71	-1.59	+1.59	-0.95

This table summarizes the effectiveness of the proposed loss functions for various VPR methods across different datasets. Distillation generally produces positive recall rate improvements with pretrained weights, outperforming fine-tuning in consistency and impact. Notably, the results for MixVPR and CricaVPR on Nordland show the distinct advantage of distillation. Cases where distillation did not yield higher recall, such as NetVLAD on Nordland or AnyLoc on Tokyo 24/7, suggest unique interactions between these methods and datasets that might require additional analysis.

The figure above further illustrates the impact of different loss combinations on VPR recall rates across methods. The consistent trends indicate that ICKD and MSE losses are generally effective, with a clear advantage for MixVPR and CricaVPR. While NetVLAD sees greater benefits from triplet loss on Nordland, ICKD and MSE prove to be the most reliable choices for maximizing VPR performance across methods and datasets.

Activation Maps

The activation maps below illustrate the effects of distillation on the feature extraction process for each VPR method: CricaVPR, NetVLAD, and MixVPR. By visually highlighting the areas of focus in each method’s encoder, these maps reveal how distillation shifts attention toward more informative, distinct scene features, while minimizing the impact of less relevant background elements, such as sky and repetitive foreground patterns. As additional reference for activation map calculation methods applicable to VPR in general, details of the mathematical approaches used in this section could be found here.

Query Image

CricaVPR

NetVLAD

MixVPR

Query Image

CricaVPR

NetVLAD

MixVPR

Query Image

CricaVPR

NetVLAD

MixVPR

Navigation Demo Video

This video demonstrates the navigation capabilities of the VPR system. Observe how the system accurately guides the user through complex environments with real-time visual recognition and localization.

BibTeX

@misc{yang2024distillationimprovesvisualplace,
      title={Distillation Improves Visual Place Recognition for Low Quality Images}, 
      author={Anbang Yang and Ge Jin and Junjie Huang and Yao Wang and John-Ross Rizzo and Chen Feng},
      year={2024},
      eprint={2310.06906},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2310.06906}, 
}

Acknowledgements

This work is supported partly by NSF Grants 2238968 and 2345139; by the National Eye Institute and Fogarty International Center under Grants R21EY033689, R33EY033689, and R01EY036667; and by the NYU IT High Performance Computing resources, services, and staff expertise. We would like to express our gratitude to Zezheng Li and Liyuan Geng for their valuable assistance with test data preprocessing. Their contributions helped facilitate the progress of this work.