Parameter-Efficient Fine-Tuning of DINOV2 Vision Transformers for Lung Nodule Classification

This paper presents an investigation into using Low Rank Adaptation (LoRA) for adapting large, Self-Supervised Vision Transformers (ViT-DINOv2) from the natural image domain to lung cancer malignancy classification. Drawing from LoRA’s successes in Natural Language Processing, it was hypothesized that LoRA would enhance performance, efficiency, and training speed in medical image analysis. Our controlled experiments using the NLSTx dataset – a collection of 857 biopsy-confirmed lung nodule CT scans – revealed that LoRA fine-tuning indeed outperforms traditional methods. LoRA fine-tuned ViT-DINOv2 demonstrated a 1.7% improvement over standard fine-tuning, a 2.7% increase in ROC AUC compared to the best existing approaches, required 85M fewer trainable parameters, and accelerated training by 25.8%. These results suggest that LoRA can be used alongside sophisticated, pretrained vision transformers to significantly advance the accuracy and efficiency of malignancy classification.