PreLoRA swaps Vision Transformers to low‑rank adapters after partial convergence, reducing trainable parameters to ~10% and tripling throughput while keeping accuracy similar. Read more: https://getnews.me/prelora-hybrid-pre-training-of-vision-transformers-cuts-90/ #prelora #visiontransformers

Krishu K Thapa, Reet Barik, Krishna Teja Chitty-Venkata, Murali Emani, Venkatram Vishwanath: PreLoRA: Hybrid Pre-training of Vision Transformers with Full Training and Low-Rank Adapters https://arxiv.org/abs/2509.21619 https://arxiv.org/pdf/2509.21619 https://arxiv.org/html/2509.21619