Retraction-Free Optimization over the Stiefel Manifold for the LoRA Fine-Tuning

arXiv:2607.25299v1 Announce Type: cross Abstract: Optimization over the Stiefel manifold plays a significant role in various machine learning tasks. Existing methods either use the retraction operators, requiring costly orthonormalization for large-scale matrices, or employ landing methods that rely on careful step size selection and penalty…

Thank you for reading this post, don't forget to subscribe!

Source: cs.AI updates on arXiv.org

Automatically aggregated summary — full article and all rights belong to the original publisher.

Leave a Comment