3

LoRA Deep Dive

The Math Behind Low-Rank Adaptation

+100 XP5 min3 / 11

Overview: LoRA Deep Dive

Overview: LoRA Deep Dive

W = W₀ + BA. B is (d×r), A is (r×k). With r=16, you train 0.1% of parameters while preserving 95%+ of full fine-tuning quality. The rank r controls the expressivity-efficiency tradeoff.

1 of 3