LoRA Variants

Try LoRA+ first — it's just a hyperparameter change (set B's LR 4-16x higher than A's). Then DoRA for low-rank scenarios (r≤8). rsLoRA only matters at r≥64. Use TIES merge when combining task-specific adapters to avoid sign conflicts.