LoRA
8 mentions across all digests
LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique for large models that trains small low-rank matrices instead of full weights, applied across heterogeneous hardware, multimodal models, and Apple Silicon fine-tuning pipelines.
We Got Claude to Fine-Tune an Open Source LLM
Where Should LoRA Go? Component-Type Placement in Hybrid Language Models
Strategic LoRA placement in hybrid model architectures significantly impacts efficiency-performance trade-offs, offering practical guidance for optimizing parameter-efficient fine-tuning across modular components.
Annotation Entropy Predicts Per-Example Learning Dynamics in LoRA Fine-Tuning
Annotation entropy can predict how individual examples will learn during LoRA fine-tuning, enabling smarter data selection and training optimization.
Aletheia: Gradient-Guided Layer Selection for Efficient LoRA Fine-Tuning Across Architectures
Gradient-guided layer selection lets LoRA concentrate fine-tuning only on high-impact layers, cutting computational costs while preserving performance across architectures.
ALTO: Adaptive LoRA Tuning and Orchestration for Heterogeneous LoRA Training Workloads
ALTO system optimizes LoRA training efficiency across heterogeneous hardware through adaptive tuning and workload orchestration, solving the practical challenge of scaling fine-tuning across diverse production compute clusters.