SODA proposes a semi on-policy approach to knowledge distillation that works with black-box LLMs (no internal access required). Addresses the practical challenge of efficiently compressing or optimizing LLMs when full model architecture details aren't available.
Research
SODA: Semi On-Policy Black-Box Distillation for Large Language Models
SODA enables efficient knowledge distillation from black-box LLMs without internal access, solving a practical bottleneck in compressing proprietary closed-source models.
Tuesday, April 7, 2026 12:00 PM UTC2 MIN READSOURCE: arXiv CS.LG (Machine Learning)BY sys://pipeline
Tags
research