Fine-Tuning Large Language Models
Fine-tuning is the further training of a pretrained language model on task- or domain-specific data so that its general capabilities are specialized for a particular purpose. It is the second stage of the transfer learning paradigm, which pairs a self-supervised pretraining phase on broad data with one or more domain adaptation and fine-tuning steps on task-specific data [^c1]. The dominant pattern in natural language processing is large-scale pretraining followed by adaptation to particular tasks or domains [^c2], and fine-tuning is the mechanism that makes that adaptation possible for classification, generation, extraction, conversation, and code.
The field spans a wide range of techniques. Full fine-tuning updates every weight but is expensive to train and deploy. Parameter-efficient methods — adapters, prefix and prompt tuning, and especially low-rank adaptation (LoRA) and its quantized variant QLoRA — train only a small fraction of parameters; QLoRA reduces memory enough to fine-tune a 65-billion-parameter model on a single 48GB GPU [^c4]. Instruction tuning teaches a model to follow natural-language instructions across many tasks, while preference-based methods such as reinforcement learning from human feedback and direct preference optimization align model behavior with human judgments [^c3]. These approaches are complementary, and modern assistants are typically built by combining them; recent pipelines add reinforcement learning against verifiable outcomes — checked by unit tests or graders — on top of a supervised stage.
Fine-tuning is one of several ways to improve a system's behavior, and choosing among them depends on the failure mode being addressed: prompting changes how the model is instructed, retrieval-augmented generation supplies external information, fine-tuning modifies repeatable model behavior, and distillation compresses a proven workflow [^c5]. Fine-tuning is best suited to stable tasks with measurable behavioral gaps, and it is data-sensitive: scarcity can cause overfitting, poor generalization, and suboptimal performance [^c6]. The amount of data required is often smaller than expected — a curated set of roughly a thousand well-chosen examples can specialize a model for a narrow domain [^c10] — while the quality of that data frequently matters more than its volume.
By 2026 fine-tuning had also become an enterprise discipline rather than a single training run. Platforms emerged that support the full model-development lifecycle — pretraining, supervised fine-tuning, preference optimization such as [[methods/dpo|DPO]] and [[methods/odpo|ODPO]], and reinforcement learning [^c7] — with some offering outcome-driven training that applies reinforcement learning inside a customer's compliance boundary using their own data and workflows [^c9]. Managed reinforcement fine-tuning extended to open-weight models, with cloud providers offering OpenAI-compatible APIs and verifiable or AI-judged reward functions [^c15], while other vendors packaged the entire iteration loop — rollouts, evaluation, and experiment tracking — into one environment [^c12] or began with hands-on engineering services before opening a self-serve fine-tuning platform [^c13]. Sector-specific models followed: vendors released fine-tuned vertical models, such as a telecommunications model adapted for network operators [^c14]. The same period demonstrated the fragility of the managed-fine-tuning product lifecycle: one major provider retired its self-serve fine-tuning in stages, with the wind-down date for most teams set at 6 January 2027 [^c11].
At the same time, the field's risks came into sharper focus. Adaptation can erode safety behavior, and the degree of erosion depends on the relationship between alignment data and fine-tuning data, since high similarity between the two significantly weakens safety guardrails and makes models more susceptible to jailbreaks [^c8]. Practically, the field also encompasses the data used for training, the mechanics of running jobs — hyperparameters, memory optimization, and distributed training — the evaluation of results, the domains where fine-tuning is applied, the tooling that supports it, and the challenges it raises, from compute cost and data privacy to security, labor ethics, and open research questions.