PEFT Tuning (LoRA)

What PEFT LoRA is#

PEFT (parameter-efficient fine-tuning) trains a small set of LoRA weight matrices on top of a frozen base model. The base weights never change, an adapter only adds low-rank deltas with an explicit rank (r) and alpha scaling to the attention/MLP modules listed in target_modules. The result is a checkpoint that is orders of magnitude smaller than a full fine-tune, cheap to train, and cheap to swap at serving time.

Adapters can be trained against these base model families:

Train an adapter (SFT)#

Supervised fine-tuning of one plain PEFT LoRA adapter, the standard supervised route, and the SFT counterpart of RL training's peft-lora policy. Context is inlined in the prompt, as in standard SFT, /v1/train/d2l remains the hypernetwork path. The run writes a real PEFT adapter directory (adapter_model.safetensors + adapter_config.json) directly consumable by POST /v1/eval/adapter, POST /v1/eval/rl with policy=peft-lora, and as an init_checkpoint_dir warm-start for /v1/train/rl.

curl -X POST http://localhost:8080/v1/train/sft \
  -H "Content-Type: application/json" \
  -d '{
    "family": "qwen3-4b",
    "lora_rank": 16,
    "epochs": 3,
    "eval_examples": 100
  }'

Response (202 Accepted):

{
  "job_id": "sft-1718102400",
  "checkpoint_dir": "~/hybrie-mounts/d2l-artifacts/sft-1718102400/"
}

SFT runs appear in Training Jobs as kind sft-peft-lora. Progress points stream step / loss samples, for SFT jobs the reward field carries the cross-entropy term. Runs are cancellable via DELETE /v1/train/jobs/:id, with partial checkpoints kept.

Train an adapter (GRPO)#

For reward-driven training, RL training with policy=peft-lora trains one plain PEFT adapter directly against a verifiable environment with GRPO. The run writes the same standard adapter directory (adapter_model.safetensors + adapter_config.json) you can register below and score with POST /v1/eval/adapter.

curl -X POST http://localhost:8080/v1/train/rl \
  -H "Content-Type: application/json" \
  -d '{
    "family": "qwen3-4b",
    "policy": "peft-lora",
    "lora_rank": 16,
    "prompts": 64
  }'

Response (202 Accepted):

{
  "job_id": "train-1718102400",
  "checkpoint_dir": "~/hybrie-mounts/d2l-artifacts/train-1718102400/"
}

See RL Training (GRPO) for the full training mechanics, group-relative advantages, environments, cancellation, and Training Jobs for tracking the run.

Register an external adapter#

Adapters trained outside the platform can be registered directly with the runtime, point artifact_ref at the adapter weights and the runtime serves them like any other registered adapter.

curl -X POST http://localhost:8080/v1/adapters \
  -H "Content-Type: application/json" \
  -d '{
    "id": "refund-policy",
    "version": "v3",
    "base_model": "qwen3-4b",
    "rank": 16,
    "alpha": 32,
    "target_modules": ["q_proj", "k_proj", "v_proj", "o_proj"],
    "artifact_ref": "/var/hybrie/adapters/refund-policy/v3"
  }'

Score it#

Score any PEFT adapter directory on held-out NIAH against the zero-LoRA baseline, the report includes adapter_acc, base_acc, and lift, plus the adapter's own r / lora_alpha / target_modules so the score is self-describing. See Evaluation for the full parameter and report reference.

stimulir lab eval adapter --family qwen3-4b --adapter-dir <dir>

Adapter registry#

The runtime keeps a registry of adapters it knows how to serve.

List all registered adapters. The response also reports the serving configuration: lora_mode, max_active_loras, and loaded_loras[], the adapters currently loaded into the runtime.

Metadata for a single adapter, base model, rank/alpha, target modules, and versions.

Health of the adapter subsystem: what is registered, what is currently loaded, and serving readiness.

From the CLI#

# Train a classical PEFT LoRA adapter with SFT, the standard supervised route
stimulir lab train sft --family qwen3-4b --lora-rank 16 --epochs 3

# Or train with GRPO against a verifiable environment
stimulir lab train rl --family qwen3-4b --policy peft-lora --lora-rank 16 --prompts 64

# Score it on held-out NIAH
stimulir lab eval adapter --family qwen3-4b --adapter-dir <dir>

# List registered adapters
stimulir lab adapters list

Next#

• Full GRPO mechanics, environments, KL penalty, cancellation, in RL Training (GRPO).
• Serve adapters at runtime, load, unload, and per-request routing, in Hot-swap Inference.
• Internalize documents as adapters with a hypernetwork in Doc-to-LoRA (Context Internalization).
• Track asynchronous training runs in Training Jobs.