Lab Workspace

RL Training (GRPO)

GRPO reinforcement-learning mechanics on the HybrIE runtime, group-relative advantages against a verifiable environment, with a hypernet policy or a classical PEFT LoRA adapter as the trained policy.

Endpoint

POST/v1/train/rl

Starts an asynchronous RL training run (202 Accepted). The policy generates completions for environment tasks, the environment scores them with a verifiable reward, and GRPO updates the policy using group-relative advantages, each prompt's completions are scored against each other, so no separate value model is needed. Two policies can be trained: a classical PEFT LoRA adapter via policy=peft-lora, or the D2L hypernet (the default).

Most users want policy=peft-lora, it trains one plain PEFT LoRA adapter. Its supervised counterpart is POST /v1/train/sft , the recommended sequence is SFT first, then continue with GRPO by passing the SFT checkpoint as init_checkpoint_dir. Start from PEFT Tuning (LoRA) for the end-to-end train / register / score workflow; this page covers the GRPO mechanics shared by both policies.

Parameters

ParameterTypeDescription
familyrequiredstringBase model family, qwen3-4b, qwen3-0.6b, or mistral-7b.
environmentstringVerifiable environment providing the reward signal. Default niah, the only reward rubric today.
policystringWhat to train: hypernet (default) or peft-lora, a classical LoRA adapter trained directly with GRPO.
lora_rankintegerLoRA rank for policy=peft-lora only; overrides the family preset.
init_checkpoint_dirstringWarm-start checkpoint: a hypernet checkpoint when policy=hypernet, or a PEFT adapter directory when policy=peft-lora, e.g. continue RL from an SFT checkpoint or an existing adapter.
promptsintegerNumber of environment prompts to train on.
group_sizeintegerGRPO group size, completions sampled per prompt.
max_new_tokensintegerGeneration budget per completion.
temperaturefloatSampling temperature during rollouts.
lrfloatLearning rate.
kl_betafloatKL penalty coefficient against the reference policy.
seedintegerSeed for reproducible task generation.
devicestringauto, metal, cuda, or cpu.
model_dirstringOptional base-model directory override.
checkpoint_dirstringArtifact destination for the trained checkpoint or adapter.

Example

curl
curl -X POST http://localhost:8080/v1/train/rl \
  -H "Content-Type: application/json" \
  -d '{
    "family": "qwen3-4b",
    "policy": "peft-lora",
    "lora_rank": 16,
    "init_checkpoint_dir": "~/hybrie-mounts/d2l-artifacts/sft-1718102400/"
  }'

Response (202 Accepted):

json
{
  "job_id": "train-1718102400",
  "checkpoint_dir": "~/hybrie-mounts/d2l-artifacts/train-1718102400/"
}

Tracking and cancelling runs

RL runs are tracked through the same job endpoints as Doc-to-LoRA training, status plus progress points (step, loss, reward). Cancellation is honored at the next prompt boundary, and a partial checkpoint is saved before the run stops:

bash
stimulir lab jobs list
stimulir lab jobs get <job-id>
stimulir lab jobs cancel <job-id>

niah is the only environment / reward rubric wired today, the runtime's Environment trait is the extensibility hook for additional verifiable environments. Measure the trained policy with POST /v1/eval/rl (run once with policy=base, then with your policy, to compute lift).