0.5.0

A grpo or gspo run on a deployment whose reinforcement engine is verl can train on multiple GPUs on one node. training info shows the deployment’s engine in its Engine row. Set execution.gpus to the number of devices. Each device holds a training rank and a rollout engine. During each step, all devices generate their share of the rollouts and then update the policy. Multi-GPU runs support checkpoints, pauses, resumes, and branches.

config.rlHyperparams.promptsPerStep must be divisible by execution.gpus. The trainer checks this when you submit the run. The deployment’s catalog states the qualified GPU counts for each model and method. If the requested count has no matching envelope, the trainer refuses the run and lists the qualified counts, such as at 1 or 4 GPU(s).

rolloutGpus and stalenessThreshold are not supported. The rollout engine runs on the training devices, so leave rolloutGpus out or set it to 0, and count every device in gpus. The trainer refuses either field above zero and says what to set instead.

To see what fits, use the MAX CLAIM column from training capacity. It shows the largest device claim that each pool accepts. training base-models get lists the runs that the deployment admits for a model, including GPU count, memory per GPU, and host memory per node.

For more information, see Execution settings.

A reinforcement run under verl publishes periodic snapshots and supports these operations:

A run that starts from another run’s adapter, through a weights branch or a warm start, must set klBeta to 0 under verl. verl’s reference policy is the base model, so a KL penalty would pull the run toward the base model rather than toward the adapter it started from. The trainer refuses a positive klBeta at submission.

For more information, see Checkpoints and Branches.

The trainer and its training jobs publish metrics, spans, and attributes for dashboards and alerts across runs. See Observability for the complete generated reference.

Stalled runs. A training job counts its completed steps. When a job stops responding, its step gauges retain their last values but the completed-step count stops increasing. The Observability page includes an alert query for this condition. A supervised or TRL job that stops completing steps also records every thread’s stack and a device-memory summary beside its metrics.

Supervised runs. A supervised run reports device-memory use and breaks down each step’s peak into weights, adapter, gradients, optimizer state, activations, allocator memory, and driver memory. It reports the duration of each step phase, evaluation, and checkpoint save, along with throughput and model FLOPs utilization (MFU). It counts rows truncated at maxSeqLength, answers truncated at that limit, and dropped rows, and reports the padding share of each batch. runs get prints these counts as Data fit, and the memory split of the step that reserved the most as Memory.

Device memory on TRL. A reinforcement run on TRL publishes the same split of each step’s peak, adding generation, and its reserved memory. The split is available only for a job that uses one device.

Reward graders. A reinforcement run measures how long each reward term takes to score a rollout, how many scores fail and why, and how many tokens a judge uses. When a judge call times out, fails to connect, or returns no verdict, the rollout scores zero for that term and the failure metric states the cause. Previously, the only sign of these failures was a lower reward.

Reward and length distributions. A reinforcement run records the reward of each rollout, in total and for each term, and the length of each completion as histograms. The step gauges give only a mean and a spread. With the histograms, a dashboard can show percentiles, a bimodal reward, and a long tail of completion lengths.

Throughput and efficiency. A reinforcement run, on verl or TRL, publishes the tokens processed by each step, throughput in tokens per second, and model FLOPs utilization (MFU) of the policy update. Previously, these figures were available only in TensorBoard.

Phase timings on TRL. A reinforcement run on TRL publishes akka.optimize.training.rl.step.timing for generation, reward scoring, and the policy update, using the same phase names as verl. Previously, only a run on verl published phase timings.

Additional step metrics. A reinforcement run publishes the shortest and longest completion of each step and the spread of each reward term’s share across the completions. A run on verl also publishes policy loss. A run on TRL publishes the mean length of completions that ended before the maximum, the share of tokens clipped at each policy-ratio bound, and, for a run that declares tools, tool calls per completion and their failure share. Previously, TRL wrote these figures only to metrics.jsonl.

GPU placement. Each process of a training job publishes the pod, node, and GPU that it runs on, so a dashboard can join the cluster’s GPU metrics to a run by the GPU’s UUID. The Observability page includes an example query.

GPU health. Every training job publishes the power, temperature, utilization, memory use, clock frequency, throttle reasons, and memory errors of each GPU that it uses, and counts the XID errors that the driver reports for it. Each series carries the GPU’s UUID. The Observability page describes how to read the throttle reasons.

Capacity over time. The service publishes gauges for the declared, observed, and claimed devices of each GPU pool, and for the claimed devices and quota of each workload. A dashboard can show pool occupancy and queued claims over time. The training capacity command shows a single point in time.

A reinforcement run keeps every rollout that it scores, with the prompt, the completion, the score, and the share of each reward term. The runs rollouts command reads them one step at a time. To check that a rising reward comes from better answers and not from reward hacking, read the worst and best rollouts of a step:

Replace RUN_ID with the ID of the run.

A run that trained on an earlier version has no rollouts.

For more information, see Rollouts.

An image-text model such as Gemma 4 E4B can take a grpo or gspo run over a text dataset. Previously, such a model supported only sft. As in a supervised run, the adapter updates the language backbone and leaves the vision and audio towers unchanged.

Under TRL, the catalog lists grpo and gspo for such a model. Under verl, it lists them where the deployment enables reinforcement over image-text models. To check which methods a model supports in your deployment, run:

A reinforcement run cannot train on images. The trainer rejects a grpo or gspo run over a dataset with images, regardless of the base model.

A reinforcement run reports the modules that its adapter covers, as a supervised run does. Another run can use that adapter as its warm start.

For more information, see Reinforcement learning.

A run that exhausts memory reports whether device memory or host memory ran out and says what you can change:

When a step of a run fails on every retry, the failure reason names what the run could not do, such as "this run’s job could not be started on the training cluster". When the training cluster does not answer while a run waits for its devices, the reason is "the training cluster could not be reached while this run waited for its devices". Previously, such a run read "step failed after retries" and the phase it was in.

For more information, see Why a run failed.

A dataset upload streams to storage as it is read, so it can use the deployment’s full upload limit, 256 MiB by default. Previously, an upload over 8 MiB was reset without a reason. A deployment sets the limit with trainer.datasets.max-bytes.

The runs grafana command prints a link to the run on the project’s training dashboard in Grafana. The link covers the run from its start to its end, and it carries a token that lets anyone who has it view the project’s Grafana until the expiry that the command prints.

For more information, see Grafana.

The --show-resolved flag on runs start --dry-run and pipelines start --dry-run prints the configuration and execution settings that a submission trains under. The output fills in every default and resolves every dataset reference, so you can check a plan before it uses any devices. The output matches what runs get prints for a started run. With -o json, the output includes the resolved settings as structured data.

To print the resolved settings for a run or a pipeline, run:

For a pipeline, the output covers each train stage whose dataset is registered. A stage that trains on the output of an earlier stage has no registered dataset to resolve, so the output omits it.

For more information, see Start a run and Pipelines.

akka can use the login of an akka context other than the current one. Name the context with the global --context flag or the AKKA_CONTEXT environment variable. akka exchanges the token at the API host that the context specifies, so one shell can reach a service on another platform. A context name that the configuration file does not contain is refused, and the error lists the contexts that the file contains. The credential environment variables, such as AKKA_ACCESS_TOKEN, still take precedence over any login.

For more information, see Authenticate.