Training projects

Create a training project, then read and list training projects:

A training project’s name is the registry namespace where every model trained by the project is registered. The name is set when you create the project and does not change. It follows the registry’s naming rules: 1 to 64 characters of lowercase letters and digits, separated by ., _ or -. It must not have the form of an ID. To describe a project, use training projects annotate.

The evaluation dataset and grader define how candidates for the task are compared. training projects scores tabulates evaluations that use both values. An evaluation specifies its own dataset and grader, so pass the project’s values when you start one.

Set the evaluation settings when you create the training project, or set them later:

Score comparisons require the same dataset and grader. Changing either value resets the default comparison table in training projects scores. To read scores from the previous settings, pass --dataset and --grader to scores.

--grader-instructions records the instructions given to a model: grader as part of the project’s grader. A judge with explicit instructions is a different grader, so training projects scores tabulates only the evaluations that gave the judge the same instructions. The project does not apply them to an evaluation: pass the same --grader and --grader-instructions to training evaluations start. To read scores under other instructions with scores --grader, copy the grader key from the table, which ends in +instructions: and the start of the instructions' SHA-256.

For the grader kinds, see Graders and rewards.

A run registers its final model in the artifact registry as a version of a named model in the project’s namespace. The run’s modelName specifies the named model and defaults to the base model’s name, so a run over Qwen/Qwen3-0.6B in the triage project registers triage/qwen3-0.6b:v1, then triage/qwen3-0.6b:v2. A project can hold several named models, such as a judge and a policy.

candidate and champion are aliases on a named model. An alias names the model it belongs to, so a project with a judge and a policy can mark the champion of each:

The trainer records every change to a training project’s evaluation settings and description. Read the history:

Tabulate the scores of a training project:

The command compares the newest evaluation of each configuration in the training project with the baselines. Each row is a base model version, such as qwen/qwen3-0.6b:v2. It uses the project’s evaluation settings. --config-name on a run, or training runs annotate, heads the configuration’s column.