Distillations

A distillation uses a teacher model to answer every decision in a chat dataset. It keeps accepted answers as training rows. Use these rows to train a small model on a larger model’s answers to your agents' conversations.

A decision is one assistant message in a source row. The teacher receives every message before it and the row’s tools. It writes its own answer in place of that message. A row with three assistant messages has three decisions.

Each accepted answer becomes a row: the decision’s history, then the teacher’s answer as one assistant message. An answer that calls tools carries its text and its calls in that message, and the calls are left unanswered. A supervised run trains on such rows. See Datasets.

Once every decision is answered or rejected, Optimize releases the distillation’s GPUs and sets its phase to ASSEMBLING. Optimize checks the saved answers and registers the accepted rows as a version of the target dataset. The distillation then ends as COMPLETED. A distillation with no accepted answer fails instead.

Start a distillation

Name the teacher, the source dataset, and the target dataset for the accepted rows:

akka training distillations create --project triage \
  --teacher google/gemma-4-31b-it:v1 --from triage/agent-traffic --to triage/distilled \
  --replicas 4

Specify the teacher by its registry reference. It can be a base model or an adapter over a base model. Teacher, source and target references without a namespace use the training project’s namespace.

A distillation runs in two steps:

The smoke answers the first 16 decisions in the source on one GPU. It skips decisions whose history cannot be made into a prompt. It also skips decisions whose prompt leaves less room than --max-tokens within --max-model-length. All but one must be accepted. If the source has fewer decisions, the smoke uses fewer. The required accepted count decreases by the same amount, to a minimum of one. To choose the decisions, specify them with --smoke. Every specified decision must be accepted. If too few are accepted, the distillation fails and reports the rejection reasons.
After the smoke passes, the distillation claims one GPU per replica and answers decisions across the whole source.

Because a distillation can hold several GPUs for days, create asks for confirmation. Optimize refuses a replica count above the training project’s GPU quota or the number of GPUs always available. It also refuses a count that fits only within a capacity window. The window can end while the distillation runs.

Optimize refuses the following and reports the reason:

a teacher with a model family whose tool calls Optimize cannot read;
--thinking on for a teacher with a model family whose reasoning Optimize cannot read;
a teacher with full weights rather than an adapter;
a teacher whose base model has no commit recorded;
a source that contains images;
a source of prepared token rows.

In advisory mode, a teacher outside the device’s serving and training measurements starts with an unmeasured warning. The requested --max-model-length must fit a measurement too, or it warns. Strict mode refuses a teacher with serving measurements alone where no serving envelope names the device, or where its context is shorter than --max-model-length.

training distillations create takes the following options:

Option Default Description

Option	Default	Description
`--teacher`	none	The model that answers, by registry reference. Required.
`--from`	none	The source chat dataset, by registry reference. Required.
`--to`	none	The named dataset for the accepted rows. Required.
`--smoke`	The first 16 decisions	The decisions the smoke answers. Use `ROW:MESSAGE` pairs separated by commas, such as `0:2,5:1`. Row and message indexes start at 0. The message index identifies an assistant message in the row’s `messages`.
`--project`	none	The training project, by name or ID. Required.
`--replicas`	`1`	The teacher replicas after the smoke, one GPU each.
`--max-tokens`	`4096`	The maximum number of tokens in an answer, including reasoning.
`--max-model-length`	`65536`	The maximum combined token count for a prompt and its answer. A decision whose prompt leaves less room than `--max-tokens` is rejected as `prompt_too_long`.
`--temperature`	`1.0`	The sampling temperature, from 0 to 2.
`--top-p`	`1.0`	The nucleus sampling bound, above 0 and at most 1.
`--seed`	`0`	The seed used to derive each decision’s sampling seed.
`--thinking`	`off`	`on` asks the teacher to reason before it answers. The rows exclude the reasoning.
`--instructions`	none	Text appended to each prompt’s system message.
`--description`	none	A description of the distillation. Display it with `get`.
`--wait`	off	Wait until the distillation ends. Waiting continues while the distillation is paused.
`--exit-status`	off	Exit with a non-zero status if the distillation ends as `FAILED` or `CANCELLED`. Requires `--wait`.

--teacher

none

The model that answers, by registry reference. Required.

--from

none

The source chat dataset, by registry reference. Required.

--to

none

The named dataset for the accepted rows. Required.

--smoke

The first 16 decisions

The decisions the smoke answers. Use ROW:MESSAGE pairs separated by commas, such as 0:2,5:1. Row and message indexes start at 0. The message index identifies an assistant message in the row’s messages.

--project

none

The training project, by name or ID. Required.

--replicas

1

The teacher replicas after the smoke, one GPU each.

--max-tokens

4096

The maximum number of tokens in an answer, including reasoning.

--max-model-length

65536

The maximum combined token count for a prompt and its answer. A decision whose prompt leaves less room than --max-tokens is rejected as prompt_too_long.

--temperature

1.0

The sampling temperature, from 0 to 2.

--top-p

1.0

The nucleus sampling bound, above 0 and at most 1.

--seed

0

The seed used to derive each decision’s sampling seed.

--thinking

off

on asks the teacher to reason before it answers. The rows exclude the reasoning.

--instructions

none

Text appended to each prompt’s system message.

--description

none

A description of the distillation. Display it with get.

--wait

off

Wait until the distillation ends. Waiting continues while the distillation is paused.

--exit-status

off

Exit with a non-zero status if the distillation ends as FAILED or CANCELLED. Requires --wait.

If the connection drops before Optimize responds, the CLI sends the same request again. The repeated request uses the distillation started by the first request. Running create again starts another distillation.

Follow a distillation

akka training distillations list --project triage
akka training distillations get DISTILLATION_ID
akka training distillations wait DISTILLATION_ID

Identify a distillation by its full ID or a unique ID prefix. wait returns once the distillation has ended. It continues waiting while the distillation is paused. Use get and wait to see the phase, completed and total decisions, and accepted and rejected counts. They also show rejection counts by reason, the smoke result, and GPU time for each attempt. Once the distillation completes, they show the registered dataset version.

The phases are AWAITING_CAPACITY, STARTING_JOB, SMOKE_TESTING, GENERATING, STOPPING, PAUSED, ASSEMBLING, COMPLETED, FAILED and CANCELLED.

Attempt numbers only increase. The smoke, each retry after a failure, and each resume use the next number. Optimize retries a retryable failure up to three times. Work is saved in chunks of the source. A retry or resume continues from saved chunks. It repeats at most the chunk each replica was processing.

Pause, resume and cancel

akka training distillations pause DISTILLATION_ID
akka training distillations resume DISTILLATION_ID
akka training distillations cancel DISTILLATION_ID

pause stops the distillation and releases its GPUs. resume claims them again and continues. cancel stops the distillation and sets its phase to CANCELLED. A cancelled distillation cannot be resumed. Its rejected outputs remain readable.

While a distillation is ASSEMBLING, it holds no GPUs:

Optimize refuses pause.
resume tries again immediately while the distillation waits for a response. Otherwise Optimize refuses resume.
cancel ends the distillation immediately, unless registration is running. While registration runs, Optimize refuses cancel, and the distillation completes or fails.

The registered dataset

Optimize registers the accepted rows as a version of the dataset you name with --to. The version’s lineage records the teacher model version and the source dataset version. It also records that the distillation produced the version. To train on this version, use its reference, such as triage/distilled:v1.

If the target already holds a version with the same content, the distillation records that version rather than registering a new one, and that version keeps its original lineage. Optimize divides the rows into parts of a configured size. A version has the same content only when it holds the same rows divided into the same parts.

If Optimize cannot register the dataset because it receives no response, the distillation waits and tries again. It does not answer any decision again. Use get to see what the distillation is waiting for in its stage.

A distillation fails in ASSEMBLING if Optimize cannot register its saved answers as they are. The failure reason begins with one of these:

Reason Cause

Reason	Cause
`nothing accepted`	None of the saved answers are accepted.
`coverage gap`	Some decisions have no saved answer.
`overlap`	A decision has more than one saved answer, or answers are out of order.
`digest mismatch`	The saved answers no longer match the record made when they were saved.
`count mismatch`	The recorded counts disagree with the saved answers or the counts Optimize reports for the dataset.
`generation mismatch`	Some answers were generated by another teacher, from another source, or with other settings.
`shard too large` or `too many shards`	The accepted rows are too large for a dataset version.
`the registry refused the dataset`	Optimize refuses the rows or the target. The reason follows.

nothing accepted

None of the saved answers are accepted.

coverage gap

Some decisions have no saved answer.

overlap

A decision has more than one saved answer, or answers are out of order.

digest mismatch

The saved answers no longer match the record made when they were saved.

count mismatch

The recorded counts disagree with the saved answers or the counts Optimize reports for the dataset.

generation mismatch

Some answers were generated by another teacher, from another source, or with other settings.

shard too large or too many shards

The accepted rows are too large for a dataset version.

the registry refused the dataset

Optimize refuses the rows or the target. The reason follows.

Optimize keeps the saved answers after a failure. You can still read the rejected outputs.

Rejected outputs

List the outputs that failed acceptance, in source order:

akka training distillations rejections DISTILLATION_ID --limit 20

The table shows each rejected output’s decision, reason, the start of its detail, and the finish reason. With -o json, each line is the whole record, which adds the teacher’s output as it was generated, its token counts, and the reasoning, answer and calls read from it. With -o json and no field selection or jq filter, each record is printed as it is read. If reading fails part way, the list ends early and the command reports an error. Rejected outputs never become training rows.

An output is accepted if it meets all of these conditions:

It finishes without running out of tokens.
Its reasoning is closed.
Every tool call is well formed and names an offered tool.
Every tool call has arguments that match the tool’s schema.
It contains text or at least one tool call.
It forms a valid training row.

A rejected output has one of the following reasons:

Reason The output

Reason	The output
`truncated`	Ran out of tokens before it finished.
`reasoning_unclosed`	Ends inside its reasoning.
`reasoning_malformed`	Has reasoning markers out of order.
`stray_marker`	Contains a model format marker in an invalid position.
`call_unframed`	Contains an unclosed tool call or unread call syntax.
`text_after_call`	Has text after its last tool call.
`call_unparsed`	Has a tool call that cannot be read.
`unknown_tool`	Calls a tool the prompt did not offer.
`arguments_invalid`	Calls a tool with arguments that are not complete JSON or do not match the tool’s schema.
`empty`	Contains no text and no tool call.
`invalid_row`	Does not form a valid training row.
`invalid_source`	Was not generated because the decision’s history could not be made into a prompt.
`generation_failed`	Was not generated because generating its chunk failed on more than one replica.
`prompt_too_long`	Was not generated because the prompt leaves less room than `--max-tokens` within the model’s length.
`engine_refused`	Was not generated because the teacher refused the request as invalid.

truncated

Ran out of tokens before it finished.

reasoning_unclosed

Ends inside its reasoning.

reasoning_malformed

Has reasoning markers out of order.

stray_marker

Contains a model format marker in an invalid position.

call_unframed

Contains an unclosed tool call or unread call syntax.

text_after_call

Has text after its last tool call.

call_unparsed

Has a tool call that cannot be read.

unknown_tool

Calls a tool the prompt did not offer.

arguments_invalid

Calls a tool with arguments that are not complete JSON or do not match the tool’s schema.

empty

Contains no text and no tool call.

invalid_row

Does not form a valid training row.

invalid_source

Was not generated because the decision’s history could not be made into a prompt.

generation_failed

Was not generated because generating its chunk failed on more than one replica.

prompt_too_long

Was not generated because the prompt leaves less room than --max-tokens within the model’s length.

engine_refused

Was not generated because the teacher refused the request as invalid.