Cost, training, and evaluation commands
These commands cover what an organization spends on inference and how models are trained and scored. Every command on this page is a subcommand of akka optimize. This page lists the commands most tasks use. See akka optimize for every command and flag.
The format a -f FILE option reads depends on the command, and --help states it. Training plans are JSON or YAML. eval datasets create reads a zip or an items.jsonl file. datasets create reads chat-format JSONL. training sessions create reads a conversation corpus. Most commands read the document from standard input when you pass -f -.
Spend
akka optimize spend reports what a period consumed, what it cost, and what one conversation cost. The alias spends is accepted.
| Command | Description |
|---|---|
|
What a period consumed and cost, divided as asked. |
|
Two periods, and the difference between them. |
|
The conversations of a period that cost the most. |
|
Which service is answering at this address. |
Callers
akka optimize callers holds the callers that spend figures are attributed to. The alias caller is accepted.
| Command | Description |
|---|---|
|
Remove one caller and keep every figure it is behind. |
|
Report what the erasure would reach without writing anything. |
Incidents
akka optimize incidents is the incident feed, unioned from its sources. The alias incident is accepted.
| Command | Description |
|---|---|
|
The incident feed: anomalies, blocked requests and scrub failures. |
Pricing catalog
akka optimize pricing-catalog holds model prices as input and output rates per million tokens for each served name. A published version is never edited. The aliases pricing-catalogs, catalog, and catalogs are accepted.
| Command | Description |
|---|---|
|
The version in force now, or at an instant. |
|
Publish a catalog version, or check one. |
|
Every published version, newest first. |
Training runs
akka optimize training runs manages training runs. The alias run is accepted.
| Command | Description |
|---|---|
|
Start a training run from a plan file. |
|
List training runs. |
|
Show one run. |
|
Wait for a run to finish. |
|
Cancel a run and free its GPUs. |
Training pipelines
akka optimize training pipelines manages multi-stage training pipelines. The alias pipeline is accepted.
| Command | Description |
|---|---|
|
Start a pipeline from a plan file. |
|
List pipelines and the stage each is on. |
|
Show a pipeline, stage by stage. |
|
Wait for every stage to finish. |
|
Cancel a pipeline and its running stage. |
Trained models
akka optimize trained-models manages the models training produced. The alias trained-model is accepted.
| Command | Description |
|---|---|
|
List models and what they scored. |
|
Show one model’s lineage, cost, and score. |
Training evaluations
akka optimize training evaluations manages evaluations of trained models. The alias evaluation is accepted.
| Command | Description |
|---|---|
|
Score a trained model against a dataset. |
|
List evaluations and what they scored. |
|
Show what an evaluation scored and where it got to. |
|
Show how a model scored, field by field. |
|
Show what the model answered, example by example. |
|
Follow an evaluation until it finishes. |
|
Stop an evaluation and release what it holds. |
Training datasets
akka optimize datasets manages training datasets. The alias dataset is accepted.
| Command | Description |
|---|---|
|
Register chat-format JSONL examples. The alias |
|
List registered datasets, newest first. |
|
Show what a dataset holds. |
|
Point a name at a registered dataset. |
|
Upload images, skipping the ones already registered. |
Sessions and curations
A session is a raw conversation corpus. A curation judges raw sessions into a training dataset.
| Command | Description |
|---|---|
|
Upload a raw conversation corpus. |
|
Start a curation run. |
|
Wait for a curation to finish. |
|
Phase, counts, mean score and resulting dataset. |
Evaluation
akka optimize eval covers datasets, experiments and the evaluators that score them.
| Command | Description |
|---|---|
|
Upload a dataset file and version it. |
|
Datasets that have been imported, most recently imported first. |
|
One dataset: what it holds, what scores it, and every version it has had. |
|
The items in one version of a dataset. |
|
How far an import has got. |
|
Start an experiment. |
|
How far an experiment has got, and what it has found. |
|
The finished record: the run’s report, or the numbers behind it. |
|
One item’s run, and what each evaluator said about it. |
|
An experiment against a baseline, item by item. |
|
Every run of a recurring experiment, most recent first. |
|
Registered evaluators, their kind and their version. |
Captures
akka optimize captures reads recorded interaction artifacts off a filesystem. The alias capture is accepted.
These commands serve debugging and diagnostics. They read artifacts from a directory on disk, which is where a capture written during local development or a support investigation lands.
|
Captures recorded in production are stored in a location the CLI cannot reach in this version. The commands below read a filesystem directory only. |
| Command | Description |
|---|---|
|
Every artifact in a directory, newest first. |
|
One artifact, read forward. |