Inference CLI commands

Feature set: Inference Contact our support for access.
This functionality evolves quickly, the behavior and APIs might change between releases without further notice.

Inference commands are top-level akka commands. They act on different resources and are documented on three pages.

Page Covers

Page	Covers
Model commands	`akka models`: deployments, the devices and accelerators they run on, the hostnames that expose them, the API keys that authorize requests, and the governance record.
Routing commands	`akka rulesets`, `akka routing-classifiers`, `akka use-cases`, and `akka routing-policies`: choosing which provider serves a request.
Cost, training, and evaluation commands	`akka budget`, `akka training-runs`, `akka eval`, and the rest of what an organization spends and scores.

Model commands

akka models: deployments, the devices and accelerators they run on, the hostnames that expose them, the API keys that authorize requests, and the governance record.

Routing commands

akka rulesets, akka routing-classifiers, akka use-cases, and akka routing-policies: choosing which provider serves a request.

Cost, training, and evaluation commands

akka budget, akka training-runs, akka eval, and the rest of what an organization spends and scores.

Every command takes --help, and most take -o json.

Files passed with -f FILE are YAML for akka models and JSON elsewhere. Passing -f - reads the document from standard input.

Where each task is described

The command pages state what each command does. The Inference section states when to run it.

Serving models walks from an unallocated GPU to a model answering requests.
Routing requests to models covers routing on declared rules and on the meaning of a request.

The features described in this section are an add-on to Akka Automated Operations. They are not included in the base product.

Inference CLI commands

Where each task is described

See also