Inference CLI commands
Model serving commands are under akka models. Routing, cost, training, and evaluation commands are under akka optimize. They act on different resources and are documented on three pages.
| Page | Covers |
|---|---|
|
|
|
|
|
Every command takes --help, and most take -o json.
akka models apply and akka models delete read a YAML model descriptor, and read it from standard input when -f is omitted. The format other commands read with -f FILE depends on the command, and --help states it. Most akka optimize commands read the document from standard input when you pass -f -.
Where each task is described
The command pages state what each command does. The Inference section states when to run it.
-
Serving models walks from an unallocated GPU to a model answering requests.
-
Routing requests to models covers routing on declared rules and on the meaning of a request.
|
The features described in this section are an add-on to Akka Automated Operations. They are not included in the base product. |