akka optimize training endpoints info

Show what an endpoint here can serve

Synopsis

Show what an endpoint of this deployment can serve and what one claims.

"Adapter ranks" is the largest LoRA rank an endpoint loads. An adapter of a lower rank loads too, and a run that trains above it is refused at submission. "Adapters loaded" is how many adapters one endpoint has loaded at once beside its base model. More can attach, and one is loaded when a request first names it. "GPUs per endpoint" and the device type are what one endpoint claims for as long as it answers. The deployment sets each of these, and "training endpoints create" takes none of them. The JSON output also lists the rank limits the serving engine accepts, which an operator chooses the deployment’s limit from.

On the vLLM engine, each architecture of a selected base model is listed with what an endpoint returns for a model of it beside its text: tool calls, where the architecture has a tool-call parser, and reasoning apart from the answer, where it has a reasoning parser.

An endpoint serves a base model and adapters over it. A model registered as whole weights is not served.

akka optimize training endpoints info [flags]

Examples

  # Show what an endpoint here can serve
  akka optimize training endpoints info

  # Read the largest adapter rank using jq
  akka optimize training endpoints info --jq .maxAdapterRank

Options

  -h, --help   help for info

Options inherited from parent commands

      --cache-file string    location of cache file (default "~/.akka/cache.yaml")
      --config string        location of config file (default "~/.akka/config.yaml")
      --context string       configuration context to use
      --disable-prompt       Disable all interactive prompts when running akka commands. If input is required, defaults will be used, or an error will be raised.
                             This is equivalent to setting the environment variable AKKA_DISABLE_PROMPTS to true.
      --force                Skip the confirmation prompt
      --jq string            Filter the JSON output with a jq expression
      --json string          Output JSON with only these fields; --json= lists the fields a command offers
      --no-color             Print without color, as NO_COLOR does
  -o, --output string        set output format to one of [text,json,json-compact,go-template=] (default "text")
      --page-mode string     the mode for paging, either paged, buffered or auto. (default "auto")
  -q, --quiet                set quiet output (helpful when used as part of a script)
      --region string        Region to use if the project has more than one region
      --timeout duration     client command timeout (default 10s)
      --trainer-url string   Trainer service URL (default TRAINER_URL, then AKKA_OPTIMIZE_URL, then http://localhost:9001)
      --url string           Optimize service URL (default OPTIMIZE_URL, then AKKA_OPTIMIZE_URL, then http://localhost:9010)
      --use-grpc-web         use grpc-web when talking to Akka APIs. This is useful when behind corporate firewalls that decrypt traffic but don't support HTTP/2.
      --verbose              set verbose output

SEE ALSO