akka optimize training endpoints

Serve a base model or a candidate to try it by hand

Synopsis

An endpoint makes a model answerable at an OpenAI-compatible address.

An endpoint is the deployment rather than the model. A serving process holds a bounded number of adapters over one resident copy of its base model, so a candidate whose base model is already served attaches to that endpoint: it answers in seconds and claims no further devices. What tells the models at one address apart is the model name a request asks for.

This is the one thing in the training service that holds GPUs while nothing is happening, which is why an endpoint is a deliberate act and why deleting one is worth doing when you are finished with it. Training and evaluation need no endpoint; they load what they need and release it.

Examples

  # Serve a selected base model
  akka-optimize training endpoints create --model Qwen/Qwen3-8B --workload triage

  # Serve a candidate, which attaches to its base model's endpoint
  akka-optimize training endpoints create --model triage-candidate

  # Wait for one that is already coming up
  akka-optimize training endpoints wait 970076e8

  # Send it one input
  akka-optimize training endpoints send 970076e8 --input "Is this ticket a billing problem?"

  # Stop answering for one model
  akka-optimize training endpoints detach 970076e8 triage-candidate

Options

  -h, --help   help for endpoints

Options inherited from parent commands

      --cache-file string    location of cache file (default "~/.akka/cache.yaml")
      --config string        location of config file (default "~/.akka/config.yaml")
      --context string       configuration context to use
      --disable-prompt       Disable all interactive prompts when running akka commands. If input is required, defaults will be used, or an error will be raised.
                             This is equivalent to setting the environment variable AKKA_DISABLE_PROMPTS to true.
      --force                Skip the confirmation prompt
      --jq string            Filter the JSON output with a jq expression
      --json string          Output JSON with only these fields; --json= lists the fields a command offers
      --no-color             Print without color, as NO_COLOR does
  -o, --output string        set output format to one of [text,json,json-compact,go-template=] (default "text")
      --page-mode string     the mode for paging, either paged, buffered or auto. (default "auto")
  -q, --quiet                set quiet output (helpful when used as part of a script)
      --region string        Region to use if the project has more than one region
      --timeout duration     client command timeout (default 10s)
      --trainer-url string   Trainer service URL (default http://localhost:9001, or TRAINER_URL)
      --url string           Optimize service URL (default http://localhost:9010, or OPTIMIZE_URL)
      --use-grpc-web         use grpc-web when talking to Akka APIs. This is useful when behind corporate firewalls that decrypt traffic but don't support HTTP/2.
      --verbose              set verbose output

SEE ALSO