Akka Optimize
Akka Optimize specializes and serves small language models for tasks in your production traffic. Use it to move suitable tasks from a frontier model to a smaller model.
The cost of a task is the price per token multiplied by the number of tokens. A frontier API fixes both values. A small language model trained for one task can lower the price per token because it runs on rented GPUs. It can also use fewer tokens because it needs less prompt context. Akka Optimize finds suitable workloads, trains a model, routes traffic to it, and captures the served requests for the next round of training.
A model trained on one task is expected to do that task as well as the frontier model, or better. An evaluation gate decides whether it does.
The loops
The system is a training loop and a serving loop that run and scale independently. The following table lists their stages:
| Loop | Stage | Description |
|---|---|---|
Training |
Target and admit |
Scores agent traffic to find tasks that could run on a smaller model, and admits suitable workloads for training. |
Training |
Training run |
Trains a model for an admitted workload, evaluates the model, and registers it as a candidate. For more information, see Trainer. |
Serving |
Route and serve |
Routes traffic between the frontier model and the specialized model according to a policy, and captures the served requests. |
Where to start
-
Trainer describes the training service and how to drive it with the
akkaCLI.