Model Descriptor reference

Feature set: Inference Contact our support for access.

The model descriptor is the YAML applied by akka models apply to a project’s region. It declares the accelerator allocations that the project claims, the models that run on them, and the endpoints that serve those models. See Serving models for the tasks these resources support.

To check a descriptor against the region without persisting anything, add --dry-run:

akka models apply -f models.yaml --dry-run

The CLI checks the shape of the descriptor before it sends anything and reports every problem at once. The region then decides whether the descriptor is admitted, including whether the requested devices fit the pool.

Document envelope

A descriptor is a multi-document YAML file. Documents are separated by a line starting with ---, and every document carries the same four top-level fields.

Field Type Description

resource

string required

One of AcceleratorAllocation, Model, ModelEndpoint.

resourceVersion

string required

Must be v1.

metadata.name

string required

Resource name. See Naming rules.

spec

object required

Shape depends on resource.

Specs are parsed strictly. An unrecognized field is an error rather than a value that is silently ignored, so a mistyped field name fails the file instead of quietly changing behavior.

A descriptor cannot contain a Service, Route, or Secret. Those resources belong to the project descriptor. Create them with akka projects apply, akka routes, or akka secrets.

Naming rules

metadata.name and the names of allocation entries must be valid DNS labels:

  • lowercase alphanumeric characters and dashes, matching [a-z0-9]([-a-z0-9]*[a-z0-9])?

  • 63 characters maximum

The platform derives object names from yours by adding suffixes, such as -engine for the service that a model answers on, and the derived names have to fit the same limit.

Apply order

Applying a descriptor writes resources in dependency order, so nothing refers to an object that does not exist yet:

  1. Accelerator allocations

  2. Models

  3. Model endpoints

akka models apply creates or updates each resource by name, and prints whether it was created, updated, or unchanged. akka models delete runs in the reverse order.

AcceleratorAllocation

An accelerator allocation partitions the region’s finite accelerator capacity to your project. The region declares which pools exist, you allocate devices from a pool, and the region fits models within the allocation.

resource: AcceleratorAllocation
resourceVersion: v1
metadata:
  name: tutorial
spec:
  allocations:
    - name: gpu
      acceleratorPool: POOL_NAME
      devices: 1

Each allocated device becomes one standing claim, named from the entry name and the device index. A claim is allocated to exactly one project, so projects cannot share a device.

Field Type Description

spec.allocations

list required

The device counts that your project holds, one entry per allocation name. At least one entry. Two entries can draw on the same pool under different names.

spec.allocations[].name

string required

The name of the entry, and what a model’s placement.allocation refers to. Keep it the same in every region, so that one descriptor deploys everywhere. Unique across every allocation in the descriptor.

spec.allocations[].acceleratorPool

string required

A pool that the region declares, for example l4. A pool name is a regional identifier, so the same hardware can have another name in another region. akka models pools list shows the pools.

spec.allocations[].devices

integer required

The number of whole devices from the pool that are allocated to the project. The minimum is 1.

Model

A model is one checkpoint served by an engine on a device of an allocation.

resource: Model
resourceVersion: v1
metadata:
  name: assistant
spec:
  model: mistralai/Ministral-3-3B-Instruct-2512
  source:
    type: huggingface
    huggingface:
      secretRef: hf-token
      revision: COMMIT_SHA
  engine:
    type: vllm
    vllm:
      checkpointFormat: mistral
      toolCalling:
        parser: mistral
  placement:
    allocation: gpu
    device: 0
  serving:
    contextLength: 8192

The schema only grows. The region refuses a field that it does not implement yet.

Field Type Description

spec.model

string required

The checkpoint to serve, passed to the model server as it is written, for example google/gemma-3-4b-it.

spec.task

string

What the engine serves: generate or embed. The default is generate.

spec.source

object required

Where the weights come from. See spec.source.

spec.engine

object required

The model server. See spec.engine.

spec.placement

object required

The device that the model runs on. See spec.placement.

spec.serving

object

The serving limits. See spec.serving.

spec.loraAdapters

list

LoRA adapters served next to the base model. See spec.loraAdapters.

spec.replicas

integer

The number of engine replicas. The only accepted value is 1, which is also the default.

spec.workloadIdentity

object

The cloud identity that the engine pod runs as, set on the model’s own service account as it is for a service. An adapter fetched from a bucket with native credentials authenticates as this identity. The akka member is not supported on a model.

spec.source

Field Type Description

type

string required

The source provider. Only huggingface is supported.

huggingface

object required when type is huggingface

The Hugging Face source.

huggingface.revision

string required

The full 40-character commit SHA, in lowercase hexadecimal, that the weights are pinned to. A branch or tag name is refused, because it moves under a running model. Requests cannot override the revision.

huggingface.secretRef

string

The name of the project secret that holds the token for gated checkpoints. The secret stores the token under the key token.

cache

object

Persist the weights across restarts.

cache.sizeGiB

integer required with cache

The size of the cache volume in GiB. The minimum is 1.

spec.engine

Field Type Description

type

string required

The model server. Only vllm is supported. It cannot change for the lifetime of the model.

vllm

object required when type is vllm

The vLLM settings. Every field is optional, and a field that is absent or auto keeps the engine default.

vllm.dtype

string

The data type of the weights and activations: auto, bfloat16, float16, or float32.

vllm.quantization

string

The weight quantization method: fp8, awq, awq_marlin, gptq, gptq_marlin, compressed-tensors, bitsandbytes, modelopt, or gguf. Leave it out to let the engine detect a pre-quantized checkpoint.

vllm.kvCacheDtype

string

The data type of the KV cache: auto, fp8, fp8_e4m3, or fp8_e5m2.

vllm.checkpointFormat

string

The checkpoint layout: hf or mistral.

vllm.chatTemplateRef

string

The name of the project secret whose template key holds the chat template, for fine-tuned checkpoints.

vllm.toolCalling.parser

string

Enables OpenAI tool calls with the named parser. One of hermes, mistral, llama3_json, llama4_json, llama4_pythonic, pythonic, granite, granite-20b-fc, internlm, jamba, deepseek_v3, xlam, qwen3_coder, kimi_k2, glm45, phi4_mini_json, or gemma4.

vllm.extraArgs

list of strings

Arguments appended to the engine command line, last and verbatim. The region refuses the flags that the platform owns: the model, port, served name, memory fraction, adapters, and parallelism.

spec.placement

Field Type Description

allocation

string required

The name of an entry in your project’s AcceleratorAllocation. It is your own name for the allocation and is the same in every region.

device

integer

The index of the device within that entry that the model runs on. The minimum and default are 0. Models that give the same index share the physical device.

memoryGiB

integer

The engine’s device-memory budget on that device, in GiB. The minimum is 1. When absent, the model takes the whole device less the type’s headroom.

spec.serving

Field Type Description

contextLength

integer

The maximum number of tokens per sequence, prompt and completion together. The device-memory budget must hold at least one sequence of this length, or the engine refuses to start. When absent, the engine uses the checkpoint’s own maximum. The minimum is 1.

maxConcurrentRequests

integer

The maximum number of requests scheduled at once. The minimum is 1.

resources

object

The CPU and memory of the engine container, applied as both request and limit. When absent, the region default applies. The device is never listed here.

spec.loraAdapters

Adapters are static. Changing the set restarts the engine.

Field Type Description

name

string required

The name that clients send in the model field to reach the adapter. Unique across the project’s routing table.

model

string required

The adapter checkpoint, for example acme/scorecard-adapter. For an adapter in object storage it only describes the adapter.

rank

integer required

The LoRA rank of the adapter. The minimum is 1. The engine computes its maximum rank from the declared ranks.

source.type

string required

huggingface or objectStorage.

source.huggingface

object

For huggingface: revision and secretRef, as for the base model.

source.objectStorage.bucket

string required

For objectStorage: the logical name of a bucket of the project, not the physical bucket name. 1 to 42 characters, matching the naming rules.

source.objectStorage.path

string required

For objectStorage: the object key prefix of the adapter directory. It holds adapter_config.json and adapter_model.safetensors. 1 to 1024 characters.

source.objectStorage.sha256

string required

For objectStorage: the SHA-256 of adapter_model.safetensors as 64 lowercase hexadecimal characters. The fetch fails when the weights do not match.

ModelEndpoint

A model endpoint serves one or more models behind one address, and clients choose between them with the OpenAI model field. The endpoint knows nothing about public hostnames. Internet exposure is a project route whose backend is the endpoint.

resource: ModelEndpoint
resourceVersion: v1
metadata:
  name: assistant-endpoint
spec:
  models:
    - name: assistant
  apiKeySecret: assistant-keys
Field Type Description

spec.models

list required

The models that the endpoint serves. At least one.

spec.models[].name

string required

The name of a Model in the project.

spec.apiKeySecret

string required

The name of the project secret whose entries are the accepted API keys. The endpoint is reachable by anything that can reach the gateway, so it must never be silently open. akka models endpoints keys add fills the secret.

A project can serve at most 64 endpoints. The region refuses the endpoint that would exceed that number.

Cross-document rules

Checked by the CLI across the whole file:

  • metadata.name must be present and must not repeat within a resource kind.

  • Allocation entry names must not repeat across every allocation in the descriptor.

  • Every allocation entry has an acceleratorPool and at least one device.

  • A model has spec.model, spec.source.type, a pinned revision in spec.source, spec.engine.type, and spec.placement.allocation.

  • An endpoint lists at least one model and names an apiKeySecret.

What the region checks

The CLI does not check anything that depends on the region. When you apply, the region admits or refuses the change. It checks, among other things:

  • Whether the devices that an allocation asks for fit the pool, and whether the allocation exceeds what the project can draw.

  • Whether a model sets only values that the region accepts. A model runs one replica, does not use the akka workload identity, and does not pass flags that the platform owns in extraArgs.

  • Whether adapter names are unique across the project’s routing table.

  • Whether the fields of a model are ones that the region implements.

Use --dry-run to run these checks without persisting anything.

Complete example

resource: AcceleratorAllocation
resourceVersion: v1
metadata:
  name: tutorial
spec:
  allocations:
    - name: gpu
      acceleratorPool: POOL_NAME
      devices: 1
---
resource: Model
resourceVersion: v1
metadata:
  name: assistant
spec:
  model: mistralai/Ministral-3-3B-Instruct-2512
  source:
    type: huggingface
    huggingface:
      secretRef: hf-token
      revision: COMMIT_SHA
  engine:
    type: vllm
    vllm:
      checkpointFormat: mistral
      toolCalling:
        parser: mistral
  placement:
    allocation: gpu
    device: 0
  serving:
    contextLength: 8192
---
resource: ModelEndpoint
resourceVersion: v1
metadata:
  name: assistant-endpoint
spec:
  models:
    - name: assistant
  apiKeySecret: assistant-keys

Resources that belong elsewhere

Accelerator pools cannot be created from a descriptor. They are regional inventory. See Reviewing available hardware.

The features described in this section are an add-on to Akka Automated Operations. They are not included in the base product.

See also