Production readiness checklist for served and trained models

This checklist covers the items that are specific to served and trained models: serving models with Inference and training models with the Trainer. It adds to the general readiness areas and to the production readiness checklist for Dedicated, BYOC, and BYOK8s, which also applies.

The features described in this section are an add-on to Akka Automated Operations. They are not included in the base product.

Use this page as a guideline: review it with your Akka team before you send production traffic to a served model, and add items for your own request flows and failure modes.

At a glance, what we ask of you

  1. Serve production traffic from a model endpoint. Trainer endpoints are for testing.

  2. Manage API keys per client. Issue one key for each client, and rehearse rotation and revocation.

  3. Keep credentials out of descriptors. Store them in project secrets and pin each model revision to a commit.

  4. Check capacity before you apply. Dry-run the descriptor and confirm the devices are bound.

  5. Save training checkpoints and plan the resume. A resumed run needs the same device count.

  6. Rehearse removal. Know which resources and secrets remain after you delete a deployment.

Serving endpoint

Role Responsibility

Customer

Chooses the endpoint type for each workload, deletes test endpoints when testing ends, and provides a custom TLS certificate where it uses one.

Akka

Provisions the TLS certificate for the hostname on the route, unless you provide a custom certificate. See TLS certificates.
Detailed checklist

Endpoint type

  • Production traffic goes to a ModelEndpoint that a project route exposes on a hostname, not to a Trainer endpoint

  • Trainer endpoints are used for model testing only

  • Trainer endpoints are deleted when testing ends, because a deleted endpoint stops and releases its devices

Hostname

  • A route sends the production hostname to the model endpoint, because the endpoint is not reachable without one

  • A client request over TLS to the hostname succeeds before go-live

API keys

Role Responsibility

Customer

Issues, stores, rotates, and revokes the keys. Owns the key secret.

Akka

Enforces the keys on the endpoint. The endpoint refuses every request until the secret holds a key.
Detailed checklist

Issuing

  • One key issued per client, so one key can be revoked without affecting the others

  • Each key stored in your own secret manager when it is issued, because the CLI prints it once and cannot read it back

  • A request without a key and a request with an invalid key are both refused

Rotation and revocation

  • Rotation rehearsed: add a key under a new client name, move the client to it, then remove the old key

  • An owner named for removing keys, because the endpoint accepts every key in the secret until the key is removed

  • Revocation of a lost key rehearsed with akka models endpoints keys remove

Credentials and descriptors

Role Responsibility

Customer

Keeps credentials out of descriptors and version control, and reviews descriptors before they are applied.

Akka

Validates the shape of the descriptor and the model revision when you apply it.
Detailed checklist

Secrets

  • The Hugging Face token is stored in a project secret and the descriptor refers to it by name

  • No descriptor in version control contains a credential

  • The Hugging Face source revision of each model (spec.source.huggingface.revision) pinned to the full 40-character commit SHA, because the region refuses a branch or tag name

Capacity

Role Responsibility

Customer

Sizes the allocation, sets the context length, and tracks the GPU quota of the training project.

Akka

The region admits or refuses an allocation against the pool.
Detailed checklist

Before you apply

  • akka models apply --dry-run run against the production region, so the region checks that the requested devices fit the pool

  • serving.contextLength set for every model, because the checkpoint maximum can exceed the device memory

  • The number of devices your project can draw confirmed with your Akka team, because the region decides this when it admits the allocation

After you apply

  • Each allocation shows Ready as True and the DEVICES and BOUND values agree

  • The model reports ready with akka models get --wait

  • The GPU quota of the training project checked before you create a Trainer endpoint, because the endpoint’s devices count against it and the trainer rejects a request that would exceed it

  • A plan for a Trainer endpoint in AWAITING_CAPACITY, where it is queued until devices free up

Training run recovery

Role Responsibility

Customer

Sets the checkpoint interval, keeps the device count available for a resume, and decides when to resume.

Akka

Lists only committed snapshots in the catalog.
Detailed checklist

Checkpoints

  • execution.saveEvery set for every run, because a run without it saves a snapshot only at the end

  • The pause policy chosen for each run, because a latest pause stops at the newest committed checkpoint and the resumed run repeats the updates after it

Resume

  • The same number of devices as the original allocation is available if you need to resume

  • A resume rehearsed from a committed snapshot, because an interrupted upload cannot be restored

  • A plan for a run that made progress without restorable state, because that run fails instead of restarting from the base model

Removing a deployment

Role Responsibility

Customer

Removes the route, resources, keys, and secrets in the right order.

Akka

Deletes the endpoint, then the model, then the allocation, when you run akka models delete.
Detailed checklist

  • The route deleted before the resources, with akka routes delete

  • The key secret deleted or its keys removed, because akka models delete does not delete secrets and the keys stay valid for any endpoint that names the same secret

See also