Operability exit conditions
Candidate exit conditions for the Operability type. See the catalog overview for how to adopt these.
Deployment & rollout strategy
| Exit condition | What it checks |
|---|---|
|
The service deploys via an AAO rolling update with no full-fleet restart or downtime window. |
|
A deploy completes with zero failed requests observed against a continuous synthetic probe. |
|
Rollout advances through a canary stage that is promoted only after health and error thresholds pass. |
|
Traffic cutover between blue and green environments is atomic and reversible within one step. |
|
Rolling update honors configured maxSurge and maxUnavailable so capacity never drops below a floor. |
|
The deployed image is an immutable, content-addressed artifact matching the promoted build digest. |
|
The running service exposes its build version and commit SHA on a discoverable endpoint. |
|
Rollout automatically halts when the new revision’s error rate exceeds the prior revision’s baseline. |
|
Re-applying the same deployment manifest produces no change and no restart. |
Health, readiness & liveness probes
| Exit condition | What it checks |
|---|---|
|
A liveness probe endpoint exists and returns healthy only when the process can make progress. |
|
A readiness probe reports not-ready until dependencies and warmup are complete. |
|
The load balancer withholds traffic from an instance until its readiness probe passes. |
|
A startup probe covers slow initialization so a warming instance is not killed prematurely. |
|
Readiness reflects the reachability of critical downstream dependencies, not just process liveness. |
|
The Akka cluster reports all expected members as Up with no unreachable nodes. |
|
Health probes complete within a bounded timeout and never block on long-running work. |
|
The liveness probe fails when the service is deadlocked or unable to serve, not just when the port is open. |
Structured logging & retention
| Exit condition | What it checks |
|---|---|
|
Application logs are emitted as structured, machine-parseable records rather than free-form text. |
|
Every log line carries a correlation or trace id that ties it to a request or workflow. |
|
Each Akka component emits lifecycle and error logs to the platform log stream. |
|
Log verbosity is configurable at runtime without a redeploy. |
|
Logs contain no credentials, tokens, or PII, verified against a redaction ruleset. |
|
A log retention period is configured and enforced against the retention policy. |
|
Error-level logs include a stack trace or structured cause sufficient to diagnose the failure. |
|
High-volume log sources are sampled or rate-limited so logging cannot exhaust I/O or budget. |
|
Log timestamps are ISO-8601 UTC with millisecond precision for cross-service ordering. |
Metrics & dashboards
| Exit condition | What it checks |
|---|---|
|
Request rate, error rate, and duration are exported for every served endpoint. |
|
Utilization, saturation, and errors are exported for CPU, memory, and thread pools. |
|
Each Akka component emits invocation, latency, and failure metrics to the platform. |
|
Endpoint latency is reported as p50, p95, and p99 rather than an average alone. |
|
A dashboard exists covering the four golden signals for the service’s primary flows. |
|
Metric label cardinality is bounded so the metrics backend cannot be overwhelmed. |
|
Internal queue and mailbox depths are exported to reveal backpressure before saturation. |
|
Exported metrics follow a consistent naming and unit convention across the service. |
|
At least one domain-level success metric is exported alongside infrastructure metrics. |
Distributed tracing
| Exit condition | What it checks |
|---|---|
|
The service exports OpenTelemetry spans to the configured collector. |
|
W3C trace context propagates across service and component boundaries without breaks. |
|
Each Akka component invocation produces a span linked to its parent request trace. |
|
Requests are traceable end-to-end in the Akka console’s request tracking view. |
|
Spans carry attributes for component name, operation, and outcome sufficient to debug failures. |
|
Trace sampling is configurable and preserves error and high-latency traces. |
|
Failed operations set span status to error and attach the exception detail. |
|
A single trace spans calls across multiple services in the request path. |
Alerting & on-call
| Exit condition | What it checks |
|---|---|
|
An alert fires when the endpoint error rate crosses a defined threshold. |
|
An alert fires when latency breaches the SLO target over the evaluation window. |
|
Every alert routes to a named on-call rotation with no unassigned severities. |
|
Each alert links to a runbook describing triage and remediation steps. |
|
Alerts are symptom-based and actionable, with no alert that requires no operator response. |
|
Alerts use hysteresis or a for-duration so transient blips do not page. |
|
An alert fires when a critical downstream dependency becomes unreachable. |
|
Alerts are classified by severity so only user-impacting issues page out of hours. |
SLOs, SLIs & error budgets
| Exit condition | What it checks |
|---|---|
|
Each user-facing flow has a measurable SLI backed by an exported metric. |
|
A numeric SLO target and evaluation window are defined for every SLI. |
|
The remaining error budget is computed and reported against its SLO. |
|
An alert fires when error-budget burn rate threatens exhaustion before the window ends. |
|
Service availability is measured from the consumer’s perspective, not just process uptime. |
|
A latency SLO defines the acceptable percentile and threshold for each critical endpoint. |
|
An SLO dashboard shows current attainment and budget for each objective. |
|
SLO attainment is reported over a defined rolling period for review. |
Configuration & secrets management
| Exit condition | What it checks |
|---|---|
|
All environment-specific configuration is externalized, not baked into the image. |
|
No secret material is present in the container image, source, or manifests. |
|
Secrets are injected at runtime from a managed secret store. |
|
Secrets can be rotated without a code change or full redeploy. |
|
Invalid or missing required configuration fails fast at startup with a clear error. |
|
Configuration defaults are safe for production and never silently enable debug behavior. |
|
Configuration and secret changes are recorded in an auditable change log. |
|
Configuration schema is identical across environments, differing only in values. |
|
Certificate and secret expiry is monitored and alerts before expiration. |
Rollback, graceful shutdown & resilience
| Exit condition | What it checks |
|---|---|
|
The prior known-good revision can be restored in a single operation. |
|
Rollback is safe against the current data and schema, with no forward-only migration blocking it. |
|
On termination the service drains in-flight requests before exiting within the grace period. |
|
The process handles SIGTERM by stopping intake, completing work, and closing resources cleanly. |
|
The load balancer stops routing to a terminating instance before it shuts down. |
|
A restarted instance rejoins the cluster and recovers durable state without data loss. |
|
A failed canary automatically rolls back without manual intervention. |
|
Operations retried after a failed deploy or restart do not produce duplicate side effects. |
Capacity, autoscaling & runtime cost
| Exit condition | What it checks |
|---|---|
|
CPU and memory requests and limits are set for every workload. |
|
An autoscaling policy scales the service on a defined saturation signal. |
|
Autoscaling has minimum and maximum replica bounds to cap both risk and cost. |
|
A load test establishes the per-instance throughput ceiling used to size capacity. |
|
A runtime cost budget is defined for the service and tracked against actuals. |
|
An alert fires when runtime spend deviates materially from its budgeted trend. |
|
The service scales down to a minimum footprint under low load without dropping availability. |
|
Steady-state utilization leaves defined headroom to absorb traffic spikes before autoscaling reacts. |