akka optimize training runs metrics

Display metrics recorded for each step of a training run

Synopsis

Display metrics recorded for each step of a training run.

Metrics become available as the run progresses. Supervised runs include loss. Reinforcement runs include the share of groups with no reward spread, the shares of rollouts that scored zero or full, and the reward.

Every run includes values that describe each optimizer step: the gradient norm before clipping, the gradient clip factor, the adapter norm after the step, and the update ratio. The gradient clip factor is the factor used to scale the gradient during clipping. It is 1 when no clipping occurs. The update ratio is the norm of the adapter update divided by the adapter norm before the step.

Use --limit to display only the most recent training steps.

Use --fields to add columns beyond the default columns. Available values include the fields that have the same name for every engine (kl, maxMemoryGb, maxMemoryReservedGb, maxDeviceMemoryGb, scoredRewardMean, trainerIdle, and rolloutIdle), every numeric value logged by the training loop (such as entropy or learning_rate), and each reward term’s mean share. Prefix a reward term with "terms:", as in terms:total. Structured output (-o json or --jq) includes all of these values whether or not --fields names them. Fields shared by every engine are at the top level. The remaining values are under "measured" and "rewardTerms".

akka optimize training runs metrics RUN [flags]

Examples

  # Display all recorded metrics for a run
  akka optimize training runs metrics 7d496e82c3a15f04

  # Display the last 20 steps
  akka optimize training runs metrics 7d496e82c3a15f04 --limit 20

  # Add columns for entropy and a reward term's mean share
  akka optimize training runs metrics 7d496e82c3a15f04 \
    --fields entropy,terms:total

  # Extract step numbers and group variance metrics using jq
  akka optimize training runs metrics 7d496e82c3a15f04 \
    --jq '.[] | [.step, .zeroVarianceGroups]'

Options

      --fields strings   Additional columns from the named and measured figures and reward terms (prefix a term with terms:)
  -h, --help             help for metrics
      --limit int        Limit output to the specified number of recent steps

Options inherited from parent commands

      --cache-file string    location of cache file (default "~/.akka/cache.yaml")
      --config string        location of config file (default "~/.akka/config.yaml")
      --context string       configuration context to use
      --disable-prompt       Disable all interactive prompts when running akka commands. If input is required, defaults will be used, or an error will be raised.
                             This is equivalent to setting the environment variable AKKA_DISABLE_PROMPTS to true.
      --force                Skip the confirmation prompt
      --jq string            Filter the JSON output with a jq expression
      --json string          Output JSON with only these fields; --json= lists the fields a command offers
      --no-color             Print without color, as NO_COLOR does
  -o, --output string        set output format to one of [text,json,json-compact,go-template=] (default "text")
      --page-mode string     the mode for paging, either paged, buffered or auto. (default "auto")
  -q, --quiet                set quiet output (helpful when used as part of a script)
      --region string        Region to use if the project has more than one region
      --timeout duration     client command timeout (default 10s)
      --trainer-url string   Trainer service URL (default TRAINER_URL, then AKKA_OPTIMIZE_URL, then http://localhost:9001)
      --url string           Optimize service URL (default OPTIMIZE_URL, then AKKA_OPTIMIZE_URL, then http://localhost:9010)
      --use-grpc-web         use grpc-web when talking to Akka APIs. This is useful when behind corporate firewalls that decrypt traffic but don't support HTTP/2.
      --verbose              set verbose output

SEE ALSO