View & Query Metrics

How to query and view metrics in GUI and CLI

Notebooks, Experiments & Deployment Metrics

Web GUI

Deployments

Deployment metrics are a Gradient Private Cluster feature. Contact Sales for inquiries!

To view Deployment metrics, navigate to the Metrics tab of the individual deployment.

Notebooks

Notebook metrics are a Gradient Private Cluster feature. Contact Sales for inquiries!

To view Notebook metrics, you will see a minimal version in the Notebook header and you can also open a more robust display with detailed information.

Detailed view

Collapsed view in Notebook header

Workflows

Coming soon!

CLI

To query the metrics for a given workload using the CLI, here's an example using Experiments (the Deployments and Notebooks syntax are the same).

Usage: gradient experiments metrics [OPTIONS] COMMAND [ARGS]...

  Read experiment metrics

Options:
  --help  Show this message and exit.

Commands:
  get     Get experiment metrics
  stream  Watch live experiment metrics
  

Get a single value for a given metric:

Syntax

gradient experiments metrics get --id <experiment id> 

Parameters

Usage: gradient experiments metrics get [OPTIONS]

  Get experiment metrics. Shows CPU and RAM usage by default

Options:
  --id TEXT                       ID of the experiment  [required]
  --metric [cpuPercentage|memoryUsage|gpuMemoryFree|gpuMemoryUsed|gpuPowerDraw|gpuTemp|gpuUtilization|gpuMemoryUtilization]
                                  One or more metrics that you want to read.
                                  Defaults to cpuPercentage and memoryUsage
  --interval TEXT                 Interval
  --start [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%d %H:%M:%S]
                                  Timestamp of first time series metric to
                                  collect
  --end [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%d %H:%M:%S]
                                  Timestamp of last time series metric to
                                  collect
  --apiKey TEXT                   API key to use this time only
  --optionsFile PATH              Path to YAML file with predefined options
  --createOptionsFile PATH        Generate template options file
  --help                          Show this message and exit.
  

Example command to get CPU usage in %:

gradient experiments metrics get --id <experiment id> --metric cpuPercentage 

Stream metrics:

Syntax

gradient experiments metrics stream --id <experiment id>

Parameters

Usage: gradient experiments metrics stream [OPTIONS]

  Get experiment metrics. Shows CPU and RAM usage by default

Options:
  --id TEXT                       ID of the experiment  [required]
  --metric [cpuPercentage|memoryUsage|gpuMemoryFree|gpuMemoryUsed|gpuPowerDraw|gpuTemp|gpuUtilization|gpuMemoryUtilization]
                                  One or more metrics that you want to read.
                                  Defaults to cpuPercentage and memoryUsage
  --interval TEXT                 Interval
  --start [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%d %H:%M:%S]
                                  Timestamp of first time series metric to
                                  collect
  --end [%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%d %H:%M:%S]
                                  Timestamp of last time series metric to
                                  collect
  --apiKey TEXT                   API key to use this time only
  --optionsFile PATH              Path to YAML file with predefined options
  --createOptionsFile PATH        Generate template options file
  --help                          Show this message and exit.