Skip to content

README.md

Latest commit

 

History

History
258 lines (191 loc) · 10.9 KB

README.md

File metadata and controls

258 lines (191 loc) · 10.9 KB

CloudZero Agent Helm Chart

Contributor Covenant License GitHub release

A Helm chart for deploying Prometheus in agent mode to send cluster metrics to the CloudZero platform.

For the latest release, see Releases. You can also enable release notifications.

Prerequisites

  • Kubernetes 1.23+
  • Helm 3+
  • A CloudZero API key
  • Each Kubernetes cluster must have a route to the internet and a rule that allows egress from the agent to the CloudZero collector endpoint at https://api.cloudzero.com on port 443

Installation

Get Helm Repository Info

helm repo add cloudzero https://cloudzero.github.io/cloudzero-charts
helm repo update

See helm repo for command documentation.

Install Helm Chart

The chart can be installed directly with Helm or any other common Kubernetes deployment tools.

If installing with Helm directly, the following command will install the chart:

helm install <RELEASE_NAME> cloudzero/cloudzero-agent \
    --set existingSecretName=<NAME_OF_SECRET> \
    --set clusterName=<CLUSTER_NAME> \
    --set-string cloudAccountId=<CLOUD_ACCOUNT_ID> \
    --set region=<REGION>

Update Helm Chart

Alternatively, if you are updating an existing installation, pull the latest chart information first:

helm repo update

Next, upgrade the installation to the latest chart version:

helm upgrade <RELEASE_NAME> cloudzero/cloudzero-agent \
    --set existingSecretName=<NAME_OF_SECRET> \
    --set clusterName=<CLUSTER_NAME> \
    --set-string cloudAccountId=<CLOUD_ACCOUNT_ID> \
    --set region=<REGION>

Mandatory Values

There are several mandatory values that must be specified for the chart to install properly. Below are the required settings along with strategies for providing custom values during installation:

Key Type Default Description
cloudAccountId string nil Account ID in AWS or Subscription ID in Azure or Project Number in GCP where the cluster is running. Must be a string due to Helm limitations.
clusterName string nil Name of the cluster. Must be RFC 1123 compliant.
host string "api.cloudzero.com" CloudZero host to send metrics to.
apiKey string nil The CloudZero API key to use for exporting metrics. Only used if existingSecretName is not set.
existingSecretName string nil Name of the secret that contains the CloudZero API key. Required if not providing the API key via apiKey.
region string nil Region where the cluster is running (e.g., us-east-1, eastus). For more information, see AWS or Azure documentation.

Overriding Default Values

Default values are specified in the chart's values.yaml file. If you need to change any of these values, it is recommended to create a values-override.yaml file for your customizations.

Using the --values Flag

You can use the --values (or short form -f) flag in your Helm commands to override values in the chart with a new file. Specify the name of the file after the --values flag:

helm install <RELEASE_NAME> cloudzero/cloudzero-agent \
    --set existingSecretName=<NAME_OF_SECRET> \
    --set clusterName=<CLUSTER_NAME> \
    --set-string cloudAccountId=<CLOUD_ACCOUNT_ID> \
    --set region=<REGION> \
    -f values-override.yaml

Ensure values-override.yaml contains only the values you wish to override from values.yaml.

Note it is possible to save values for different environments, or based on other criteria into seperate values files and multiple files using the -f helm parameters.

Using the --set Flag

You can use the --set flag in Helm commands to directly set or override specific values from values.yaml. Use dot notation to specify nested values:

helm install <RELEASE_NAME> cloudzero/cloudzero-agent \
    --set existingSecretName=<NAME_OF_SECRET> \
    --set clusterName=<CLUSTER_NAME> \
    --set-string cloudAccountId=<CLOUD_ACCOUNT_ID> \
    --set region=<REGION> \
    --set server.resources.limits.memory=2048Mi \
    -f values-override.yaml

Secret Management

The chart requires a CloudZero API key to send metric data. Admins can retrieve API keys here.

The API key can be supplied as an existing secret (default) or created by the chart. Ensure the Secret is in the same namespace as the chart and follows this format:

values-override.yaml

data:
  value: <API_KEY>

Example of creating a secret:

kubectl create secret -n example-namespace generic example-secret-name --from-literal=value=<example-api-key-value>

The secret can then be used with existingSecretName.

Memory Sizing

Please see the sizing guide in the docs directory.

Metric Exporters

This chart depends on metrics from kube-state-metrics and node-exporter projects as subcharts.

By default, these subcharts are disabled to allow scraping from existing instances. Configure the cloudzero-agent to use existing service endpoint addresses in values.yaml:

validator:
  serviceEndpoints:
     kubeStateMetrics: <kube-state-metrics>.<example-namespace>.svc.cluster.local:8080
     prometheusNodeExporter: <node-exporter>.<example-namespace>.svc.cluster.local:9100

Alternatively, deploy them automatically by enabling settings in values-override.yaml:

kube-state-metrics:
  enabled: true
prometheus-node-exporter:
  enabled: true

Passing Values to Subcharts

Values can be passed to subcharts like kube-state-metrics and prometheus-node-exporter by adding entries in values-override.yaml as per their specifications.

A common addition may be to pull the container images from custom image registries/repositories:

values-override.yaml

kube-state-metrics:
  enabled: true
  image:
    registry: my-custom-registry.io
    repository: my-custom-kube-state-metrics/kube-state-metrics

prometheus-node-exporter:
  enabled: true
  image:
    registry: my-custom-registry.io
    repository: my-custom-prometheus/node-exporter

Custom Scrape Configs

If running without the default exporters, adjust Prometheus scrape configs:

values-override.yaml

prometheusConfig:
  scrapeJobs:
    kubeStateMetrics:
      enabled: false # this disables the default kube-state-metrics scrape job, which will be replaced by an entry in additionalScrapeJobs
    additionalScrapeJobs:
    - job_name: custom-kube-state-metrics
      honor_timestamps: true
      scrape_interval: 1m
      scrape_timeout: 10s
      metrics_path: /metrics
      static_configs:
        - targets:
          - 'my-kube-state-metrics-service.default.svc.cluster.local:8080'
          - 'my-node-exporter.default.svc.cluster.local:9100'
      relabel_configs:
      - separator: ;
        regex: __meta_kubernetes_service_label_(.+)
        replacement: $1
        action: labelmap
      - source_labels: [__meta_kubernetes_namespace]
        separator: ;
        regex: (.*)
        target_label: namespace
        replacement: $1
        action: replace
      - source_labels: [__meta_kubernetes_service_name]
        separator: ;
        regex: (.*)
        target_label: service
        replacement: $1
        action: replace
      - source_labels: [__meta_kubernetes_pod_node_name]
        separator: ;
        regex: (.*)
        target_label: node
        replacement: $1
        action: replace

Exporting Pod Labels

Pod labels can be exported as metrics using kube-state-metrics. To customize the labels for export, modify the values-override.yaml file as shown below:

Example: Exporting only the pod labels named foo and bar:

kube-state-metrics:
  extraArgs:
     - --metric-labels-allowlist=pods=[foo,bar]

This is preferable to including all labels with * because the performance and memory impact is reduced. Regular expression matching is not currently supported. See the kube-state-metrics documentation for more details.

⚠️ Important: If you are running an existing kube-state-metrics instance, ensure that the labels you want to use are whitelisted. kube-state-metrics version 2.x and above will not export the kube_pod_labels metrics unless they are explicitly allowed. This prevents the use of those labels for cost allocation and other purposes. Make sure you have configured the labels at the appropriate level using the --metric-labels-allowlist parameter:

eg: - --metric-labels-allowlist=namespaces=[*],pods=[*],deployments=[app.kubernetes.io/*,k8s.*]

Dependencies

Repository Name Version
https://prometheus-community.github.io/helm-charts kube-state-metrics 5.15.*
https://prometheus-community.github.io/helm-charts prometheus-node-exporter 4.24.*

Enabling Release Notifications

To receive a notification when a new version of the chart is released, you can watch the repository:

  1. Navigate to the repository main page.
  2. Select Watch > Custom.
  3. Check the Releases box.
  4. Select Apply.

Useful References