Prerequisites
The agent works with read-only access by default: it has no access to Secrets or ConfigMaps. It acts only in the actions mode of the scheduler or with the auto-resize, which you enable, with dedicated permissions, and each action goes to the journal.
You need:
- Kubernetes 1.22 or later.
- kubectl with cluster-admin rights, because the chart creates a ClusterRole.
- Helm 3.14 or later.
- Outbound HTTPS from the cluster to the Onicost endpoint, directly or through a proxy.
- The agent token of the cluster, shown in the Onicost interface when you add the cluster. Each cluster has its own.
Store the token and install the chart
The token is read without being displayed, then goes into a Secret of the namespace. It appears neither on a command line nor in the Helm values, and the chart accepts no token as a value.
In the last command, replace <version> with the latest version in the changelog, 0.6.0 when this page was written. The Onicost interface gives the same command with the version filled in. The endpoint value is required: it is the base URL of the ingestion gateway, without a trailing slash.
Run these commands in order, in the same shell:
- Create the namespace: kubectl create namespace onicost
- Enter the token: printf "Agent token: " && read -rs ONICOST_TOKEN && echo
- Store it in a Secret: printf "%s" "$ONICOST_TOKEN" | kubectl create secret generic onicost-agent-token --namespace onicost --from-file=token=/dev/stdin
- Install the chart: helm upgrade --install onicost-agent oci://ghcr.io/onicost/charts/onicost-agent --version <version> --namespace onicost --set endpoint=https://ingest.onicost.com
Check the agent
The pod is Running and ready (1/1) within a minute. At startup, the log says whether the API server and the endpoint are reachable, and names any blocked flow.
Start with the first two commands, then use the cases below if something is wrong:
- List the pods: kubectl --namespace onicost get pods
- Read the logs: kubectl --namespace onicost logs deployment/onicost-agent
- Pod stuck in ContainerCreating: the token Secret is missing. Check with kubectl --namespace onicost get secret onicost-agent-token
- Pod running but never ready: the agent cannot reach the API server, usually because of a default-deny NetworkPolicy. Look for kubernetes api unreachable in the logs, then enable the network policy below.
- No data in Onicost: look for gateway unreachable or a 401 in the logs. Check that the Secret holds the current token, then check the endpoint and the proxy with helm get values onicost-agent --namespace onicost
Proxy, network policy and kubelets
Add these options to the install or upgrade command.
- Outbound proxy. It is used only to reach the endpoint. Credentials in the URL show in clear text in the pod spec, so prefer a proxy that needs none. Option: --set proxy.url=http://proxy.internal:3128
- Network policy. If the namespace denies traffic by default, the agent needs it to become ready. It allows egress to the cluster DNS and to TCP 443, 6443 and 10250, for the endpoint, the API server and the kubelets. Option: --set networkPolicy.enabled=true
- Other ports: add the proxy port or the API server port to networkPolicy.egressPorts. Find the API server port with kubectl get endpointslices -n default -l kubernetes.io/service-name=kubernetes
- Other DNS, such as NodeLocal DNSCache: --set networkPolicy.dns.namespaceSelector=null --set networkPolicy.dns.podSelector=null allows port 53 to any destination.
- Destination allow-list: networkPolicy.egressCidrs must cover the API server, the node addresses and the endpoint. Leave it empty with Cilium, which does not match node addresses with ipBlock by default.
- Kubelet certificates: on some managed clusters, Scaleway Kapsule for one, the agent cannot verify them. --set kubelet.apiServerProxy=true goes through the API server and grants get on nodes/proxy, which opens the whole kubelet API of every node, pod logs included. Use it only when the logs show errorClass: tls_verification.
Upgrade and uninstall
Read the changelog before each upgrade and always pass a version. The --reset-then-reuse-values option starts from the defaults of the new chart version and keeps your options. To drop an option, set it back to its default, for example with --set proxy.url= in the command.
In actions mode, pause or delete the scheduler rules in Onicost and wait for the workloads to come back on before you uninstall. Uninstalling also deletes the ClusterRole and the ClusterRoleBinding. Deleting the namespace deletes the token Secret.
- Upgrade: helm upgrade onicost-agent oci://ghcr.io/onicost/charts/onicost-agent --version <version> --namespace onicost --reset-then-reuse-values
- See what runs: helm list --namespace onicost
- Uninstall: helm uninstall onicost-agent --namespace onicost
- Delete the namespace: kubectl delete namespace onicost
What the agent sends and what you control
The agent sends the inventory (nodes, namespaces, workloads with their labels, requests and limits, pods, volumes) and usage (CPU and memory of each container, throttling, OOM kills, network, volumes), aggregated per minute. It does not collect Secrets, ConfigMaps, logs or application data. Your data stays in France, with Scaleway, in the Paris region, for 13 months.
To send less, exclude.namespaces drops whole namespaces and exclude.labelKeys removes label keys. Keep the GPU labels and eks.amazonaws.com/compute-type: Onicost prices GPU and Fargate nodes from them. For the scheduler, add --set actions.enabled=true (chart 0.5.0 or later). It grants write permissions on workloads, so read the security page first.
The 14-day Team trial needs no card and starts with the agent’s first data, no later than 30 days after your organization is created.
Frequently asked questions
How do I rotate the token?
Write the new token to the same Secret, for example by adding --dry-run=client -o yaml | kubectl apply -f - to the kubectl create secret command. The agent picks it up within two minutes, without a restart.
How do I monitor the agent?
Port 8080 serves /healthz, /readyz and /metrics in the Prometheus format. The chart creates no Service. With the network policy on, allow Prometheus with networkPolicy.metricsFrom.