Registering a Cluster

A cluster must be registered with kubenest before you can deploy applications to it. Registration is always a two-part process: creating a cluster record in the backend (which issues a cluster JWT), and installing the operator on the actual Kubernetes cluster (which uses that JWT to authenticate with the hub). How the Kubernetes cluster itself comes to exist is what separates the two registration paths.

Path 1: Manual registration (BYO cluster)

Bring-your-own-cluster (BYOC) registration is for clusters that already exist — whether that is a managed service (EKS, GKE, AKE), a self-managed kubeadm cluster, or a local k3s instance. kubenest does not need access to the cloud provider; you install the operator on the cluster yourself.

ORG_ID=$(curl -s -H "Authorization: Bearer $TOKEN" \
  https://api.your-domain.com/api/v1/orgs | jq -r '.[0].id')

curl -X POST "https://api.your-domain.com/api/v1/orgs/$ORG_ID/clusters" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "prod-cluster",
    "description": "AWS EKS us-east-1 production cluster"
  }'

{
  "id": "a1b2c3d4-0001-0000-0000-000000000000",
  "name": "prod-cluster",
  "status": "pending",
  "connection_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "install_command": "helm install kubenest-operator kubenesthq/kubenest-operator -n kubenest-system --create-namespace --set operator.hub.url=wss://hub.your-domain.com --set operator.hub.token=eyJhbGc... --set operator.cluster.id=a1b2c3d4-..."
}

CLUSTER_ID="a1b2c3d4-0001-0000-0000-000000000000"

curl -H "Authorization: Bearer $TOKEN" \
  "https://api.your-domain.com/api/v1/clusters/$CLUSTER_ID/install-command"

operator:
  hub:
    url: wss://hub.your-domain.com
    token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...   # from cluster record
  cluster:
    id: a1b2c3d4-0001-0000-0000-000000000000
 
gitops:
  repoUrl: https://github.com/your-org/gitops-repo
  branch: main
  token: ghp_...     # personal access token with repo:write scope
 
argocd:
  enabled: false     # assumes ArgoCD is already installed
  namespace: argocd

helm repo add kubenesthq https://charts.kubenestapp.com
helm repo update
 
helm install kubenest-operator kubenesthq/kubenest-operator \
  --namespace kubenest-system \
  --create-namespace \
  --values kubenest-operator-values.yaml

kubectl get pods -n kubenest-system -w

NAME                                        READY   STATUS    RESTARTS   AGE
kubenest-operator-controller-manager-xyz   1/1     Running   0          30s

kubectl logs -n kubenest-system \
  -l app.kubernetes.io/name=kubenest-operator \
  --tail=30

{"level":"info","msg":"connected to hub","cluster_id":"a1b2c3d4-...","hub":"wss://hub.your-domain.com"}

curl -H "Authorization: Bearer $TOKEN" \
  "https://api.your-domain.com/api/v1/clusters/$CLUSTER_ID" | jq .status
# "connected"

Path 2: Cloud-provisioned clusters (AWS EC2)

kubenest can provision a Kubernetes cluster on AWS by creating EC2 instances and bootstrapping Kubernetes automatically. This path is appropriate when you do not have an existing cluster and want kubenest to manage the infrastructure lifecycle as well as the application lifecycle.

Prerequisites

1. Create a CloudCredential record with your AWS credentials:

Credential fields are flat, not nested — the payload rejects unknown fields.

curl -X POST "https://api.your-domain.com/api/v1/orgs/$ORG_ID/credentials" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "aws-prod",
    "provider": "aws",
    "region": "us-east-1",
    "access_key_id": "AKIAIOSFODNN7EXAMPLE",
    "secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
  }'

2. The credential ID from the response is used in the cluster create request.

Create a provisioned cluster

Provisioning options are flat fields on the cluster body, not a nested provision_config. Setting both provider and credential_id is what switches this from a BYO registration to a provisioning request.

curl -X POST "https://api.your-domain.com/api/v1/orgs/$ORG_ID/clusters" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "staging-aws",
    "provider": "aws",
    "credential_id": "cred-uuid-...",
    "region": "us-east-1",
    "instance_type": "t3.medium",
    "agent_count": 3,
    "disk_size_gb": 40
  }'

agent_count is the number of agent nodes (0–20) alongside the control-plane node, and disk_size_gb accepts 10–500. If you omit region, the credential’s own region is used. The Kubernetes version is not selectable — provisioned clusters run the k3s version baked into the provisioner image.

Status transitions

Cloud-provisioned clusters move through more states than BYOC registrations:

pending
  │
  ▼
provisioning          ← Terraform creates VPC, security groups, EC2 instances
  │
  ▼
awaiting_operator     ← VMs are up; k3s is bootstrapped; waiting for operator first connect
  │
  ▼
connected             ← Operator has authenticated; cluster is ready

Poll or stream status:

curl -N -H "Authorization: Bearer $TOKEN" \
  "https://api.your-domain.com/api/v1/events/stream?cluster_id=$CLUSTER_ID"

Provisioning typically takes 5–10 minutes depending on the region and instance count.

SSH access

The provisioner generates an SSH key pair during provisioning. The private key is stored in the backend database (encrypted), tied to the cluster record. Retrieve it if needed for troubleshooting:

curl -H "Authorization: Bearer $TOKEN" \
  "https://api.your-domain.com/api/v1/clusters/$CLUSTER_ID/ssh-key" | jq -r .private_key > cluster.pem
chmod 600 cluster.pem
ssh -i cluster.pem ec2-user@<node-ip>

The private key is only available to organization admins. Treat it with the same care as the cluster JWT.

Scaling a cloud-provisioned cluster

curl -X POST https://api.your-domain.com/api/v1/clusters/$CLUSTER_ID/scale \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"desired_node_count": 5}'

kubenest runs a Terraform apply to add or remove EC2 instances and registers the new nodes with the existing Kubernetes control plane. BYOC clusters are not affected by the scale endpoint — node management for those clusters is handled outside kubenest.

Connection state reference

The operator implements exponential backoff reconnection. Most disconnected states resolve within 30–60 seconds without user intervention.

Deleting a cluster

Deleting a cluster record from the backend does not automatically uninstall the operator or delete the Kubernetes cluster.

For BYOC clusters, clean up in this order:

# 1. Delete all projects on the cluster first
#    (the API will reject cluster delete if projects exist)
 
# 2. Uninstall the operator from the cluster
helm uninstall kubenest-operator -n kubenest-system
 
# 3. Optionally remove the namespace
kubectl delete ns kubenest-system
 
# 4. Delete the cluster record from the backend
curl -X DELETE https://api.your-domain.com/api/v1/clusters/$CLUSTER_ID \
  -H "Authorization: Bearer $TOKEN"

For cloud-provisioned clusters, the backend runs a terraform destroy when you delete the cluster record, which tears down the EC2 instances, VPC, and associated resources. Make sure you have deleted all projects and apps first.

Troubleshooting

Operator pod is in CrashLoopBackOff. The most common cause is a malformed or expired cluster JWT. Check the pod logs for "invalid token" or "token expired". Issue a new token via GET /api/v1/clusters/{id}/install-command, update the kubenest-operator-token Secret, and restart the pod.

Cluster stuck in awaiting_operator after operator install. The operator is not reaching the hub. Verify:

• The pod is Running (not CrashLoopBackOff).
• The operator.hub.url value starts with wss:// (not ws://).
• Outbound HTTPS/WSS traffic from the cluster to the hub URL is permitted by your network policy.
• DNS resolves the hub URL from inside the cluster: kubectl exec -n kubenest-system <operator-pod> -- curl -I https://hub.your-domain.com.

connected then immediately disconnected. This is usually a heartbeat timeout. The hub expects a heartbeat from the operator every 30 seconds. Check for high CPU or memory pressure on the operator pod that might delay heartbeats. Adjust operator.heartbeat.intervalSeconds in the Helm values if your cluster is resource-constrained.

Provisioning stuck in provisioning for more than 20 minutes. Check the provisioner job logs on the backend pod:

kubectl logs -n kubenest-system \
  -l app.kubernetes.io/name=kubenest-backend \
  --tail=50 | grep provisioner

Common causes: AWS credentials lack the required IAM permissions, the requested instance type is unavailable in the region, or a VPC limit has been reached.

See also:

• Architecture Overview — how the operator, hub, and backend fit together and why the JWT model exists
• Installation — the full Helm values reference for the operator chart
• Projects — the next step after a cluster is connected