Skip to Content
Install the platform

Install the platform

The KubeNest Platform is a pinned, tested bundle of everything a Kubernetes cluster needs above the control plane — ingress, certificates, storage, backup, upgrade orchestration and OS patching — installed together as one versioned unit onto Ubuntu hosts you supply.

This page covers installing that bundle onto machines that have no Kubernetes on them yet.

What “one click” means here

The claim is deliberately narrow, because a claim we can test is worth more than a claim that sounds bigger:

From a single machine with SSH access to your target nodes, one command installs the complete core bundle in under fifteen minutes, with no further human steps.

What that does not mean:

  • It does not provision machines. You supply the hosts, the network and the disks.
  • It does not configure your DNS, your load balancer or your firewall.
  • It does not mean nothing can fail. It means a failure names the component, the stage and the remedy — see When it fails.

Before you start

Supported hosts

Ubuntu 24.04 LTS only. That is the release the bundle is tested on. The installer refuses to run on any other release or any other distribution, rather than half-installing one.

Locking the operating system is what makes the bundle testable: a known kernel and a deterministic LVM layout are preconditions for the storage and OS-patching components, and for replicated storage later.

Host sizing

Preflight checks every node against these before anything is installed.

vCPURAMDiskBuy a machine advertised as
Server node — hard floor23.7 GiB36 GiB4 GB / 40 GB
Server node — recommended47.4 GiB92 GiB8 GB / 100 GB
Agent node — hard floor23.7 GiB36 GiB4 GB / 40 GB
Agent node — recommended47.4 GiB92 GiB8 GB / 100 GB

Preflight fails below the hard floor and warns below the recommendation. The floor is k3s’s own documented minimum plus headroom for the rest of core. The recommendation is what to buy if the cluster is going to carry real workloads.

The thresholds themselves live in the bundle manifest under limits.resources, so they are versioned with the release rather than compiled into the installer.

The two columns are the same machine. The last column is what a vendor advertises; the others are what the kernel reports once you own it. A host sold as 8 GB gives /proc/meminfo 7.57 GiB; sold as 80 GB it leaves roughly 74.8 GiB on the root filesystem. Measured on a real cloud host, not rounded on paper.

Preflight compares against the GiB columns. Comparing a 4 GB floor against MemTotal as though it were GiB refuses a machine that meets the specification — and because the floor fails rather than warns, the customer is told to buy a bigger box for a box that was already correct.

Disks and storage

The bundle uses OpenEBS Local PV LVM for persistent volumes. Replicated block storage is a profile, not core — it is itself a source of operational dread, and durability in the default configuration comes from verified backups plus application-level replication instead.

Local PV LVM provisions volumes out of an LVM volume group that must exist on each node that will host persistent data. There are two supported ways to get one, and you pick per install.

Option 1 — you create the volume group (default). Create a volume group named kubenest-vg on each data-bearing node before running the installer. Preflight verifies it exists and has free extents. The installer never touches your block devices.

This asks nothing extra of you in packaging terms: lvm2 ships on the stock Ubuntu 24.04 cloud image, so pvcreate and vgcreate are already present on a fresh host.

Option 2 — the installer creates it. Name a blank device explicitly:

--storage-device /dev/nvme1n1

The installer creates kubenest-vg on that device. Preflight refuses if the device holds a partition table, a filesystem or an existing volume group, so this cannot silently overwrite data. The flag is required — there is no device auto-detection, because guessing which disk is disposable on someone else’s infrastructure is not a risk worth taking.

Creating a volume group from free space on the root disk is not supported.

Which option you used is recorded against the cluster, because it determines what uninstall is allowed to remove.

Backup

Velero is installed as part of core. Configuring a backup target at install time is optional.

If you supply an S3-compatible target with --backup-target, Velero is configured and the first scheduled backup runs per the schedule on the backup and restore page. If you do not, Velero is installed unconfigured and the installer says so as it passes stage 8.

A cluster that has never taken a backup is exactly the quiet failure this product exists to prevent, so it is surfaced continuously rather than blocking the install.

The agent reports backup state in every health report, and the control plane turns an unconfigured target into a BACKUP_TARGET_UNCONFIGURED warning. The same check reports a failed backup, a backup that has never run, and one that has gone stale past the manifest’s max-backup-age. The console shows the verdict on the cluster. Configure a target after the fact with kubenest backup set-target — see Backup and restore.

Velero itself is not optional and is not a profile. Core is a fixed set — variation comes from profiles only, or the tested-configuration count stops being N+2.

Network

Between cluster nodes:

PortProtocolDirectionPurpose
6443TCPagents → serverKubernetes API
8472UDPall ↔ allFlannel VXLAN overlay
10250TCPall ↔ allKubelet metrics
2379–2380TCPserver ↔ serveretcd peer replication — 3-node ha tier only

Outbound from the cluster and from the machine running the installer:

  • HTTPS to container registries and Helm repositories, for the duration of the install.
  • HTTPS to your KubeNest control plane, from the installer.
  • Persistent outbound WSS to your KubeNest hub, from the cluster. The agent dials out, so no node needs a public address for that session.

One inbound path is easy to miss when planning firewall rules: the control plane connects to your cluster’s Kubernetes API server for some operations, using the credentials the agent advertises. That endpoint must be reachable from the control plane.

Air-gapped installs are not supported. Both the install and the running cluster require outbound internet access, and so does installing with no control plane reachable.

Access

The machine you run the installer from needs SSH to every target node, with passwordless sudo on each.

The installer uses your existing SSH setup: ~/.ssh/config, ssh-agent, or a key file named with --ssh-key. Key material stays on your machine. It is never uploaded to the control plane, never stored by KubeNest, and never written to the installer’s logs.

Preflight runs sudo -n true on each node and fails with the exact remediation if it does not succeed. The default ubuntu user on official Ubuntu cloud images already has passwordless sudo configured by cloud-init, so on a stock AWS Ubuntu host this needs nothing from you.

An existing cluster on these hosts

Not supported. The installer builds new clusters only. Preflight refuses to run on any host where k3s, RKE2, kubelet or containerd is already present, and says so plainly.

Adopting a cluster that already exists means inheriting whatever ingress, CSI and cert manager are already installed on it — which is the untested component combination the bundle exists to eliminate. It is a different product and it is out of scope.

Choose your tiers before you run

Two choices are made at install time and recorded permanently against the cluster.

HA tier

There is no supported path from single-server to ha after install. Both tiers run embedded etcd, so the obstacle is not the datastore — growing to ha is joining two more servers to an existing etcd cluster. The installer performs that join at install time; no command performs it against a cluster that already exists. Treat the tier as an install-time choice and pick deliberately.

single-server — one control-plane node running single-node embedded etcd, plus verified snapshots. Simpler, cheaper, fewer moving parts. The honest promise is “we restore, and the restore is drilled”, never “highly available”. Restore duration depends on your data volume, so the drill reports your own measured figure rather than quoting one.

Be clear on what a single-server control-plane failure actually does: running pods keep running, because the kubelet does not need the API server. But nothing new schedules, nothing self-heals, no deploy or scale succeeds, and any pod or node that dies is never replaced. Ingress keeps serving its last known configuration. The cluster is frozen, not down — until the second failure, which cascades with no recovery path.

ha — three control-plane nodes with etcd replicated across them. Survives the loss of one node. Triples control-plane cost and introduces etcd quorum operations.

Profiles

Core is always installed. Profiles are chosen at install and are the only supported way to vary what a cluster runs — free-form component toggles would mean 2N test configurations instead of N+2.

ProfileContents
observabilityVictoriaMetrics, Grafana, Loki
secretssealed-secrets
ha3-node embedded etcd control plane

Profiles, what changing one later does, and what it does not do, are covered on the Profiles page.

The install command

The installer is the kubenest CLI, run from your own workstation or bastion. Your SSH keys stay on your machine.

Authenticate to your control plane once:

terminal
kubenest login --control-plane https://api.your-domain.com

Then install:

terminal
kubenest platform install \ --bundle 1.0 \ --name prod-1 \ --server 10.0.1.10 \ --agent 10.0.1.11 \ --agent 10.0.1.12 \ --ha single-server \ --ssh-user ubuntu \ --ssh-key ~/.ssh/id_ed25519

--bundle selects the platform version. Every component version is pinned by that number — the bundle is the unit of test, of support and of upgrade, and a cluster is always at exactly one bundle version plus a recorded profile set. Install Platform 1.0. The catalog also carries Platform 0.9, which exists so the upgrade path has a real predecessor to move from — it differs from 1.0 in the k3s and agent pins only, and there is no reason to install it deliberately. A version the control plane does not offer is refused by preflight. What each version contains is on the Bundle contents page.

Re-running the same command against the same cluster is safe. The installer converges: it does not duplicate resources and it does not fail because something is already present.

Re-running is a journal replay rather than a second install. preflight, register and verify always run; every stage the journal records as completed is skipped. A measured second run of an identical command took 14 seconds and skipped ten of the thirteen stages.

A control plane is required

kubenest platform install registers the cluster with your control plane as part of the install, and will not run without it. That is deliberate rather than incidental:

  • The cluster’s bundle version, profile set and HA tier have to be recorded somewhere durable, or no later upgrade is safe. You cannot upgrade a cluster whose contents you only think you know.
  • Fleet health telemetry starts at install. Not knowing the state of a cluster is what turns support into an unbounded commitment.
  • Upgrade orchestration, scoped per-client access and the restore drills all read that record.

Installing with no control plane reachable is not supported — see the note on air-gapped installs above.

Provisioning the machines

kubenest platform install expects hosts to exist. It does not create them.

You bring the machines. Provision them however you already provision servers — your cloud console, your Terraform, your hardware. The installer takes it from there.

Machine provisioning through kubenest is not part of this release. If it arrives, it will be a separate command rather than a flag on install: provisioning and installing fail in different ways and are re-run under different circumstances, and a failed install must never re-run terraform apply. GCP and Azure are not planned.

Do customers manage clusters through the console?

For installing, no — the CLI is sufficient and self-contained. For everything after, yes. The console and the control plane own the bundle and profile record, fleet health, upgrades, backup and restore drills, and scoped per-client access. Day 1 is a CLI. Day 2 is the product.

What the installer does, in order

Thirteen stages. Each emits a progress event so the console can show real progress rather than a spinner, and each names itself in any failure.

The order is not arbitrary — every stage depends on the ones above it.

#StageWhat happens
1preflightControl-plane authentication, then every check in the table below, against every node
2registerThe cluster record is created and the agent JWT issued
3k3s-serverk3s at the pinned version on the control-plane node, or all three for the ha tier
4k3s-agentsAgents join the server
5platform-networkingTraefik with the Gateway API provider, and the Gateway API CRDs
6platform-certscert-manager
7platform-storageThe volume group is verified or created, then OpenEBS Local PV LVM and the default StorageClass
8platform-backupVelero, configured if a target was supplied
9platform-day2system-upgrade-controller and kured
10kubenest-agentThe KubeNest agent, the hub handshake, and the first telemetry heartbeat
11profilesEach selected profile, in the order listed above. No component profile is built yet, so this stage refuses one rather than installing core and calling it done
12recordThe bundle version, profile set, HA tier and volume-group ownership are written against the cluster record
13verifyThe acceptance checks below are run and reported

Stage 1 writes nothing anywhere. Stage 2 writes only to the control plane. The first change to any of your machines happens in stage 3.

Everything above the labelled edge is recoverable by walking away. That is the property the ordering exists to give you: the expensive checks happen while abandoning the install costs nothing.

The whole sequence must complete in under fifteen minutes, for both reference shapes: a single-server core install, and a three-node install with every profile enabled. If a measured install exceeds it, that is a defect to fix, not a number to revise upward.

A single-server core install on a 4 vCPU / 8 GB Ubuntu 24.04 host completes all thirteen stages in about 4m14s, and the release test asserts the budget on both shapes.

The budget and the timeout are different numbers, deliberately. Fifteen minutes is the target, and failing it is a defect in the installer. limits.timeouts.install-total (default 30m) is the deadline, after which the install aborts and reports which stage was still running.

Collapsing the two would mean either a target so loose it asserts nothing, or an install that aborts on a slow image pull. A run that takes twenty minutes has failed its budget and succeeded at its job — those deserve different outcomes.

Traefik with Gateway API, not ingress-nginx. kubernetes/ingress-nginx reached end of life on 24 March 2026. The repository is read-only: no features, no fixes and no CVE patches — after four HIGH-severity CVEs disclosed together in February 2026 and the IngressNightmare unauthenticated RCE before them. Traefik is already the k3s default, so this is the lighter choice as well as the safe one.

Converting existing Ingress resources to Gateway API with ingress2gateway is not part of the install flow. It is a separate migration.

Preflight checks

Preflight is the whole reason a failed install is cheap. Everything here is checked on every node before the first byte is written anywhere, and any failure aborts before a machine is touched.

CheckFails when
Control planeThe control plane is unreachable, or the CLI is not logged in
SSH reachabilityA node cannot be reached, or the key is rejected
Operating systemThe node is not running Ubuntu 24.04 LTS
Privilegesudo -n true fails
Existing Kubernetesk3s, RKE2, kubelet or containerd is already present
Volume groupkubenest-vg is missing, or --storage-device names a device that is not blank — the check is that blkid on the device returns nothing. A device carrying a partition table, a filesystem or an existing LVM PV reports a TYPE, and that is a refusal
Node-to-node portsAny port in the table above is blocked between nodes
Outbound egressRegistries, Helm repositories, the control plane or the hub are unreachable
Host resourcesReported CPU, MemTotal or filesystem free space is below limits.resources.floor, compared in binary units
Node countFewer than three control-plane nodes for --ha ha
Bundle availabilityThe requested bundle version does not exist, or does not offer the requested tier

Verifying the install

Stage 13 runs these automatically and reports the result. They are also the checks to run by hand when something looks wrong, and they are the acceptance criteria the release tests assert against a real cluster.

These are convergence checks, not snapshots. Every one of them waits for a condition to hold within a window and reports the last state it saw. None of them samples once and judges.

This is not a refinement. A clean k3s install puts helm-install-traefik into Error before it retries and completes — observed on a real host. A check that sampled at that moment would fail a healthy install, and the operator would be debugging the installer rather than using the cluster.

Each check has a deadline from limits.timeouts in the bundle manifest, and each reports one of three outcomes, never two:

OutcomeMeaning
passThe condition held within its window
convergingNot there yet, deadline not reached. Progress is printed, not silence
failThe deadline passed. Reports the last observed state and which object was stuck

converging is the outcome that stops false failures, and printing progress while it lasts is what stops the operator killing an install that was going to succeed.

Every node is Ready

terminal
kubectl get nodes

Every node reaches Ready within limits.timeouts.node-ready (default 5m).

Every core component is Running

terminal
kubectl get pods -A

Traefik, cert-manager, OpenEBS, Velero, system-upgrade-controller, kured and the KubeNest agent all reach Ready within limits.timeouts.component-ready (default 10m per release).

CrashLoopBackOff and Pending are not immediate failures. Both are legitimate transient states during an install — images pull, dependencies order themselves, a Helm hook retries. They fail the check only if they are still present at the deadline. What the check reports on failure is the pod, its state, and its last event, because “traefik is Pending, no node matches its node selector” is a fix and “install failed” is not.

Storage provisions a volume for real

A test PersistentVolumeClaim against the default StorageClass binds, rather than the StorageClass merely existing. A StorageClass that cannot actually provision is the most common way a storage install looks successful and is not.

The cluster reports in

The cluster shows as connected in the console and the first fleet-telemetry heartbeat has arrived. A cluster the control plane still records as install_failed does not pass.

The heartbeat carries node count, Kubernetes version and node resource data. The bundle version, profile set, HA tier and volume-group ownership reach the control plane from stage 12, not from the heartbeat.

Alongside it the agent sends a health report — nodes, control plane, certificate expiry dates, backup and restore-drill evidence, and its own version — which the control plane evaluates into per-check verdicts and alerts. Two groups it cannot collect yet, datastore health and drift from the recorded bundle, report the reason they are missing and evaluate to unknown. unknown is never rendered as healthy, so an unmeasured check is visible as unmeasured rather than counted green.

The recorded manifest matches reality

Every core pin in the bundle is compared against what is actually on the cluster — the kubelet version on each node, and the chart version of each component k3s reconciles. Nothing else in the day-2 story is trustworthy if this drifts.

When it fails

A failed install that says error is worse than no installer. Every failure reports three things: which stage, which component, and what to do next.

There is no automatic rollback. A failed stage stops the install and leaves completed stages in place. Automatic teardown would destroy the evidence needed to diagnose the failure, can itself fail and leave a worse state, and is more expensive than resuming.

What makes that safe is that a half-installed cluster is never an unexplained state:

  • The installer keeps a journal. Each completed stage is recorded, locally and against the cluster record. Resume is deterministic — it reads the journal rather than relying on every component happening to be idempotent.
  • The cluster is marked install-failed in the control plane, carrying the failing stage. It never appears as a healthy managed cluster. Fleet telemetry knows about it immediately, so a support call about it is never a surprise.
  • There are exactly two supported exits, and the failure message prints both.

To recover, either:

terminal
# Resume: fix what the error names, then re-run the identical command. kubenest platform install ... # skips completed stages, resumes at the failure
terminal
# Or start over: return the hosts to a known state. kubenest platform uninstall --confirm

If preflight failed, nothing was written to any node and no cluster record exists. Fix the reported condition and re-run. This is the common case, and it is why preflight is thorough.

Uninstall

Enterprise buyers ask about the exit before they commit to the entry, so the exit is documented and tested rather than improvised.

terminal
kubenest platform uninstall --confirm

This removes k3s and every component the installer placed, and leaves the machines in a known state.

Uninstall never destroys data by default. Persistent volumes and their contents survive.

To remove them, pass --destroy-data as well. Even then, uninstall only removes the kubenest-vg volume group if the installer created it — recorded at install time. A volume group you created yourself is never removed, on either path.

Last updated on