Install the platform
The KubeNest Platform is a pinned, tested bundle of everything a Kubernetes cluster needs above the control plane — ingress, certificates, storage, backup, upgrade orchestration and OS patching — installed together as one versioned unit onto Ubuntu hosts you supply.
This page covers installing that bundle onto machines that have no Kubernetes on them yet.
What “one click” means here
The claim is deliberately narrow, because a claim we can test is worth more than a claim that sounds bigger:
From a single machine with SSH access to your target nodes, one command installs the complete core bundle in under fifteen minutes, with no further human steps.
What that does not mean:
- It does not provision machines. You supply the hosts, the network and the disks.
- It does not configure your DNS, your load balancer or your firewall.
- It does not mean nothing can fail. It means a failure names the component, the stage and the remedy — see When it fails.
Before you start
Supported hosts
Ubuntu 24.04 LTS only. That is the release the bundle is tested on. The installer refuses to run on any other release or any other distribution, rather than half-installing one.
Locking the operating system is what makes the bundle testable: a known kernel and a deterministic LVM layout are preconditions for the storage and OS-patching components, and for replicated storage later.
Host sizing
Preflight checks every node against these before anything is installed.
| vCPU | RAM | Disk | Buy a machine advertised as | |
|---|---|---|---|---|
| Server node — hard floor | 2 | 3.7 GiB | 36 GiB | 4 GB / 40 GB |
| Server node — recommended | 4 | 7.4 GiB | 92 GiB | 8 GB / 100 GB |
| Agent node — hard floor | 2 | 3.7 GiB | 36 GiB | 4 GB / 40 GB |
| Agent node — recommended | 4 | 7.4 GiB | 92 GiB | 8 GB / 100 GB |
Preflight fails below the hard floor and warns below the recommendation. The floor is k3s’s own documented minimum plus headroom for the rest of core. The recommendation is what to buy if the cluster is going to carry real workloads.
The thresholds themselves live in the bundle manifest under limits.resources, so they are
versioned with the release rather than compiled into the installer.
The two columns are the same machine. The last column is what a vendor advertises; the
others are what the kernel reports once you own it. A host sold as 8 GB gives /proc/meminfo
7.57 GiB; sold as 80 GB it leaves roughly 74.8 GiB on the root filesystem. Measured on a real
cloud host, not rounded on paper.
Preflight compares against the GiB columns. Comparing a 4 GB floor against MemTotal as
though it were GiB refuses a machine that meets the specification — and because the floor fails
rather than warns, the customer is told to buy a bigger box for a box that was already correct.
Disks and storage
The bundle uses OpenEBS Local PV LVM for persistent volumes. Replicated block storage is a profile, not core — it is itself a source of operational dread, and durability in the default configuration comes from verified backups plus application-level replication instead.
Local PV LVM provisions volumes out of an LVM volume group that must exist on each node that will host persistent data. There are two supported ways to get one, and you pick per install.
Option 1 — you create the volume group (default). Create a volume group named kubenest-vg
on each data-bearing node before running the installer. Preflight verifies it exists and has free
extents. The installer never touches your block devices.
This asks nothing extra of you in packaging terms: lvm2 ships on the stock Ubuntu 24.04 cloud
image, so pvcreate and vgcreate are already present on a fresh host.
Option 2 — the installer creates it. Name a blank device explicitly:
--storage-device /dev/nvme1n1The installer creates kubenest-vg on that device. Preflight refuses if the device holds a
partition table, a filesystem or an existing volume group, so this cannot silently overwrite data.
The flag is required — there is no device auto-detection, because guessing which disk is
disposable on someone else’s infrastructure is not a risk worth taking.
Creating a volume group from free space on the root disk is not supported.
Which option you used is recorded against the cluster, because it determines what uninstall is allowed to remove.
Backup
Velero is installed as part of core. Configuring a backup target at install time is optional.
If you supply an S3-compatible target with --backup-target, Velero is configured and the first
scheduled backup runs per the schedule on the backup and restore page. If you do not, Velero is
installed unconfigured and the installer says so as it passes stage 8.
A cluster that has never taken a backup is exactly the quiet failure this product exists to prevent, so it is surfaced continuously rather than blocking the install.
The agent reports backup state in every health report, and the control plane turns an unconfigured
target into a BACKUP_TARGET_UNCONFIGURED warning. The same check reports a failed backup, a
backup that has never run, and one that has gone stale past the manifest’s max-backup-age. The
console shows the verdict on the cluster. Configure a target after the fact with
kubenest backup set-target — see Backup and restore.
Velero itself is not optional and is not a profile. Core is a fixed set — variation comes from profiles only, or the tested-configuration count stops being N+2.
Network
Between cluster nodes:
| Port | Protocol | Direction | Purpose |
|---|---|---|---|
| 6443 | TCP | agents → server | Kubernetes API |
| 8472 | UDP | all ↔ all | Flannel VXLAN overlay |
| 10250 | TCP | all ↔ all | Kubelet metrics |
| 2379–2380 | TCP | server ↔ server | etcd peer replication — 3-node ha tier only |
Outbound from the cluster and from the machine running the installer:
- HTTPS to container registries and Helm repositories, for the duration of the install.
- HTTPS to your KubeNest control plane, from the installer.
- Persistent outbound WSS to your KubeNest hub, from the cluster. The agent dials out, so no node needs a public address for that session.
One inbound path is easy to miss when planning firewall rules: the control plane connects to your cluster’s Kubernetes API server for some operations, using the credentials the agent advertises. That endpoint must be reachable from the control plane.
Air-gapped installs are not supported. Both the install and the running cluster require outbound internet access, and so does installing with no control plane reachable.
Access
The machine you run the installer from needs SSH to every target node, with passwordless sudo on each.
The installer uses your existing SSH setup: ~/.ssh/config, ssh-agent, or a key file named
with --ssh-key. Key material stays on your machine. It is never uploaded to the control plane,
never stored by KubeNest, and never written to the installer’s logs.
Preflight runs sudo -n true on each node and fails with the exact remediation if it does not
succeed. The default ubuntu user on official Ubuntu cloud images already has passwordless sudo
configured by cloud-init, so on a stock AWS Ubuntu host this needs nothing from you.
An existing cluster on these hosts
Not supported. The installer builds new clusters only. Preflight refuses to run on any host where k3s, RKE2, kubelet or containerd is already present, and says so plainly.
Adopting a cluster that already exists means inheriting whatever ingress, CSI and cert manager are already installed on it — which is the untested component combination the bundle exists to eliminate. It is a different product and it is out of scope.
Choose your tiers before you run
Two choices are made at install time and recorded permanently against the cluster.
HA tier
There is no supported path from single-server to ha after install. Both tiers run embedded
etcd, so the obstacle is not the datastore — growing to ha is joining two more servers to an
existing etcd cluster. The installer performs that join at install time; no command performs it
against a cluster that already exists. Treat the tier as an install-time choice and pick
deliberately.
single-server — one control-plane node running single-node embedded etcd, plus verified
snapshots. Simpler, cheaper, fewer
moving parts. The honest promise is “we restore, and the restore is drilled”, never “highly
available”. Restore duration depends on your data volume, so the drill reports your own measured
figure rather than quoting one.
Be clear on what a single-server control-plane failure actually does: running pods keep running, because the kubelet does not need the API server. But nothing new schedules, nothing self-heals, no deploy or scale succeeds, and any pod or node that dies is never replaced. Ingress keeps serving its last known configuration. The cluster is frozen, not down — until the second failure, which cascades with no recovery path.
ha — three control-plane nodes with etcd replicated across them. Survives the loss of one
node. Triples
control-plane cost and introduces etcd quorum operations.
Profiles
Core is always installed. Profiles are chosen at install and are the only supported way to vary what a cluster runs — free-form component toggles would mean 2N test configurations instead of N+2.
| Profile | Contents |
|---|---|
observability | VictoriaMetrics, Grafana, Loki |
secrets | sealed-secrets |
ha | 3-node embedded etcd control plane |
Profiles, what changing one later does, and what it does not do, are covered on the Profiles page.
The install command
The installer is the kubenest CLI, run from your own workstation or bastion. Your SSH keys stay
on your machine.
Authenticate to your control plane once:
kubenest login --control-plane https://api.your-domain.comThen install:
kubenest platform install \
--bundle 1.0 \
--name prod-1 \
--server 10.0.1.10 \
--agent 10.0.1.11 \
--agent 10.0.1.12 \
--ha single-server \
--ssh-user ubuntu \
--ssh-key ~/.ssh/id_ed25519--bundle selects the platform version. Every component version is pinned by that number — the
bundle is the unit of test, of support and of upgrade, and a cluster is always at exactly one
bundle version plus a recorded profile set. Install Platform 1.0. The catalog also carries
Platform 0.9, which exists so the upgrade path has a real predecessor to move from — it differs
from 1.0 in the k3s and agent pins only, and there is no reason to install it deliberately. A
version the control plane does not offer is refused by preflight. What each version contains is
on the Bundle contents page.
Re-running the same command against the same cluster is safe. The installer converges: it does not duplicate resources and it does not fail because something is already present.
Re-running is a journal replay rather than a second install. preflight, register and verify
always run; every stage the journal records as completed is skipped. A measured second run of an
identical command took 14 seconds and skipped ten of the thirteen stages.
A control plane is required
kubenest platform install registers the cluster with your control plane as part of the install,
and will not run without it. That is deliberate rather than incidental:
- The cluster’s bundle version, profile set and HA tier have to be recorded somewhere durable, or no later upgrade is safe. You cannot upgrade a cluster whose contents you only think you know.
- Fleet health telemetry starts at install. Not knowing the state of a cluster is what turns support into an unbounded commitment.
- Upgrade orchestration, scoped per-client access and the restore drills all read that record.
Installing with no control plane reachable is not supported — see the note on air-gapped installs above.
Provisioning the machines
kubenest platform install expects hosts to exist. It does not create them.
You bring the machines. Provision them however you already provision servers — your cloud console, your Terraform, your hardware. The installer takes it from there.
Machine provisioning through kubenest is not part of this release. If it arrives, it will be a
separate command rather than a flag on install: provisioning and installing fail in different
ways and are re-run under different circumstances, and a failed install must never re-run
terraform apply. GCP and Azure are not planned.
Do customers manage clusters through the console?
For installing, no — the CLI is sufficient and self-contained. For everything after, yes. The console and the control plane own the bundle and profile record, fleet health, upgrades, backup and restore drills, and scoped per-client access. Day 1 is a CLI. Day 2 is the product.
What the installer does, in order
Thirteen stages. Each emits a progress event so the console can show real progress rather than a spinner, and each names itself in any failure.
The order is not arbitrary — every stage depends on the ones above it.
| # | Stage | What happens |
|---|---|---|
| 1 | preflight | Control-plane authentication, then every check in the table below, against every node |
| 2 | register | The cluster record is created and the agent JWT issued |
| 3 | k3s-server | k3s at the pinned version on the control-plane node, or all three for the ha tier |
| 4 | k3s-agents | Agents join the server |
| 5 | platform-networking | Traefik with the Gateway API provider, and the Gateway API CRDs |
| 6 | platform-certs | cert-manager |
| 7 | platform-storage | The volume group is verified or created, then OpenEBS Local PV LVM and the default StorageClass |
| 8 | platform-backup | Velero, configured if a target was supplied |
| 9 | platform-day2 | system-upgrade-controller and kured |
| 10 | kubenest-agent | The KubeNest agent, the hub handshake, and the first telemetry heartbeat |
| 11 | profiles | Each selected profile, in the order listed above. No component profile is built yet, so this stage refuses one rather than installing core and calling it done |
| 12 | record | The bundle version, profile set, HA tier and volume-group ownership are written against the cluster record |
| 13 | verify | The acceptance checks below are run and reported |
Stage 1 writes nothing anywhere. Stage 2 writes only to the control plane. The first change to any of your machines happens in stage 3.
Everything above the labelled edge is recoverable by walking away. That is the property the ordering exists to give you: the expensive checks happen while abandoning the install costs nothing.
The whole sequence must complete in under fifteen minutes, for both reference shapes: a single-server core install, and a three-node install with every profile enabled. If a measured install exceeds it, that is a defect to fix, not a number to revise upward.
A single-server core install on a 4 vCPU / 8 GB Ubuntu 24.04 host completes all thirteen stages in about 4m14s, and the release test asserts the budget on both shapes.
The budget and the timeout are different numbers, deliberately. Fifteen minutes is the
target, and failing it is a defect in the installer. limits.timeouts.install-total
(default 30m) is the deadline, after which the install aborts and reports which stage was
still running.
Collapsing the two would mean either a target so loose it asserts nothing, or an install that aborts on a slow image pull. A run that takes twenty minutes has failed its budget and succeeded at its job — those deserve different outcomes.
Traefik with Gateway API, not ingress-nginx. kubernetes/ingress-nginx reached end of life
on 24 March 2026. The repository is read-only: no features, no fixes and no CVE patches —
after four HIGH-severity CVEs disclosed together in February 2026 and the IngressNightmare
unauthenticated RCE before them. Traefik is already the k3s default, so this is the lighter
choice as well as the safe one.
Converting existing Ingress resources to Gateway API with ingress2gateway is not part of the
install flow. It is a separate migration.
Preflight checks
Preflight is the whole reason a failed install is cheap. Everything here is checked on every node before the first byte is written anywhere, and any failure aborts before a machine is touched.
| Check | Fails when |
|---|---|
| Control plane | The control plane is unreachable, or the CLI is not logged in |
| SSH reachability | A node cannot be reached, or the key is rejected |
| Operating system | The node is not running Ubuntu 24.04 LTS |
| Privilege | sudo -n true fails |
| Existing Kubernetes | k3s, RKE2, kubelet or containerd is already present |
| Volume group | kubenest-vg is missing, or --storage-device names a device that is not blank — the check is that blkid on the device returns nothing. A device carrying a partition table, a filesystem or an existing LVM PV reports a TYPE, and that is a refusal |
| Node-to-node ports | Any port in the table above is blocked between nodes |
| Outbound egress | Registries, Helm repositories, the control plane or the hub are unreachable |
| Host resources | Reported CPU, MemTotal or filesystem free space is below limits.resources.floor, compared in binary units |
| Node count | Fewer than three control-plane nodes for --ha ha |
| Bundle availability | The requested bundle version does not exist, or does not offer the requested tier |
Verifying the install
Stage 13 runs these automatically and reports the result. They are also the checks to run by hand when something looks wrong, and they are the acceptance criteria the release tests assert against a real cluster.
These are convergence checks, not snapshots. Every one of them waits for a condition to hold within a window and reports the last state it saw. None of them samples once and judges.
This is not a refinement. A clean k3s install puts helm-install-traefik into Error before it
retries and completes — observed on a real host. A check that sampled at that moment would fail
a healthy install, and the operator would be debugging the installer rather than using the
cluster.
Each check has a deadline from limits.timeouts in the bundle manifest, and each reports one of
three outcomes, never two:
| Outcome | Meaning |
|---|---|
pass | The condition held within its window |
converging | Not there yet, deadline not reached. Progress is printed, not silence |
fail | The deadline passed. Reports the last observed state and which object was stuck |
converging is the outcome that stops false failures, and printing progress while it lasts is
what stops the operator killing an install that was going to succeed.
Every node is Ready
kubectl get nodesEvery node reaches Ready within limits.timeouts.node-ready (default 5m).
Every core component is Running
kubectl get pods -ATraefik, cert-manager, OpenEBS, Velero, system-upgrade-controller, kured and the KubeNest
agent all reach Ready within limits.timeouts.component-ready (default 10m per release).
CrashLoopBackOff and Pending are not immediate failures. Both are legitimate transient
states during an install — images pull, dependencies order themselves, a Helm hook retries. They
fail the check only if they are still present at the deadline. What the check reports on failure
is the pod, its state, and its last event, because “traefik is Pending, no node matches its
node selector” is a fix and “install failed” is not.
Storage provisions a volume for real
A test PersistentVolumeClaim against the default StorageClass binds, rather than the StorageClass merely existing. A StorageClass that cannot actually provision is the most common way a storage install looks successful and is not.
The cluster reports in
The cluster shows as connected in the console and the first fleet-telemetry heartbeat has
arrived. A cluster the control plane still records as install_failed does not pass.
The heartbeat carries node count, Kubernetes version and node resource data. The bundle version, profile set, HA tier and volume-group ownership reach the control plane from stage 12, not from the heartbeat.
Alongside it the agent sends a health report — nodes, control plane, certificate expiry dates,
backup and restore-drill evidence, and its own version — which the control plane evaluates into
per-check verdicts and alerts. Two groups it cannot collect yet, datastore health and drift from
the recorded bundle, report the reason they are missing and evaluate to unknown. unknown is
never rendered as healthy, so an unmeasured check is visible as unmeasured rather than counted
green.
The recorded manifest matches reality
Every core pin in the bundle is compared against what is actually on the cluster — the kubelet version on each node, and the chart version of each component k3s reconciles. Nothing else in the day-2 story is trustworthy if this drifts.
When it fails
A failed install that says error is worse than no installer. Every failure reports three things:
which stage, which component, and what to do next.
There is no automatic rollback. A failed stage stops the install and leaves completed stages in place. Automatic teardown would destroy the evidence needed to diagnose the failure, can itself fail and leave a worse state, and is more expensive than resuming.
What makes that safe is that a half-installed cluster is never an unexplained state:
- The installer keeps a journal. Each completed stage is recorded, locally and against the cluster record. Resume is deterministic — it reads the journal rather than relying on every component happening to be idempotent.
- The cluster is marked
install-failedin the control plane, carrying the failing stage. It never appears as a healthy managed cluster. Fleet telemetry knows about it immediately, so a support call about it is never a surprise. - There are exactly two supported exits, and the failure message prints both.
To recover, either:
# Resume: fix what the error names, then re-run the identical command.
kubenest platform install ... # skips completed stages, resumes at the failure# Or start over: return the hosts to a known state.
kubenest platform uninstall --confirmIf preflight failed, nothing was written to any node and no cluster record exists. Fix the reported condition and re-run. This is the common case, and it is why preflight is thorough.
Uninstall
Enterprise buyers ask about the exit before they commit to the entry, so the exit is documented and tested rather than improvised.
kubenest platform uninstall --confirmThis removes k3s and every component the installer placed, and leaves the machines in a known state.
Uninstall never destroys data by default. Persistent volumes and their contents survive.
To remove them, pass --destroy-data as well. Even then, uninstall only removes the
kubenest-vg volume group if the installer created it — recorded at install time. A volume group
you created yourself is never removed, on either path.