HA tiers

Two tiers. The difference is what happens when a control-plane node dies, and it is a large difference. Choose it at install — see why.

This page states what each tier does and does not promise. That second half is the point. A customer who is surprised by their tier’s behaviour during an incident was mis-sold, and the sentence they needed was one we chose not to write.

The two tiers

	`single-server`	`ha`
Control-plane nodes	1	3
Datastore	Embedded etcd, single node	Embedded etcd, replicated across three
Survives losing a control-plane node	No	Yes, one of three
API available during a control-plane reboot	No	Yes
Control-plane node reboots for OS patching	Needs a human	Automatic, one at a time
Honest promise	”We restore, and the restore is drilled"	"Survives the loss of one node”
Cost	1 node	3 nodes, plus etcd operations

`single-server`

One control-plane node. Simpler, cheaper, fewer moving parts, and genuinely the right answer for plenty of workloads.

What it promises: that when the control-plane node is lost, the cluster can be rebuilt from datastore snapshots and workload backups, and that the restore path has been tested rather than assumed. That second half is now literal: the release gate rebuilt embedded etcd from a snapshot held only in S3, with the node-local copy deleted first.

It deliberately does not promise a single number of hours. Restore time is dominated by how much volume data you hold, so one figure quoted for every cluster would be wrong for almost all of them. What the platform gives you instead is your figure: every scheduled restore drill reports its own duration, against your data, on your hardware — so the number you plan around is measured rather than promised.

What it does not promise: high availability. It is not a high-availability configuration and should never be described as one.

What actually happens when the control-plane node dies

Read this before choosing the tier, because the failure mode is unusual and it is easy to misjudge.

Your applications on other nodes keep running. The kubelet does not need the API server to keep existing containers alive. Traffic keeps flowing. Ingress keeps serving its last known configuration. From outside, for a while, nothing looks wrong.

Workloads on the control-plane node itself stop. They died with the node, and with no API server nothing reschedules them elsewhere. So this is a placement condition, not a property of the tier: only replicas already on surviving agent nodes keep serving. The platform does not restrict workloads to agent nodes — unless you have constrained scheduling yourself, some replicas may be on the control-plane node. If the control-plane node is the only node, everything stops.

Nothing else works:

Nothing new schedules
Nothing self-heals — a crashed pod is not restarted, a failed node is not drained
No deploy, no scale, no config change
kubectl does not respond
Autoscaling, if you have it, silently stops

The cluster is frozen, not down. That is a strange state and a dangerous one, because it looks survivable and is not. The first failure took away every mechanism that would handle the second one. Until you restore, you are running without a safety net, and the clock is quiet.

This is why the restore drill is not optional. The single-server tier is honest exactly to the degree that the restore path is proven. Verified restore drills are in core and run weekly for this reason: each one restores real objects and real volume bytes and compares them, a failure is an alert rather than a log line, and the result is the evidence the upgrade pre-flight reads before it will touch anything — see backup and restore.

Snapshot freshness is watched on the same terms: a cluster whose hourly datastore snapshots stop raises an alert well before the gap becomes the thing you find out about during a recovery.

It is also why we tell you this rather than burying it. Because KubeNest abstracts k3s away, you are less able to diagnose a frozen cluster than you would be if you had assembled it yourself. That is a fair trade only if we have told you, in advance and in writing, exactly what to expect and exactly what to do.

What to do when it happens

Confirm the diagnosis. kubectl is unresponsive but your applications on other nodes are still serving. The console will show the cluster’s telemetry has stopped.
Do not restart workloads or nodes. Everything still running is running fine, and a node that restarts now will not come back into a cluster with no API server.
Rebuild and restore, following restoring a whole cluster. Same bundle version, same profile set — both are on the cluster record.

`ha`

Three control-plane nodes with embedded etcd, maintaining quorum.

What it promises: the loss of one control-plane node is survivable without restoring anything. The API stays available, workloads keep scheduling and self-healing, and control-plane nodes can reboot for OS patching one at a time without a human deciding.

That last one is the tier’s whole day-to-day advantage — see OS patching for how the reboots are coordinated.

What it does not promise:

Surviving the loss of two. Three nodes with quorum of two means two failures is a stopped cluster, and recovering an etcd cluster that has lost quorum is worse than restoring a single-server one.
Freedom from backups. Replication is not backup. It faithfully replicates a deletion to all three nodes. You still need the backups and the drills.
Zero operational weight. etcd brings quorum, compaction and defragmentation. We absorb that on your behalf, but it is real, and it is why the tier is priced differently rather than offered as a free upgrade.
Telling you the moment you have spent your one failure. Fleet health derives redundancy from control-plane node health, so a lost node is reported as degraded quorum. An etcd member that fails while its node stays Ready is not: the operator cannot read etcd membership from inside a pod yet, so that cluster still reads as having quorum.

Two things the first three-node install is expected to test, because a single-server one could not: whether etcd is actually healthy before the rest of the bundle is applied — the installer waits for three nodes to report Ready, which is not the same as a quorum that has formed — and whether requiring the kubenest-vg volume group on all three control-plane nodes is right for servers that will hold no workloads. Preflight checks every node for it today.

Why this is chosen at install

Because moving between the tiers afterwards is not something we currently support, and getting that wrong is expensive.

Today a cluster installed as single-server stays single-server unless it is rebuilt. The obstacle is not the datastore — both tiers run embedded etcd, so growing to ha is joining two more servers rather than converting anything, and the installer already performs exactly that join on a fresh --ha ha install. What does not exist is the day-2 path: nothing joins servers to a cluster that is already installed. So the choice is made deliberately, priced accordingly, and recorded against the cluster, so the control plane knows which tier every cluster bought rather than anyone having to remember.

Choosing

Take ha when any of these is true:

Scheduling and self-healing must keep working through a node failure
The cluster runs something where a restore-shaped outage is not acceptable
You do not want a human involved in every control-plane reboot for OS patching
The applications on it cannot tolerate a frozen control plane for the restore duration

If you are choosing ha today, tell us before you run it. The tier is built, and the install around it is proven on a single server. The first install on three real machines has not happened yet, and it should not be one nobody is watching.

Take single-server when:

The workloads tolerate a rebuild-and-restore outage
The restore time, measured on your own data volume, is acceptable to the business
Cost matters more than the difference, which for three control-plane nodes is not small
Someone will actually be available to act on a frozen cluster

There is no shame in single-server. Plenty of production workloads are fine with it. What is not fine is choosing it while believing it is highly available, which is why this page is written the way it is.