swarmexec › Docs › TUI: the Nodes tab
TUI: the Nodes tab_
Every node, what it holds, and what it could give back.
Nodes tab
Lists the swarm's nodes with general info and a few aggregations — NODE
(hostname; managers in aqua, the leader marked ★), ROLE
(manager / worker), AVAIL (active / pause / drain, colour-coded),
STATE (ready / down), CPU% and MEM% (what the
node's containers use, as a share of the node's cores and memory — measured by its
agent on the same pass that puts usage badges on the tree, so containers only, not
the kernel or anything outside docker; yellow from 80 %, red from 90 %,
– when the agent did not answer and … while it has its
first CPU sample only), TASKS
(running tasks scheduled on the node), VOLS (volumes the node holds —
node-local, so it fills in a moment after the rest; – when the node's
agent did not answer, rather than a 0 that would read as “no volumes”),
ENGINE version and LABELS (count). The ? overlay
explains the ★ too.
Every action on this tab has a command-line twin under
swarmexec node: node ls
for the list, node prune, node update --availability /
--label-add / --label-rm, and node prune-schedule.
| Key | Action |
|---|---|
| Enter / i | node details — a read-only overlay with hostname, id, role/leader, availability, state, address, engine, platform, CPUs, memory, running-task count, the reserved resources, in-use and images on this node blocks (see below), volume count, the full label list, and the reclaimable and prune schedule blocks. Inside it M measures the reclaimable space now, P opens the prune dialog and S the schedule editor |
| l | edit the selected node's labels — a staged list editor (a add / e edit / d delete / Ctrl-S or w apply / Esc cancel) over key=value entries. Applying does one NodeUpdate — unlike a service, a node update applies immediately (no rolling update). Node labels are commonly used as placement-constraint targets (node.labels.<k>). If the node changed after the tab last loaded it, the apply is refused (“node … was changed by someone else … nothing was written”) and your staged labels stay on screen; refresh the tab and apply again |
| a | set the node's availability — a menu of active / pause / drain that marks the state the node is currently in. active schedules tasks here normally; pause keeps the running tasks and only stops new placements; drain moves every task off the node. Active and Pause apply straight away; Drain goes through a confirm naming how many running tasks swarm will stop and reschedule elsewhere (services whose tasks can't be placed anywhere else go unschedulable). Like a label change it is one NodeUpdate (read-modify-write on the node spec) and applies immediately — nodes have no rolling update. Remappable as node_availability |
| P | prune the node (capital P, drawn red). Opens a dialog with one checkbox per category, the expected savings, then a confirm and the result per category — see pruning a node. Against an agent older than this release it opens the old two-entry image menu instead. Remappable as node_prune (the old name node_prune_images still works) |
| S | the prune schedule for all nodes — see prune schedule. Remappable as node_prune_schedule |
Reserved resources. The node detail carries a
reserved by tasks (scheduler's view) block: for CPU and
memory, a bar plus reserved / capacity and how much is
free. The bar is green, turns yellow from 75% and red at 100% or above;
over-commitment is called out in words, and “free” never goes negative. A node that
reports no capacity says so instead of drawing a bar. The block costs no extra API
call — the nodes list already fetches the task list.
Reservations — exactly the arithmetic the swarm
scheduler itself does when deciding whether a task fits on a node, and the usual
reason a service sits unschedulable. Live CPU/RAM consumption is node-local and is
not what this shows — it has a block of its own, right below. Tasks that
declare no reservation are
counted and reported separately: they book nothing here yet can
still consume the whole node, so a low percentage does not mean the node is idle —
and in most clusters few services set reservations at all. A task holds its
reservation from the moment it is assigned until it reaches a terminal state, so
tasks that are preparing or starting count, while shutdown / failed / complete /
rejected ones do not (swarm keeps those in the task list as history).
In use by containers. Beneath that sits a second block — in use by containers (measured on the node) — drawn with the same proportional bars: what the node's containers are actually using right now, CPU in cores and memory, each against the node's own capacity. The two blocks are kept deliberately apart because they answer different questions and are routinely far apart: a node can be fully booked and idle, or barely booked and on fire. The figures come from that node's agent, the same source as the usage markers in the service tree — while the CPU delta isn't ready yet the line reads measuring…, and a node whose agent is unreachable or too old gets no block at all rather than a row of zeros that would read as “idle”.
Images on this node. Last in the detail comes an images on this node block: how much the node's layer store holds and in how many images, and how much of that is reclaimable in untagged leftovers (a node with nothing untagged says so rather than offering a zero). When there is any, one further line notes how much more is tagged but unused — phrased as a cost, removing it means pulling those images again, not as an offer. The block exists because images are the one node-local resource the cluster side cannot see at all: the manager API has no view of images whatsoever, unlike volumes, which it at least knows are mounted. Volumes could already be managed cluster-wide from swarmexec; a node quietly filling up with old layers could not even be seen. It loads on its own, in parallel with the node list, so it appears a moment after the rest of the overlay.
Reclaimable space and pruning a node. The node detail ends with a reclaimable (measured on the node) block: per category, how many objects a prune would remove and how much it would free, and when that was measured (measured 23 min ago). The agent measures in the background, shortly after it starts and then hourly, and after every prune; m in the detail asks for a fresh measurement (the agent allows one every 30 seconds). P opens the prune dialog: one checkbox per category with its figure, the expected total underneath, then a confirm that lists every ticked category, and finally the result — freed bytes and count per category.
| Category | What a prune removes | Policy action |
|---|---|---|
| stopped containers | containers that stopped more than 24 h ago (t in the dialog changes the age). Counted from when they stopped, not when they were created, so the logs of a task that just crashed stay readable. The detail shows how many are kept for now | container.prune |
| untagged images | the layers left behind by rebuilds, which nothing can start from. Safe: it cannot stop a service and it cannot force a pull | image.prune |
| every unused image | also removes tagged images that no container is running right now — every service scaled to zero and every task between restarts will have to pull its image again | image.prune.all |
| build cache | all build cache no build is using | buildcache.prune |
| unused local networks | networks of scope local that no container references, stopped ones included. Never bridge, host, none, docker_gwbridge, ingress or any overlay | network.prune |
| volumes | every volume no container uses, named volumes included — their data is deleted | volume.prune |
The dialog starts with containers, untagged images, build cache and networks ticked. a ticks everything except volumes; volumes are a separate, red checkbox that is never ticked for you. The two image entries exclude each other, because every unused image already includes the untagged ones.
One permission per category. The agent authorizes each category as
its own action (table above; reading the figures is reclaim.read), so
a policy can allow the cheap sweeps without the destructive ones. Every
category of every prune is written to the agent's audit log
(event=prune) with how much it freed and how many objects went.
docker system df reports from, and match it exactly — verified on a
real node at 56 images, 32.70 GB on disk, 22.99 GB reclaimable. In
particular swarmexec does not add up the individual image sizes: an
image's size includes every layer it is built from, and layers are shared, so that
sum over-reports badly — during development it claimed 32.7 GB where the daemon
said 22.2 GB. One caveat worth stating: the untagged-only figure is a
lower bound — a layer shared by two untagged images belongs to
neither one's unique size — so an untagged prune may free a little more
than advertised, never less.
Same precondition as the usage and health numbers above: it needs
agents from this release or newer. A node whose agent is older
shows agent too old for prune stats in the reclaimable block, and
P falls back to the two-entry image menu (untagged leftovers / every
unused image) such an agent still understands. Update the agent image — the
agent service spec did not change, so init --force is not needed.
A node whose agent does not answer at all shows the first cause on one line —
agent not reachable on 10.0.1.5:9443: connection refused — swarmexec
doctor — and the full error goes to the client log (`).
Prune schedule (all nodes). S on the Nodes tab opens an editor for a schedule every agent runs on its own node: weekday toggles, the start time, runs per day (the day is split evenly from the start: 2 runs from 00:00 are 00:00 and 12:00, 3 are 00:00, 08:00 and 16:00), the time zone, the categories — images as every unused, untagged only or off, and volumes as an option that is off by default — how long stopped containers are kept, and an optional label filter. A live preview shows the resulting run times and the next runs. Ctrl-S (or w) applies after a warning.
Each agent reads the labels when it starts; labels that do not parse are logged
and switch the schedule off, they never stop the agent. A scheduled run is
authorized like a request from the identity scheduler (so a policy
rule such as deny: [volume.prune] also covers the schedule) and
audited with trigger=schedule. The node detail shows the schedule as
that node's agent sees it, the next run, and the last run with what it freed per
category and any error. swarmexec init --force keeps the labels; the
CLI equivalent is node prune-schedule.
| Container label | Default | Meaning |
|---|---|---|
swarmexec.prune.enabled | no labels: off | true runs the schedule |
swarmexec.prune.schedule.days | every day | mon,tue,…,sun, ranges like mon-fri |
swarmexec.prune.schedule.start | 00:00 | first run of the day |
swarmexec.prune.schedule.times | 1 | runs per day (1–24) |
swarmexec.prune.schedule.tz | UTC | IANA time zone |
swarmexec.prune.categories | containers,images,buildcache,networks | also dangling-images and volumes |
swarmexec.prune.containers.until | 24h | keep stopped containers this long after they stop |
swarmexec.prune.filter.label | — | only objects with these labels (key or key=value, comma-separated); build cache is then skipped |