swarmexec

swarmexec › Docs › Authentication & policy

Authentication & policy_

Who may talk to the agents, and what they may do.

Authentication

The client authenticates to the agent in one of two modes.

Mode A — mutual TLS (default)

Set ca, cert and key (all three required). The client verifies the agent against your CA and presents its certificate; the agent authorises and audits you by the certificate's CN. This is the default and the recommended posture.

$ swarmexec --ca ca.crt --cert me.crt --key me.key ps

Mode B — self-signed agent + shared secret

Simpler to run (one secret, no PKI) — this is what init sets up. Set agent_secret (or agent_secret_file), and either a ca to verify the agent or insecure: true to skip verification. A client cert is optional (but cert and key must be set together or both empty). Your audit identity is the operator value (default: your OS username).

$ swarmexec --agent-secret "$SECRET" --insecure ps
insecure skips the agent's server-certificate check. It does not put the secret at risk: the client never sends it, only a proof computed from the secret and the certificate of the connection it travels on, so a server you did not verify receives nothing that works against a real agent. What remains is that the agent is not authenticated to you — whoever answers can read what that session sends them. Prefer a ca on any network you do not trust, and keep the secret out of your shell history (use agent_secret_file or the config file).

Upgrading to v1.17.3 — update the agents first

A v1.17.3 client cannot authenticate to an older agent. It sends only the connection-bound proof, and agents before v1.17.3 do not recognise that form — they answer invalid or missing agent secret even though your secret is correct. Update them first:
$ swarmexec init --force
While a fleet is mixed, legacy_secret: true in the client config also sends the raw secret, with the exposure described above. Remove it once the agents are current, then re-run swarmexec init --force --allow-legacy-secret=false so no client can put the credential on the wire by accident.

When the agent rejects the client

The message names the credential you are configured with. In Mode B it reads agent rejected the shared secret, with the v1.17.3 hint above. In Mode A a certificate the agents' CA does not trust already fails at the TLS handshake (cannot reach agent … mTLS/connection error); a trusted certificate the agent cannot take an identity from (no CN) reads agent did not accept this client's certificate identity, followed by the agent's reason. A Mode A client pointed at Mode B agents is told the agents require a shared secret and that this client has none configured.

Authorization policy

By default an agent allows every request that passed authentication. A rule file narrows that: which actions (container.list, container.events, exec, logs, portforward, stats.read, image.list, image.prune, image.prune.all, volume.list, volume.create, volume.remove, network.inspect (a node's view of its overlay networks' endpoints, read-only), reclaim.read, container.prune, buildcache.prune, network.prune, volume.prune, inspect (the node-local container inspect, including the container's environment), kill (SIGKILL a swarm task container, which Swarm then replaces), copy.from (read a file out of a container) and copy.to (write one into it, the sharpest permission the agent has); globs like volume.* work) for which services, ports, exec users, volumes and — with client certificates only — operator identities. Within a rule every condition must hold; the first rule that matches decides, and default (required) covers the rest. The agent's version probe is not an action and is never refused by a policy, so doctor can still report an agent's version to a denied identity.

# policy.yaml — the first rule that matches decides; "default" when none does default: allow rules: - deny: [image.prune.all] reason: no sweeping prune on production - deny: [portforward] ports: [5432, 3306] reason: no forwards to the databases - deny: [exec] users: [root, "0"] reason: no root shell - deny: [volume.remove, volume.create] volumes: ["prod_*"] reason: production volumes stay - deny: [copy.to] reason: files may be read out, never written in
swarmexec init --force --policy-file policy.yaml

A refusal reaches the client as permission denied with the rule and its reason, and the same text lands in the audit log, in an auth_decision record that also names the requested action and its object (container_id, service, and where they apply port, user, volume, client_addr; never the command line or environment). Allowed reads a client repeats on a timer (stats.read, volume.list and the like) are recorded once per identity and action per minute with a suppressed count; refusals are never thinned out. A failed shared-secret authentication is counted (swarmexec_auth_failures_total) and logged as a warning, throttled per peer. init checks the file before deploying anything, and the agent refuses to start on one it cannot read: an unknown action, a misspelt key or a broken pattern is an error, never a rule that silently matches nothing. Rules on identities are refused while the agents use a shared secret without client certificates — the identity is then whatever the client claims. Change the file and re-run init --force; run it without the flag to drop the policy.

Which condition applies to which action

An agent only knows a fact where the request carries it, so a condition on an action that never carries it could never hold. Such a rule is an error (rule 4: services never applies to volume.list, volume.create, …), and an agent whose policy file has one refuses to start. A condition that applies to at least one action the rule names is fine: deny: ["*"] with services: ["prod_*"] covers every container action of those services.

ActionConditions it can carry (besides identities)
execservices, users
portforwardservices, ports
logs, container.events, kill, inspect, copy.from, copy.toservices
volume.create, volume.removevolumes (glob on the volume name; the volume's consumers are not looked up)
every other action: container.list, stats.read, image.*, volume.list, network.inspect, reclaim.read, all *.prunenone — only identities

Upgrading: a policy that has a rule like deny: ["volume.*"] with services: [...] never matched anything. It is now refused when the agent starts (and by init before it deploys). Replace the condition by volumes: ["prod_*"].

Exec users

users is matched against the user the process will run as: the -u you gave, otherwise the user the image is configured with (USER), otherwise root. The group part (:gid) is split off, and leading zeros of a numeric id are dropped, so root:root, 0:1 and 00 are all caught by users: [root, "0"]. A pattern that names a full user:group ("0:0") matches that exact pair. The agent cannot map a name to an id inside the image, so root and 0 are two entries: list both. An image that switches user in its entrypoint instead of USER is not visible to the agent. users: [""] still matches an exec that passed no -u, whatever the image runs as.

kill and inspect deliberately sit outside the container.* glob: a rule written as allow: ["container.*"] for the listing and the event watch does not start allowing anyone to stop a workload or read a container's environment. Only "*" or the action's own name covers them. Both act on swarm task containers only, need an agent from the release that introduced them, and are audited (container_kill, container_inspect — the inspected document itself is never logged).

What this is not. A policy prevents accidents and records intent — no sweeping prune on production, no forward onto the database. It is not access control: everything swarmexec changes on services, secrets, networks and stacks goes to the swarm manager with your own Docker credentials and never passes an agent, and whoever holds those can deploy a service that is root on every node. Restrict the Docker credentials for that.