mirror of
https://scm.tikali.ai/tikali/applications/monky/monky-deployd.git
synced 2026-09-18 04:36:15 +00:00
fix(onboarding): identity read that survives a rewrite, disk refused before the pull — 0.1.8
Three faults from one agent-managed onboarding (env-dev-08, 2026-09-09), each of which pointed the diagnosis away from the actual fault. 1. install.sh granted the agent's read on the ziti identity with a POSIX ACL. ziti-edge-tunnel rewrites that file on a controller config update and the rewrite drops the ACL: the agent applied cleanly at 01:21 and was failing every tick by 01:32. Group membership survives the rewrite (the file stays ziti:ziti 0640), so install.sh and the package postinstall now add monky-deployd to the `ziti` group, and a default ACL on the identity directory carries the grant onto a newly created file. The explicit ACLs stay for the boxes that need them. 2. openziti.load() accepts an unreadable or malformed identity: the C SDK logs "configuration is invalid" and returns a context that only fails at dial, as a bare TypeError, which the transport reported as a missing intercept or a policy gap. The SDK transport now reads and parses the identity itself and names the real fault first. 3. The disk pre-flight ran only when the bundle declared disk_need_bytes, so a bundle without one died mid-pull with containerd's "no space left on device" — which reads as a registry fault. A bundle that declares no size now has to clear the headroom floor, and the pre-flight measures containerd's root as well as the docker data-root: docker 29 keeps image layers in the containerd image store, and on env-dev-08 those sat on different filesystems (93 GiB free where the agent looked, 2.8 GiB where the pull wrote). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
This commit is contained in:
+9
-2
@@ -22,7 +22,13 @@ monky-deployd status --json | jq .
|
||||
Files: `/etc/monky-deployd/config.yaml` (0640 root:monky-deployd), `/etc/monky-deployd/openbao-ca.pem`,
|
||||
`/etc/monky-deployd/bootstrap.jwt` (only until the first login), `/var/lib/monky-deployd/{bao.token,state.json,lock}`,
|
||||
`/var/lib/monky-deployd/<env>/releases/<sha>/` + `current` (the compose project dir, `.env` 0600),
|
||||
`/opt/openziti/etc/identities/monky-host.<env>.json` (ziti:ziti 0600 + ACL `u:monky-deployd:r`).
|
||||
`/opt/openziti/etc/identities/monky-host.<env>.json` (ziti:ziti 0640; the agent reads it as a member of
|
||||
the `ziti` **group**, plus an explicit ACL `u:monky-deployd:r` and a default ACL on the directory).
|
||||
|
||||
> The group is the grant that matters. `ziti-edge-tunnel` rewrites the identity file whenever the
|
||||
> controller sends a config update, and the rewrite drops POSIX ACLs — an ACL-only box works until the
|
||||
> first refresh and then fails every tick (env-dev-08, 2026-09-09). On a host where the tunneller was
|
||||
> installed independently of the kit, check `id monky-deployd` for `ziti` before anything else.
|
||||
|
||||
## Reading the journal
|
||||
|
||||
@@ -31,9 +37,10 @@ Files: `/etc/monky-deployd/config.yaml` (0640 root:monky-deployd), `/etc/monky-d
|
||||
| `checkin: action=none desired=… applied=…` then `healthy; heartbeat reported` | converged |
|
||||
| `checkin: action=apply …` → `read 3 secret(s): GEMINI_API_KEY, …` → `promoted release …` → `applied …` | a deploy |
|
||||
| `refused: ENV_INCOMPLETE: unresolved: X` | the bundle needs a variable no manifest entry supplies — fix the descriptor / set the secret in the console; nothing was started |
|
||||
| `refused: DISK_INSUFFICIENT: docker data-root has N MiB free, bundle needs M MiB` | free space (the env-dev-09 lesson): grow the data-root disk or prune |
|
||||
| `refused: DISK_INSUFFICIENT: image storage (…) has N MiB free, …` | free space: grow the disk or prune. The paths in the message are every filesystem a pull can fill — docker's data-root AND containerd's root, which docker 29 fills with image layers and which is often a different disk (env-dev-08). A bundle that declares no size must still clear `disk.headroom_bytes` |
|
||||
| `refused: PRIVILEGED_REFUSED` / `ROLLBACK_REFUSED` | the bundle needs `allow_privileged` / `allow_rollback` in its `agent` profile |
|
||||
| `temporary network failure` (exit 75) | mesh/tenancy unreachable — check `ziti-edge-tunnel`, the identity's terminators, `monky.tenancy.deploy` health |
|
||||
| `transport sdk: identity … is not readable by this user` | the agent lost its read on the identity file (usually a tunneller rewrite dropping the ACL) — `usermod -a -G ziti monky-deployd` |
|
||||
| `AGENT_UNAUTHENTICATED: bearer refused` (exit 1) | the grant was superseded (kit re-revealed / retire) or the token revoked → re-run the install kit |
|
||||
| `AGENT_ENV_MISMATCH` (exit 78) | the token belongs to another env than `config.yaml` — fix the config or re-issue the identity; the timer keeps firing but every tick exits 78 immediately (no storm) |
|
||||
| `failed: docker compose pull failed (rc=1)` | registry/pull problem; compose output is in the report's tail and in the journal |
|
||||
|
||||
Reference in New Issue
Block a user