Files
monky-deployd/CHANGELOG.md
T
mdella 43615a6fda fix(onboarding): identity read that survives a rewrite, disk refused before the pull — 0.1.8
Three faults from one agent-managed onboarding (env-dev-08, 2026-09-09), each of which pointed
the diagnosis away from the actual fault.

1. install.sh granted the agent's read on the ziti identity with a POSIX ACL. ziti-edge-tunnel
   rewrites that file on a controller config update and the rewrite drops the ACL: the agent
   applied cleanly at 01:21 and was failing every tick by 01:32. Group membership survives the
   rewrite (the file stays ziti:ziti 0640), so install.sh and the package postinstall now add
   monky-deployd to the `ziti` group, and a default ACL on the identity directory carries the
   grant onto a newly created file. The explicit ACLs stay for the boxes that need them.

2. openziti.load() accepts an unreadable or malformed identity: the C SDK logs "configuration is
   invalid" and returns a context that only fails at dial, as a bare TypeError, which the
   transport reported as a missing intercept or a policy gap. The SDK transport now reads and
   parses the identity itself and names the real fault first.

3. The disk pre-flight ran only when the bundle declared disk_need_bytes, so a bundle without one
   died mid-pull with containerd's "no space left on device" — which reads as a registry fault.
   A bundle that declares no size now has to clear the headroom floor, and the pre-flight measures
   containerd's root as well as the docker data-root: docker 29 keeps image layers in the
   containerd image store, and on env-dev-08 those sat on different filesystems (93 GiB free where
   the agent looked, 2.8 GiB where the pull wrote).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-09 01:48:49 +00:00

9.0 KiB
Raw Permalink Blame History

Changelog

0.1.8 — onboarding: keep the identity readable, refuse a full disk (2026-09-09)

Three faults from one onboarding (env-dev-08, agent-managed, 2026-09-09), each of which sent the diagnosis somewhere other than the fault.

  • The agent's read on the ziti identity now survives a tunneller rewrite. install.sh granted it with a POSIX ACL. ziti-edge-tunnel rewrites the identity file whenever the controller sends a config update, and the rewrite drops the ACL — the agent had applied cleanly and was failing every tick six minutes later. The durable grant is group membership: install.sh and the package's postinstall now add monky-deployd to the identity file's group (ziti) when it exists, and a default ACL on the identity directory carries the grant onto a freshly created file. The explicit ACLs stay.
  • An unusable identity says so. openziti.load() accepts an unreadable or malformed identity, logs "configuration is invalid" at the C layer and hands back a context that fails LATER, at dial, as a bare TypeError — which the transport reported as a missing intercept or a policy gap. The SDK transport now reads and parses the identity itself first, and names the real fault (absent, not readable by this uid, or not JSON) before anything touches the mesh.
  • A full disk is refused before the pull, and the right disk is measured. The pre-flight ran only when the bundle declared disk_need_bytes; a bundle without one sailed past it and died mid-pull with containerd's no space left on device, which reads as a registry fault. A bundle that declares no size now has to clear the headroom floor. The pre-flight also measures containerd's root as well as the docker data-root: docker 29 keeps image layers in the containerd image store, and on env-dev-08 those were on different filesystems — 93 GiB free where the agent looked, 2.8 GiB where the pull wrote.

0.1.7 — registry credential + a pull error you can read (2026-09-08)

  • No hand docker login on a new box (monky-design-docs !225, doc 24 §4a). A manifest entry marked use: registry-auth is the Harbor pull credential, not an env var: it is written to <state_dir>/docker/config.json (0600, in a directory the agent owns) and the docker CLI is pointed at it with an explicit DOCKER_CONFIG. That last part is the trap — the unit runs as monky-deployd, so a docker login by a human or by root is invisible to the agent and looks exactly like no credential at all (env-dev-01, 2026-09-08). Both seeded shapes are accepted: a JSON object, or username:password with the registry from the new registry_host config.
  • compose pull failures carry the registry's own message instead of rc=1. "no basic auth credentials", "manifest unknown" and DNS failures now reach the journal and the report to tenancy.

0.1.6 — 2026-09-07

  • A valid lease token is reused across applies. Every apply requested a new lease, so a failing deploy retried by the 60 s timer burned tenancy's 5-leases-per-hour budget and then failed on LEASE_RATE_LIMITED forever (env-qa-02 pilot). Now: reuse a lease-derived token while lookup-self says it is valid; swap the bootstrap token for a lease once; and if tenancy rate-limits the lease while a working token exists, apply with it and defer the swap instead of failing the deploy.

0.1.5 — 2026-09-07

  • tenancy.port defaults to 443 everywhere — the ansible role default and the TenancyCfg default now match install.sh/config.example.yaml (0.1.5 only changed those two); PROTOCOL.md states the intercept port separately from the in-pod 8081, that the broker does not add #monky-deploy-agent yet, that the jwt mount keeps no jti state, and that release:gitea has not run (doc-drift DD-0523/0525/0527/0528).

  • ${VAR} inside comment lines is not a reference. The renderer's .env.template header literally says "substitutes every ${VAR}", which the refusal check counted as an unresolved variable (ENV_INCOMPLETE: unresolved: VAR) — the first bundle on env-qa-02 was refused for it.

  • config: tenancy.port is the service's intercept port (443), not the in-pod 8081. With 8081 the SDK found no intercept and the dial failed (service not available, then a bare TypeError from the SDK's fallback). install.sh now writes 443; config.example.yaml updated.

  • transport sdk: clear error instead of a TypeError when an address has no intercept or the identity has no dial policy for the service (TransportError names the host:port and what to check).

0.1.4 — 2026-09-07

  • install.sh: ziti-edge-tunnel.service failed to start after enrolment (203/EXEC: the package's ExecStartPre script is not executable by user ziti on ziti-edge-tunnel 1.18.x) and, because the unit was enabled with --now under set -e, the script aborted BEFORE writing config/ACLs/grant/timer — a second kit was needed. The run-host drop-in now clears ExecStartPre=; a tunneler start failure is a WARNING until everything else is in place, then a hard error at the end.

0.1.3 — 2026-09-06

  • install.sh: --version was clobbered by /etc/os-release (it defines VERSION), so the kit tried to download monky-deployd_26.04 LTS (Resolute Raccoon)_amd64.deb (env-qa-02 pilot, first real kit run). Script variables are now DEPLOYD_VERSION; os-release is read in a subshell.
  • install.sh installs Docker Engine when absent (Docker's apt suite for the codename → fallback docker.io), with --docker-data-root <dir> to place the data-root before first start. One script for the end user, per the operator's rule.

v0.1.2 — 2026-09-05

  • The GitLab project is private (its parent groups are private, so it cannot be made public; found when the v0.1.1 one-liner returned 401 anonymously). install.sh gains --token <deploy-token> / MONKY_DEPLOYD_TOKEN: every download from the generic package registry — the script itself included — sends DEPLOY-TOKEN: <token>, a read-only GitLab deploy token (scope read_package_registry only, revocable), seeded in OpenBao at monky/monky-tenancy/deployd-download (key token) and handed to the box by the monky-tenancy install kit. The token never reaches the command line, the log or an xtrace (curl -K config file, 0600, deleted after the download; set +x forced). The one-liner now fetches install.sh from the registry (…/monky-deployd/<ver>/install.sh) instead of -/raw/main, and takes the bootstrap grant via --bootstrap-file — with curl … | bash -s -- stdin IS the script, so it is no longer read for the grant in that mode. --source gitea (no token) stays the off-estate alternative.
  • Ansible role: monky_deployd_download_token (vaulted) → DEPLOY-TOKEN header on both fetches, no_log: true.
  • README / OPERATIONS: why (private project; split-horizon Gitea, cbs/iac#102), token scope, revocation. No agent behaviour change.

v0.1.1 — 2026-09-05

  • install.sh / ansible role / README: the primary download is the public GitLab project's generic package registry on scm.tikali.ai (/api/v4/projects/69/packages/generic/monky-deployd/<ver>/…); the one-liner fetches the script from scm.tikali.ai/…/-/raw/main/packaging/install.sh. Inside the estate gitea.cbs.tikali.net is split-horizon to jump1's RED EIP (no HTTP ingress), so backend boxes could not fetch the artefacts (cbs/iac#102). The Gitea release stays the off-estate alternative (--source gitea / MONKY_DEPLOYD_SOURCE=gitea). No agent behaviour change.

v0.1.0 — 2026-09-05

First release (MONKY-ADR-0028 §D, Reconciliation v2, Gate 1 v2).

  • Stdlib-only Python 3.12 agent: run --once|loop, status, bootstrap, --prune.
  • Protocol: POST /v1/agent/checkin (action apply|none|down), GET /v1/agent/bundle/{env}/{sha} (sha256 verified), POST /v1/agent/leasedeploy grant (login_jwt), POST /v1/agent/report (result applied|failed|down, redacted log_tail). An AppRole-shaped lease is refused (LEASE_SHAPE).
  • OpenBao: POST /v1/auth/jwt-tenancy/login {"role":"see-env","jwt":…}; KV-v2 reads pinned to the manifest's versions, paths pinned to monky/data/<env>/see/; renew-self / re-lease before max TTL.
  • Refusals: ENV_INCOMPLETE (names only), PRIVILEGED_REFUSED, ROLLBACK_REFUSED, DISK_INSUFFICIENT (need × 1.5 + 2 GiB vs docker data-root), BUNDLE_SHA_MISMATCH, BUNDLE_ENV_MISMATCH.
  • Transports sdk (openziti SDK), proxy (ziti tunnel proxy 18443/18200), system.
  • Exit codes 0 / 75 / 78 (AGENT_ENV_MISMATCH, no retry storm) / 1; laptop mode (offline exits 0).
  • Packaging: hardened monky-deployd.service oneshot + 60 s timer, monky-deployd-proxy.service, nfpm .deb with /opt/monky-deployd/venv, packaging/install.sh (Ubuntu 26.04), ansible role skeleton.
  • Known divergence: monky-tenancy main (MR !15) still ships the AppRole lease/kit; the JWT-grant follow-up is the tenancy side of this release (docs/PROTOCOL.md "Divergences").