deployd#3 (DD-0620): every kit for a backend registered since 2026-09-08 died at `--site`.
env-dev-08 (2026-09-11..13): two days of "identity is not readable" ticks — ziti-edge-tunnel
re-creates the file with mode 0600, the ACL mask goes to ---, group membership stops helping.
identity-acl.sh + monky-deployd-identity-acl.path re-apply the grant on every directory change.
Doc-Drift: DD-0620 fixed
Closes#3
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASnneBmT7rfaJLE8NGNw7S
dpkg calls the OLD package's prerm on an UPGRADE as well as on a removal (rpm passes a
remaining-instance count), and preremove.sh ran `systemctl disable --now monky-deployd.timer`
unconditionally. Upgrading env-dev-01 and env-dev-08 to 0.1.8 today stopped and disabled both
agents.
The failure is silent, which is the dangerous part: the box stays reachable, the containers keep
running, and nothing reports that check-ins have ceased — the backend just stops converging. A
fleet upgrade would have taken every agent offline at once and looked like a success.
preremove.sh now returns early for every upgrade shape (upgrade, failed-upgrade, deconfigure,
rpm's 1) and only disables on a real removal. postinstall.sh try-restarts the long-lived proxy
unit so it picks up the new code; the timer needs nothing, since each tick is a fresh process.
Tests drive the script with a fake systemctl on PATH and assert an upgrade touches no units.
OPERATIONS.md warns that a box coming FROM 0.1.8 or earlier still needs its timer re-enabled by
hand, because the old prerm has already run by then.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
Three faults from one agent-managed onboarding (env-dev-08, 2026-09-09), each of which pointed
the diagnosis away from the actual fault.
1. install.sh granted the agent's read on the ziti identity with a POSIX ACL. ziti-edge-tunnel
rewrites that file on a controller config update and the rewrite drops the ACL: the agent
applied cleanly at 01:21 and was failing every tick by 01:32. Group membership survives the
rewrite (the file stays ziti:ziti 0640), so install.sh and the package postinstall now add
monky-deployd to the `ziti` group, and a default ACL on the identity directory carries the
grant onto a newly created file. The explicit ACLs stay for the boxes that need them.
2. openziti.load() accepts an unreadable or malformed identity: the C SDK logs "configuration is
invalid" and returns a context that only fails at dial, as a bare TypeError, which the
transport reported as a missing intercept or a policy gap. The SDK transport now reads and
parses the identity itself and names the real fault first.
3. The disk pre-flight ran only when the bundle declared disk_need_bytes, so a bundle without one
died mid-pull with containerd's "no space left on device" — which reads as a registry fault.
A bundle that declares no size now has to clear the headroom floor, and the pre-flight measures
containerd's root as well as the docker data-root: docker 29 keeps image layers in the
containerd image store, and on env-dev-08 those sat on different filesystems (93 GiB free where
the agent looked, 2.8 GiB where the pull wrote).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
!9 (d34189c6) rewrote PROTOCOL.md §Where and how and README §Security model
to say monky-ziti does not add #monky-deploy-agent / #openbao-client yet and an
operator adds them by hand. That was verified against monky-ziti at the sweep
HEAD b44c50a4 (2026-09-05), not its live HEAD: monky-ziti ac258352 (2026-09-07
06:12Z, "feat(environments): host_extra_attrs — host-template-only attrs for
the agent's dial right", merged a82359dd) and monky-tenancy !29 9b84f9ee
(06:23Z, app/jobs_backends.py sends host_extra_attrs for non-k8s backends)
landed hours before !9 was opened. Both sentences now state what happens:
tenancy sends the attrs at register, the broker puts them on the host identity,
and only pre-!29 identities (env-qa-02's) were hand-added.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AW3QqEpwLV69KHn24Re45Q
DD-0523 — !7 (31586c30, 0.1.5) moved install.sh and config.example.yaml to the
443 intercept after env-qa-02 hit "service not available". The ansible role
default (monky_deployd_tenancy_port) and TenancyCfg.port still said 8081, so
an ansible-installed box or a config that omits `port` still dialled the wrong
port; both now default to 443, the proxy-mapping and config tests follow, and
PROTOCOL.md §Where and how states the intercept port separately from the
in-pod 8081 and names openziti state/overlay/configs.json as the authority.
DD-0525 — PROTOCOL.md and README said "the broker adds the attr when the
identity is created at kit reveal". monky-ziti at b44c50a4 has no such code
(app/fabric.py host_identity_attrs carries the env template only) and openziti
docs/services.md says "Nothing carries the attr yet". Both now state the
dependency: an operator adds #monky-deploy-agent/#openbao-client on the
controller until the ADR-0028 addendum lands in monky-ziti.
DD-0527 — "the old kit's grant fails at login (unknown/used jti)". The
jwt-tenancy mount keeps no replay state (openbao terraform/jwt-tenancy.tf
see_env role: signature, aud, bound_claims, exp); a superseded grant logs in
until exp and the refusal is tenancy's 401 on the first bearer call. The
second-reveal paragraph, the grant-flow diagram and README §Security model say
so; FakeBao no longer pops a grant at login (the suite's superseded-token test
already goes through FakeTenancy.superseded_jtis, which is the real model).
DD-0528 — "Both locations keep being published": release:gitea has been a
never-run manual job on every tag pipeline (6999, 7044, 7066); README and
OPERATIONS.md now say when the Gitea mirror is published and that it has not
been yet.
Gates (local, py3.12): ruff format, ruff check, pytest 50 passed,
bash -n packaging/install.sh. `git grep 8081` afterwards hits only the in-pod
listener statements.
Doc-Drift: DD-0523 fixed
Doc-Drift: DD-0525 fixed
Doc-Drift: DD-0527 fixed
Doc-Drift: DD-0528 fixed
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AW3QqEpwLV69KHn24Re45Q
DD-0524 — PROTOCOL.md §Divergences bullets 1-2 said monky-tenancy `main`
"still implements the AppRole lease and install kit" and that "the kit's
generated config uses tenancy.base_url". Both were false when the page was
published: tenancy !17 (288df791, merged 07:48Z) shipped AgentLeaseOut{env_id,
login_jwt, ttl_s, mount, role, addr}, the ES256 grant signer and the JWKS
sixteen minutes before v0.1.0 was tagged, and !22 (61bd0281) made the kit run
install.sh with flags instead of writing a config. The two bullets are now
dated "Resolved" notes; bullets 3-4 (report `detail`, X-Bundle-Sha) stand.
DD-0526 — the lease example sent `reason` and received a nested `vault{}`;
tenancy's AgentLease is `{env_id}` and AgentLeaseOut carries `addr` at the top
level (no vault object). The example now shows tenancy's shapes (with a note
that the agent still sends `reason` and tenancy ignores unknown fields), the
checkin example gains `auth_mount`/`auth_role` so it is the full AgentVaultOut,
tests/fakes.py emits `addr` the way tenancy does, and tenancy.py's docstring
and lease() read `addr` first (the `vault{}` fallback is kept so an older fake
or tenancy still leases). Verified against monky-tenancy app/schemas_backends.py
at 1fd51454 (AgentLease 296-297, AgentLeaseOut 300-309, AgentVaultOut 278-283).
Gates (local, py3.12): ruff format --check, ruff check, pytest 50 passed,
bash -n packaging/install.sh.
Doc-Drift: DD-0524 fixed
Doc-Drift: DD-0526 fixed
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AW3QqEpwLV69KHn24Re45Q
The GitLab project is private (its parent groups are private, so it cannot be
made public): the v0.1.1 one-liner answered 401 anonymously. install.sh gains
--token / MONKY_DEPLOYD_TOKEN and sends `DEPLOY-TOKEN: <token>` (a GitLab deploy
token, scope read_package_registry only, revocable) on every registry download,
the script itself included; the token goes through a 0600 curl -K file (never the
command line, the log or an xtrace). The grant is taken via --bootstrap-file when
the script is piped (stdin IS the script). Ansible: monky_deployd_download_token
(vaulted) -> DEPLOY-TOKEN header, no_log. Docs explain why, the token's scope and
the --source gitea alternative (split-horizon Gitea, cbs/iac#102).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
Inside the estate gitea.cbs.tikali.net resolves to jump1's RED EIP (10.10.0.175),
which has no HTTP ingress, so backend boxes could not download the install
artefacts from the Gitea mirror (cbs/iac#102). scm.tikali.ai is reachable from
those boxes and the project is now public, so the GitLab generic package
registry becomes the PRIMARY source:
- packaging/install.sh: default source = scm.tikali.ai generic package registry
(projects/69/packages/generic/monky-deployd/<ver>/...); `--source gitea` /
MONKY_DEPLOYD_SOURCE=gitea keeps the Gitea release as the off-estate
alternative; --base-url / MONKY_DEPLOYD_BASE_URL still override the base.
- ansible role defaults: monky_deployd_base_url/_deb_url point at the registry,
Gitea layout kept as a commented alternative.
- README / docs/OPERATIONS.md / CLAUDE.md / CI comments + release description:
both locations keep being published (release + release:gitea).
- Version 0.1.1 (the tag gate refuses v* tags whose version != __version__);
tests compare against __version__ instead of a literal. No agent change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
mdella
Claude-Docs-Manager