mirror of
https://scm.tikali.ai/tikali/applications/monky/monky-deployd.git
synced 2026-09-18 05:36:15 +00:00
feat: monky-deployd v0.1.0 — pull agent over the mesh (ADR-0028)
Stdlib-only Python 3.12 agent for docker VMs and laptops: flock → checkin
(bearer = the agent's OpenBao token, bootstrapped from the install kit's
jwt-tenancy deploy grant) → action apply|none|down → bundle (sha256
verified) → refusal checks (unresolved ${VAR} names only, manifest paths
pinned to monky/data/<env>/see/, privileged/host-network, rollback, disk
need×1.5+headroom) → lease → POST /v1/auth/jwt-tenancy/login → KV reads →
.env 0600 → promote → compose pull/up → wait healthy → report; finally
renew-self / re-lease before max TTL, scrub. Exit 0/75/78/1. Redactor log
filter. Transports sdk (openziti) / proxy (ziti tunnel proxy 18443/18200) /
system. Laptop mode.
Packaging: hardened oneshot + 60 s timer + proxy unit, nfpm .deb with
/opt/monky-deployd/venv, install.sh for Ubuntu 26.04 (Gitea release
download, enrol, ACLs, bootstrap from stdin), ansible role skeleton for
osg1-07. CI: lint/test on every change; wheel (openziti on ubuntu:26.04) and
package (nfpm) allow_failure until runner egress is proven; GitLab release +
release:gitea on v* tags. Docs: README, PROTOCOL, OPERATIONS, CHANGELOG,
CLAUDE/AGENTS.
Divergence noted: monky-tenancy main (MR !15) still ships the AppRole lease
and kit; this agent implements the plan's Gate 1 RESULT (login_jwt, no
unwrap) and refuses an AppRole lease loudly (LEASE_SHAPE).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
This commit is contained in:
@@ -0,0 +1,148 @@
|
||||
<!-- xlate:verbatim-fences -->
|
||||
# monky-deployd protocol
|
||||
|
||||
The agent side of the backend lifecycle agent protocol — [MONKY-ADR-0028](https://scm.tikali.ai/tikali/applications/monky/monky-design-docs/-/blob/develop/adr/0028-backend-lifecycle-ownership-and-execution.md)
|
||||
§2/§3 (amendment 2026-09-05) and [design doc 24](https://scm.tikali.ai/tikali/applications/monky/monky-design-docs/-/blob/develop/docs/24-backend-lifecycle.md)
|
||||
§2.1a, §3.3, §4. The tenancy side is monky-tenancy `app/agent_main.py` / `app/api/agent.py` /
|
||||
`app/schemas_backends.py`; its shapes are the contract, this page is how the agent uses them.
|
||||
|
||||
## Where and how
|
||||
|
||||
- **Where:** tenancy's **agent entrypoint** — a second container (`app.agent_main`,
|
||||
`127.0.0.1:8081`) bound to the ziti service **`monky.tenancy.deploy`** by a `ziti-edge-tunnel
|
||||
run-host` sidecar. Unreachable from the public ingress. Only host identities carrying
|
||||
`#monky-deploy-agent` may dial (the broker adds the attr when the identity is created at kit reveal).
|
||||
- **Transport:** plain HTTP inside the mesh (`transport: sdk` dials by service name;
|
||||
`proxy` → `127.0.0.1:18443`). OpenBao is the existing `openbao` ziti service (`#openbao-client`),
|
||||
dialled as `https://bao.cbs.tikali.net:8200` (an intercept name), TLS validated against the
|
||||
`openbao-ca` certificate (`bao.ca_bundle`).
|
||||
- **Auth:** `Authorization: Bearer <the agent's OpenBao token>`. Tenancy verifies it with
|
||||
`auth/token/lookup` (cached 60 s), pins the request's `env_id` to the token's `meta.env_id`
|
||||
(`403 AGENT_ENV_MISMATCH` → exit 78, never retried), and compares `meta.grant_jti` against
|
||||
`backend_leases` (`401 AGENT_UNAUTHENTICATED` when the grant was superseded or revoked).
|
||||
- **Rate buckets per env:** checkin 6/min, bundle 10/min, report 30/min, lease **5/h**
|
||||
(`429 LEASE_RATE_LIMITED`, `Retry-After`) → the agent treats 429 as temporary (exit 75).
|
||||
- **Errors** are `{code, detail}` JSON; the agent maps them to `TenancyError` subclasses.
|
||||
|
||||
## Bootstrap → lease → token (the grant flow)
|
||||
|
||||
```
|
||||
kit reveal (admin, once) tenancy signs a bootstrap DEPLOY GRANT: ES256 JWT
|
||||
iss <tenancy issuer>, aud openbao-see-env, sub agent:<env_id>,
|
||||
env_id, kind deploy-grant, jti (single-use, recorded), exp now+1h
|
||||
install.sh stages it at /etc/monky-deployd/bootstrap.jwt (0600)
|
||||
first tick POST https://bao…/v1/auth/jwt-tenancy/login {"role":"see-env","jwt":<grant>}
|
||||
→ auth.client_token (ttl 24 h, max 30 d, policy see-env,
|
||||
metadata {env_id, grant_jti} from the mount's claim_mappings)
|
||||
token → /var/lib/monky-deployd/bao.token (0600); grant deleted
|
||||
every tick Bearer <token> → checkin / bundle / lease / report
|
||||
apply POST /v1/agent/lease → {login_jwt,…} → jwt-tenancy login → NEW token
|
||||
(becomes the bearer; the old one is revoke-self'd) → KV reads
|
||||
upkeep (finally) lookup-self; ttl < 12 h → renew-self; age > 30 d − 2 d → re-lease
|
||||
```
|
||||
|
||||
OpenBao side (Terraform, `tikali/services/security/openbao`): mount `jwt-tenancy`
|
||||
(`jwks_url` = tenancy's `GET /.well-known/agent-jwks.json`, in-cluster), role `see-env`
|
||||
(`role_type=jwt`, `user_claim=env_id` ⇒ one alias/entity **per env**, `bound_audiences=["openbao-see-env"]`,
|
||||
`bound_claims={"kind":"deploy-grant"}`, `claim_mappings={"env_id":"env_id","jti":"grant_jti"}`,
|
||||
`token_ttl=86400`, `token_max_ttl=2592000`, `token_no_default_policy=true`), policy `see-env` =
|
||||
`monky/data/{{identity.entity.aliases.<accessor>.metadata.env_id}}/see/*` read + metadata list +
|
||||
`auth/token/{renew-self,lookup-self}`. **Gate 1 v2 passed 2026-09-05 07:40Z** (distinct entity per env).
|
||||
|
||||
**Second-reveal semantics.** A kit re-reveal (or a retire) supersedes every earlier grant of the
|
||||
backend and revokes the token accessors tenancy knows. The old kit's grant fails at login
|
||||
(unknown/used `jti`), and a token already minted from it is refused at the next call with
|
||||
`401 AGENT_UNAUTHENTICATED` (its `meta.grant_jti` is superseded) — **not** `AGENT_ENV_MISMATCH`.
|
||||
The agent then deletes its token; if a fresh `bootstrap.jwt` is on disk it bootstraps again in the
|
||||
same tick, otherwise it exits 1 and says "re-run the install kit".
|
||||
|
||||
## The four calls
|
||||
|
||||
### `POST /v1/agent/checkin`
|
||||
|
||||
```json
|
||||
{"env_id": "env-qa-02", "agent_version": "0.1.0", "applied_sha": "3f9c…",
|
||||
"host": {"hostname": "env-qa-02", "os": "ubuntu-26.04", "kernel": "7.0.0-30-generic",
|
||||
"docker": "28.3.0", "compose": "2.32.0", "free_bytes": 61234567890,
|
||||
"agent_version": "0.1.0", "transport": "sdk", "site": "cbs", "laptop_mode": false},
|
||||
"containers": [{"name": "see-backend", "state": "running", "health": "healthy"}]}
|
||||
```
|
||||
```json
|
||||
{"env_id": "env-qa-02", "desired_sha": "7a10…", "action": "apply", "purge_volumes": false,
|
||||
"bundle_url": "/v1/agent/bundle/env-qa-02/7a10…", "checkin_interval_s": 60,
|
||||
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": "monky", "prefix": "env-qa-02/see"}}
|
||||
```
|
||||
`action`: `apply` (desired ≠ applied), `none` (converged → heartbeat), `down` (retire; `purge_volumes`
|
||||
is only meaningful here). `vault.mount` is the **KV** mount; the agent adopts it if it differs from
|
||||
config. `vault.addr` is informational — the transport decides how OpenBao is reached.
|
||||
|
||||
### `GET /v1/agent/bundle/{env_id}/{sha}`
|
||||
|
||||
`application/x-tar`, `Cache-Control: no-store`, `X-Bundle-Sha` (tenancy) / `X-Bundle-Sha256` (doc 24;
|
||||
both accepted). Members: `docker-compose.yml` (or `compose.yaml`), `.env.template`,
|
||||
`secrets.manifest.json`, `bundle.json`, optionally `.env.example`. **No secret material** — the agent
|
||||
refuses a manifest entry carrying a `value`. The agent recomputes
|
||||
|
||||
```
|
||||
sha256( for name in sorted(files): name + "\0" + content + "\0" )
|
||||
```
|
||||
|
||||
and refuses (`BUNDLE_SHA_MISMATCH`) unless it equals `desired_sha`. `404 BUNDLE_NOT_FOUND` when
|
||||
the sha is not published.
|
||||
|
||||
`secrets.manifest.json` (paths + versions, never values):
|
||||
```json
|
||||
{"vault_mode": "openbao",
|
||||
"entries": [{"var": "GEMINI_API_KEY", "path": "monky/env-qa-02/see/gemini_api_key", "kind": "supplied", "version": 2},
|
||||
{"var": "POSTGRES_PASSWORD", "path": "monky/env-qa-02/see/pg_password", "kind": "generated", "version": 1},
|
||||
{"var": "SEE_ADMIN_TOKEN", "path": "monky/env-qa-02/see/admin_token", "kind": "generated", "version": 1}]}
|
||||
```
|
||||
Path forms `monky/<env>/see/<name>` (tenancy) and `monky/data/<env>/see/<name>` (doc 24) both map
|
||||
to `GET /v1/monky/data/<env>/see/<name>?version=N`; the `<env>` segment **must** equal the agent's
|
||||
`env_id`. `version: null` = latest. The secret document is `{"value": "…"}`.
|
||||
|
||||
`bundle.json` (monky-deploy `render_bundle_json`): `env_id, tier, site, bundle, target, target_class,
|
||||
address, pull, renderer, images_policy, secrets_provider, agent{…}, files[]`. The agent honours
|
||||
`allow_privileged`, `allow_rollback`, `disk_need_bytes` (top level or under `agent`), refuses a
|
||||
`renderer` other than `compose`, and treats `tier == "prod"` as prod.
|
||||
|
||||
### `POST /v1/agent/lease`
|
||||
|
||||
```json
|
||||
{"env_id": "env-qa-02", "reason": "apply"} // reason: apply | renew
|
||||
```
|
||||
```json
|
||||
{"env_id": "env-qa-02", "login_jwt": "eyJ…", "ttl_s": 3600, "mount": "jwt-tenancy", "role": "see-env",
|
||||
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": "monky"}}
|
||||
```
|
||||
Then `POST /v1/auth/{mount}/login {"role": "{role}", "jwt": "{login_jwt}"}` on OpenBao. A body with
|
||||
`wrapping_token` / `role_id` (the pre-Gate-1 AppRole lease) is refused with `LEASE_SHAPE` → report
|
||||
`failed`. `429 LEASE_RATE_LIMITED` → exit 75.
|
||||
|
||||
### `POST /v1/agent/report`
|
||||
|
||||
```json
|
||||
{"env_id": "env-qa-02", "sha": "7a10…", "result": "applied",
|
||||
"log_tail": "INFO checkin: action=apply …\nINFO applied 7a10c0ffee11 (2 container(s) healthy)",
|
||||
"containers": [{"name": "see-backend", "state": "running", "health": "healthy"}],
|
||||
"detail": "ENV_INCOMPLETE: unresolved: SEE_ADMIN_TOKEN"} // detail only on failed
|
||||
```
|
||||
`result`: `applied` (sets `applied_sha`, health ok — also the heartbeat on `none`), `failed`
|
||||
(health degraded; `detail` = code + names), `down` (clears `applied_sha`). `log_tail` ≤ 16 KiB, the
|
||||
first 2 KiB land in the audit log — it has been through the redactor.
|
||||
|
||||
## Divergences (2026-09-05)
|
||||
|
||||
- **monky-tenancy `main` (MR !15) still implements the AppRole lease and install kit**
|
||||
(`AgentLeaseOut{wrapping_token, role_id}`, `bootstrap.wrap`, `bao.approle` in the kit's config). The
|
||||
binding design is the plan's Gate 1 RESULT / ADR-0028 amendment: `{login_jwt, ttl_s, mount, role}`
|
||||
and `POST /v1/auth/jwt-tenancy/login`. This agent implements the latter; against an un-migrated
|
||||
tenancy it reports `failed` with `LEASE_SHAPE` and refuses `bao.approle` in its config. The tenancy
|
||||
follow-up (deploy-grant signer, JWKS, `lease` shape, kit → `bootstrap.jwt`) is tracked on
|
||||
monky-tenancy.
|
||||
- The kit's generated config uses `tenancy.base_url: http://monky.tenancy.deploy:8081` — accepted as
|
||||
an alias for `tenancy.{scheme,host,port}`.
|
||||
- `report` gains an optional `detail` (doc 24 §3.3); tenancy's `AgentReport` ignores unknown fields
|
||||
today — if `strict` bodies land, `detail` folds into `log_tail`.
|
||||
- Bundle sha header: tenancy sends `X-Bundle-Sha`, doc 24 says `X-Bundle-Sha256`; the agent reads
|
||||
either and trusts only its own computation.
|
||||
Reference in New Issue
Block a user