9 Commits

Author SHA1 Message Date
mdella b749a6d3b4 Merge branch 'docs/dd-0524-0526-protocol-shapes' into 'main'
docs(protocol): AppRole divergence is resolved history; lease example + fake follow tenancy's AgentLeaseOut (DD-0524/DD-0526)

See merge request tikali/applications/monky/monky-deployd!8
2026-09-06 23:23:32 -07:00
Claude-Docs-Manager 05724edde2 docs(protocol): the AppRole divergence is resolved history; the lease example and fake follow tenancy's AgentLeaseOut
DD-0524 — PROTOCOL.md §Divergences bullets 1-2 said monky-tenancy `main`
"still implements the AppRole lease and install kit" and that "the kit's
generated config uses tenancy.base_url". Both were false when the page was
published: tenancy !17 (288df791, merged 07:48Z) shipped AgentLeaseOut{env_id,
login_jwt, ttl_s, mount, role, addr}, the ES256 grant signer and the JWKS
sixteen minutes before v0.1.0 was tagged, and !22 (61bd0281) made the kit run
install.sh with flags instead of writing a config. The two bullets are now
dated "Resolved" notes; bullets 3-4 (report `detail`, X-Bundle-Sha) stand.

DD-0526 — the lease example sent `reason` and received a nested `vault{}`;
tenancy's AgentLease is `{env_id}` and AgentLeaseOut carries `addr` at the top
level (no vault object). The example now shows tenancy's shapes (with a note
that the agent still sends `reason` and tenancy ignores unknown fields), the
checkin example gains `auth_mount`/`auth_role` so it is the full AgentVaultOut,
tests/fakes.py emits `addr` the way tenancy does, and tenancy.py's docstring
and lease() read `addr` first (the `vault{}` fallback is kept so an older fake
or tenancy still leases). Verified against monky-tenancy app/schemas_backends.py
at 1fd51454 (AgentLease 296-297, AgentLeaseOut 300-309, AgentVaultOut 278-283).

Gates (local, py3.12): ruff format --check, ruff check, pytest 50 passed,
bash -n packaging/install.sh.

Doc-Drift: DD-0524 fixed
Doc-Drift: DD-0526 fixed
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AW3QqEpwLV69KHn24Re45Q
2026-09-06 23:13:09 -07:00
monky-deployd-merger b39a60af46 Merge branch 'fix/tenancy-intercept-port' into 'main'
fix: tenancy.port is the intercept port (443); clear sdk dial error; 0.1.5

See merge request tikali/applications/monky/monky-deployd!7
2026-09-07 06:12:42 +00:00
mdella fcf2dcb1eb fix(bundle): ${VAR} inside comment lines is not a reference
The renderer's .env.template header literally says "substitutes every
${VAR}", which the refusal check counted as an unresolved variable
(ENV_INCOMPLETE: unresolved: VAR) — env-qa-02's first bundle was refused.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-07 06:09:08 +00:00
mdella 31586c3058 fix: tenancy.port is the intercept port (443); clear sdk dial error; 0.1.5
install.sh wrote tenancy.port 8081 (the in-pod port) but the
monky.tenancy.deploy intercept is 443, so the SDK found no intercept and the
check-in failed with 'service not available' followed by the SDK's bare
TypeError. Also wrap non-OSError SDK failures in a TransportError that names
the host:port and what to check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-07 06:05:21 +00:00
monky-deployd-merger 41541629ec Merge branch 'fix/tunnel-execstartpre' into 'main'
fix(install.sh): clear ExecStartPre in the run-host drop-in; config/grant before aborting; 0.1.4

See merge request tikali/applications/monky/monky-deployd!6
2026-09-07 06:00:16 +00:00
mdella 744041ca1e fix(install.sh): clear ExecStartPre in the run-host drop-in; don't abort before config/grant land; 0.1.4
On env-qa-02 (ziti-edge-tunnel 1.18.7) the package's ExecStartPre script is
not executable by user ziti (203/EXEC), so the unit never started, and under
set -e `systemctl enable --now` aborted install.sh before config.yaml, ACLs,
the bootstrap grant and the timer were written — a second kit was needed.

- run-host drop-in clears ExecStartPre= (run-host does not auto-enrol JWTs)
- a tunneler start failure is a WARNING until everything else is installed,
  then a hard error at the end (re-run needs no new kit)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-07 05:58:57 +00:00
monky-deployd-merger b08e0adf82 Merge branch 'fix/install-version-clobber-docker' into 'main'
fix(install.sh): --version clobbered by os-release; Docker Engine install; 0.1.3

See merge request tikali/applications/monky/monky-deployd!5
2026-09-06 16:20:12 +00:00
mdella 837a7e5bb1 fix(install.sh): --version clobbered by /etc/os-release; install Docker Engine when absent; 0.1.3
The first real kit run (env-qa-02) downloaded
monky-deployd_26.04 LTS (Resolute Raccoon)_amd64.deb: sourcing /etc/os-release
inline overwrote VERSION. Script vars are now DEPLOYD_VERSION and os-release is
read in a subshell.

Per the operator's rule (end users run ONE script), install.sh now installs
Docker Engine when absent — Docker's apt suite for the host codename, falling
back to Ubuntu's docker.io — with --docker-data-root <dir> to place the
data-root before first start. Existing Docker is left untouched.

bash -n + shellcheck -S warning clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-06 16:18:47 +00:00
14 changed files with 161 additions and 46 deletions
+28
View File
@@ -1,6 +1,34 @@
<!-- xlate:verbatim-fences --> <!-- xlate:verbatim-fences -->
# Changelog # Changelog
## 0.1.5 — 2026-09-07
- **`${VAR}` inside comment lines is not a reference.** The renderer's `.env.template` header literally says
"substitutes every ${VAR}", which the refusal check counted as an unresolved variable
(`ENV_INCOMPLETE: unresolved: VAR`) — the first bundle on env-qa-02 was refused for it.
- **config: `tenancy.port` is the service's intercept port (443), not the in-pod 8081.** With 8081 the SDK
found no intercept and the dial failed (`service not available`, then a bare `TypeError` from the SDK's
fallback). `install.sh` now writes 443; `config.example.yaml` updated.
- **transport sdk: clear error instead of a TypeError** when an address has no intercept or the identity
has no dial policy for the service (`TransportError` names the host:port and what to check).
## 0.1.4 — 2026-09-07
- **install.sh: `ziti-edge-tunnel.service` failed to start after enrolment** (`203/EXEC`: the package's
`ExecStartPre` script is not executable by user `ziti` on ziti-edge-tunnel 1.18.x) and, because the
unit was enabled with `--now` under `set -e`, the script aborted BEFORE writing config/ACLs/grant/timer —
a second kit was needed. The run-host drop-in now clears `ExecStartPre=`; a tunneler start failure is a
WARNING until everything else is in place, then a hard error at the end.
## 0.1.3 — 2026-09-06
- **install.sh: `--version` was clobbered by `/etc/os-release`** (it defines `VERSION`), so the kit tried to download
`monky-deployd_26.04 LTS (Resolute Raccoon)_amd64.deb` (env-qa-02 pilot, first real kit run). Script variables are now
`DEPLOYD_VERSION`; os-release is read in a subshell.
- **install.sh installs Docker Engine when absent** (Docker's apt suite for the codename → fallback `docker.io`), with
`--docker-data-root <dir>` to place the data-root before first start. One script for the end user, per the operator's rule.
## v0.1.2 — 2026-09-05 ## v0.1.2 — 2026-09-05
- **The GitLab project is private** (its parent groups are private, so it cannot be made public; - **The GitLab project is private** (its parent groups are private, so it cannot be made public;
+9 -2
View File
@@ -23,9 +23,9 @@ you the enrolment JWT, a one-time **bootstrap deploy grant** and the read-only *
```sh ```sh
T=<deploy token> # read-only GitLab deploy token (read_package_registry); the kit carries it T=<deploy token> # read-only GitLab deploy token (read_package_registry); the kit carries it
curl -sSf -H "DEPLOY-TOKEN: $T" https://scm.tikali.ai/api/v4/projects/69/packages/generic/monky-deployd/0.1.2/install.sh \ curl -sSf -H "DEPLOY-TOKEN: $T" https://scm.tikali.ai/api/v4/projects/69/packages/generic/monky-deployd/0.1.5/install.sh \
| sudo bash -s -- --env env-qa-02 --site cbs --token "$T" --bootstrap-file bootstrap.jwt --enrol-jwt ./monky-host.env-qa-02.jwt | sudo bash -s -- --env env-qa-02 --site cbs --token "$T" --bootstrap-file bootstrap.jwt --enrol-jwt ./monky-host.env-qa-02.jwt
# [--transport sdk|proxy|system] [--version 0.1.2] [--laptop] [--bao-ca openbao-ca.pem] [--source gitlab|gitea] # [--transport sdk|proxy|system] [--version 0.1.5] [--docker-data-root /home/docker-data] [--laptop] [--bao-ca openbao-ca.pem] [--source gitlab|gitea]
``` ```
`install.sh` installs `ziti-edge-tunnel` (OpenZiti `jammy` suite) and `docker-compose-plugin` if `install.sh` installs `ziti-edge-tunnel` (OpenZiti `jammy` suite) and `docker-compose-plugin` if
@@ -167,3 +167,10 @@ the by-hand recipe). Both locations keep being published.
- `docs/PROTOCOL.md`, `docs/OPERATIONS.md`, `CHANGELOG.md` - `docs/PROTOCOL.md`, `docs/OPERATIONS.md`, `CHANGELOG.md`
- monky-tenancy `docs/usage.md` (agent protocol), `app/api/agent.py`, `app/schemas_backends.py` - monky-tenancy `docs/usage.md` (agent protocol), `app/api/agent.py`, `app/schemas_backends.py`
- monky-deploy (the renderer whose `render_files` produces the bundle) - monky-deploy (the renderer whose `render_files` produces the bundle)
### Docker Engine (since 0.1.3)
`install.sh` installs Docker Engine when it is absent (Docker's apt suite for the host's Ubuntu codename, falling back to
Ubuntu's `docker.io`), so an end user runs exactly one script. `--docker-data-root <dir>` writes `/etc/docker/daemon.json`
before the daemon first starts (pilot VMs keep the data-root on a dedicated disk, e.g. `/home/docker-data`). A pre-existing
Docker is left untouched. macOS/Windows laptops still need Docker Desktop from the user.
+1 -1
View File
@@ -16,7 +16,7 @@ template → optional openbao-ca PEM → optional `monky-deployd-proxy.service`
| var | note | | var | note |
|---|---| |---|---|
| `monky_deployd_version` | pinned release, e.g. `0.1.2` | | `monky_deployd_version` | pinned release, e.g. `0.1.5` |
| `monky_deployd_download_token` | **vaulted**: GitLab deploy token, scope `read_package_registry` only (revocable) — the registry is private; seeded in OpenBao at `monky/monky-tenancy/deployd-download` key `token` (path/key are the operator's choice). Empty = no header (only works with the Gitea `base_url`) | | `monky_deployd_download_token` | **vaulted**: GitLab deploy token, scope `read_package_registry` only (revocable) — the registry is private; seeded in OpenBao at `monky/monky-tenancy/deployd-download` key `token` (path/key are the operator's choice). Empty = no header (only works with the Gitea `base_url`) |
| `monky_deployd_env_id` / `_site` / `_transport` | per host (`env-dev-06`, `cbs`, `sdk`) | | `monky_deployd_env_id` / `_site` / `_transport` | per host (`env-dev-06`, `cbs`, `sdk`) |
| `monky_deployd_bootstrap_grant` | tenancy-minted deploy grant (1 h) — `ansible-vault` or a lookup at play time; empty keeps the existing token | | `monky_deployd_bootstrap_grant` | tenancy-minted deploy grant (1 h) — `ansible-vault` or a lookup at play time; empty keeps the existing token |
@@ -1,7 +1,7 @@
--- ---
# monky_deployd — install and configure the Monky backend pull agent (MONKY-ADR-0028 §D). # monky_deployd — install and configure the Monky backend pull agent (MONKY-ADR-0028 §D).
# Copy this role into osg1-07 (roles/monky_deployd) and roll to env-dev-06..09 after the pilot. # Copy this role into osg1-07 (roles/monky_deployd) and roll to env-dev-06..09 after the pilot.
monky_deployd_version: "0.1.2" monky_deployd_version: "0.1.5"
monky_deployd_deb: "monky-deployd_{{ monky_deployd_version }}_amd64.deb" monky_deployd_deb: "monky-deployd_{{ monky_deployd_version }}_amd64.deb"
# PRIMARY download = the GitLab project's generic package registry on scm.tikali.ai. Inside the estate # PRIMARY download = the GitLab project's generic package registry on scm.tikali.ai. Inside the estate
# gitea.cbs.tikali.net is split-horizon to jump1's RED EIP (10.10.0.175, no HTTP ingress), so backend # gitea.cbs.tikali.net is split-horizon to jump1's RED EIP (10.10.0.175, no HTTP ingress), so backend
+2 -2
View File
@@ -1,4 +1,4 @@
# /etc/monky-deployd/config.yaml — monky-deployd v0.1.2 (MONKY-ADR-0028 §D) # /etc/monky-deployd/config.yaml — monky-deployd v0.1.5 (MONKY-ADR-0028 §D)
# Written by packaging/install.sh (or the ansible role monky_deployd). YAML *subset*: maps, scalars, # Written by packaging/install.sh (or the ansible role monky_deployd). YAML *subset*: maps, scalars,
# simple lists, comments. Keys not listed here are a config error. # simple lists, comments. Keys not listed here are a config error.
@@ -10,7 +10,7 @@ identity: /opt/openziti/etc/identities/monky-host.env-qa-02.json # the box's h
tenancy: tenancy:
service: monky.tenancy.deploy # ziti service bound by the tenancy sidecar -> 127.0.0.1:8081 (agent entrypoint) service: monky.tenancy.deploy # ziti service bound by the tenancy sidecar -> 127.0.0.1:8081 (agent entrypoint)
host: monky.tenancy.deploy # intercept host (sdk/system); defaults to `service` host: monky.tenancy.deploy # intercept host (sdk/system); defaults to `service`
port: 8081 port: 443 # intercept port of monky.tenancy.deploy (in-pod 8081)
scheme: http # plain HTTP inside the mesh; the mesh is the transport security scheme: http # plain HTTP inside the mesh; the mesh is the transport security
proxy_addr: 127.0.0.1:18443 # transport: proxy proxy_addr: 127.0.0.1:18443 # transport: proxy
timeout_s: 30 timeout_s: 30
+19 -12
View File
@@ -70,7 +70,8 @@ same tick, otherwise it exits 1 and says "re-run the install kit".
```json ```json
{"env_id": "env-qa-02", "desired_sha": "7a10…", "action": "apply", "purge_volumes": false, {"env_id": "env-qa-02", "desired_sha": "7a10…", "action": "apply", "purge_volumes": false,
"bundle_url": "/v1/agent/bundle/env-qa-02/7a10…", "checkin_interval_s": 60, "bundle_url": "/v1/agent/bundle/env-qa-02/7a10…", "checkin_interval_s": 60,
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": "monky", "prefix": "env-qa-02/see"}} "vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": "monky", "prefix": "env-qa-02/see",
"auth_mount": "jwt-tenancy", "auth_role": "see-env"}}
``` ```
`action`: `apply` (desired ≠ applied), `none` (converged → heartbeat), `down` (retire; `purge_volumes` `action`: `apply` (desired ≠ applied), `none` (converged → heartbeat), `down` (retire; `purge_volumes`
is only meaningful here). `vault.mount` is the **KV** mount; the agent adopts it if it differs from is only meaningful here). `vault.mount` is the **KV** mount; the agent adopts it if it differs from
@@ -109,12 +110,16 @@ address, pull, renderer, images_policy, secrets_provider, agent{…}, files[]`.
### `POST /v1/agent/lease` ### `POST /v1/agent/lease`
```json ```json
{"env_id": "env-qa-02", "reason": "apply"} // reason: apply | renew {"env_id": "env-qa-02"}
``` ```
(The agent also sends `"reason": "apply" | "renew"` for its own logs; tenancy's `AgentLease` is
`{env_id}` and ignores unknown fields.)
```json ```json
{"env_id": "env-qa-02", "login_jwt": "eyJ…", "ttl_s": 3600, "mount": "jwt-tenancy", "role": "see-env", {"env_id": "env-qa-02", "login_jwt": "eyJ…", "ttl_s": 3600, "mount": "jwt-tenancy", "role": "see-env",
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": "monky"}} "addr": "https://bao.cbs.tikali.net:8200"}
``` ```
`addr` is the OpenBao address for the login below, top-level (tenancy `AgentLeaseOut`); the KV
mount/prefix come from `checkin`'s `vault`, not from the lease.
Then `POST /v1/auth/{mount}/login {"role": "{role}", "jwt": "{login_jwt}"}` on OpenBao. A body with Then `POST /v1/auth/{mount}/login {"role": "{role}", "jwt": "{login_jwt}"}` on OpenBao. A body with
`wrapping_token` / `role_id` (the pre-Gate-1 AppRole lease) is refused with `LEASE_SHAPE` → report `wrapping_token` / `role_id` (the pre-Gate-1 AppRole lease) is refused with `LEASE_SHAPE` → report
`failed`. `429 LEASE_RATE_LIMITED` → exit 75. `failed`. `429 LEASE_RATE_LIMITED` → exit 75.
@@ -133,15 +138,17 @@ first 2 KiB land in the audit log — it has been through the redactor.
## Divergences (2026-09-05) ## Divergences (2026-09-05)
- **monky-tenancy `main` (MR !15) still implements the AppRole lease and install kit** - **Resolved 2026-09-05 (before v0.1.0 was tagged):** monky-tenancy !17 (`288df791`) landed the
(`AgentLeaseOut{wrapping_token, role_id}`, `bootstrap.wrap`, `bao.approle` in the kit's config). The tenancy side of the Gate 1 RESULT / ADR-0028 amendment — `AgentLeaseOut{env_id, login_jwt, ttl_s,
binding design is the plan's Gate 1 RESULT / ADR-0028 amendment: `{login_jwt, ttl_s, mount, role}` mount, role, addr}`, the ES256 deploy-grant signer (`app/agent_keys.py`), `GET
and `POST /v1/auth/jwt-tenancy/login`. This agent implements the latter; against an un-migrated /.well-known/agent-jwks.json`, no AppRole and no response wrapping anywhere in tenancy. The
tenancy it reports `failed` with `LEASE_SHAPE` and refuses `bao.approle` in its config. The tenancy `LEASE_SHAPE` refusal of a `wrapping_token`/`role_id` body stays in this agent as a guard against a
follow-up (deploy-grant signer, JWKS, `lease` shape, kit → `bootstrap.jwt`) is tracked on stale tenancy, not as a description of `main`.
monky-tenancy. - **Resolved 2026-09-05:** the kit no longer generates a config file. Since monky-tenancy !22
- The kit's generated config uses `tenancy.base_url: http://monky.tenancy.deploy:8081` — accepted as (`61bd0281`) the one-time install script stages the bootstrap grant and runs `install.sh --env …
an alias for `tenancy.{scheme,host,port}`. --site … --version … --bootstrap-file …`, and `install.sh` writes `/etc/monky-deployd/config.yaml`
(tenancy service name, OpenBao address and `jwt-tenancy`/`see-env`). `tenancy.base_url:` remains an
accepted alias for `tenancy.{scheme,host,port}` (`config.py`) for hand-written configs.
- `report` gains an optional `detail` (doc 24 §3.3); tenancy's `AgentReport` ignores unknown fields - `report` gains an optional `detail` (doc 24 §3.3); tenancy's `AgentReport` ignores unknown fields
today — if `strict` bodies land, `detail` folds into `log_tail`. today — if `strict` bodies land, `detail` folds into `log_tail`.
- Bundle sha header: tenancy sends `X-Bundle-Sha`, doc 24 says `X-Bundle-Sha256`; the agent reads - Bundle sha header: tenancy sends `X-Bundle-Sha`, doc 24 says `X-Bundle-Sha256`; the agent reads
+1 -1
View File
@@ -4,4 +4,4 @@ Dials monky-tenancy over the mesh with the box's host identity, fetches the rend
leases a deploy grant, logs in to OpenBao, reads its own secrets, runs `docker compose`, leases a deploy grant, logs in to OpenBao, reads its own secrets, runs `docker compose`,
reports. Stdlib only; the optional `openziti` SDK is the `sdk` transport.""" reports. Stdlib only; the optional `openziti` SDK is the `sdk` transport."""
__version__ = "0.1.2" __version__ = "0.1.5"
+8 -2
View File
@@ -144,12 +144,18 @@ def parse(data: bytes, *, max_bytes: int = 4 * 1024 * 1024) -> Bundle:
# --- refusal checks (pure; names only, never values) ----------------------------------------- # --- refusal checks (pure; names only, never values) -----------------------------------------
def _code_lines(text: str) -> str:
"""Drop comment lines: a `# … ${VAR} …` remark in .env.template (the renderer writes one)
is not a reference. Compose/dotenv comments start with `#` after optional whitespace."""
return "\n".join(ln for ln in text.splitlines() if not ln.lstrip().startswith("#"))
def referenced_vars(text: str) -> set[str]: def referenced_vars(text: str) -> set[str]:
return {m.group(1) for m in _VAR_RE.finditer(text)} return {m.group(1) for m in _VAR_RE.finditer(_code_lines(text))}
def defaulted_vars(text: str) -> set[str]: def defaulted_vars(text: str) -> set[str]:
return {m.group(1) for m in _VAR_DEFAULTED_RE.finditer(text)} return {m.group(1) for m in _VAR_DEFAULTED_RE.finditer(_code_lines(text))}
def unresolved_vars(bundle: Bundle, provided: set[str]) -> list[str]: def unresolved_vars(bundle: Bundle, provided: set[str]) -> list[str]:
+4 -2
View File
@@ -4,7 +4,7 @@ Bearer = the agent's OpenBao token (from the `jwt-tenancy` login). Tenancy pins
the token's `meta.env_id` (403 AGENT_ENV_MISMATCH -> exit 78, never retried) and refuses a token the token's `meta.env_id` (403 AGENT_ENV_MISMATCH -> exit 78, never retried) and refuses a token
whose deploy grant was superseded (401 AGENT_UNAUTHENTICATED -> re-bootstrap or re-run the kit). whose deploy grant was superseded (401 AGENT_UNAUTHENTICATED -> re-bootstrap or re-run the kit).
Lease shape of record (Gate 1 v2, 2026-09-05): `{login_jwt, ttl_s, mount, role, vault}`. An Lease shape of record (Gate 1 v2, 2026-09-05): `{env_id, login_jwt, ttl_s, mount, role, addr}`. An
AppRole-era body (`wrapping_token`, `role_id`) is refused loudly — there is nothing to unwrap.""" AppRole-era body (`wrapping_token`, `role_id`) is refused loudly — there is nothing to unwrap."""
from __future__ import annotations from __future__ import annotations
@@ -158,13 +158,15 @@ class TenancyClient:
"(ADR-0028 amendment 2026-09-05)", "(ADR-0028 amendment 2026-09-05)",
) )
raise TenancyError(200, "LEASE_SHAPE", "lease response carries no login_jwt") raise TenancyError(200, "LEASE_SHAPE", "lease response carries no login_jwt")
# shape of record (tenancy AgentLeaseOut): `addr` is top-level; a pre-0.1.x `vault{}` object
# is still read as a fallback so an older fake or tenancy does not break the lease
v = js.get("vault") or {} v = js.get("vault") or {}
return Lease( return Lease(
login_jwt=str(js["login_jwt"]), login_jwt=str(js["login_jwt"]),
ttl_s=int(js.get("ttl_s") or 3600), ttl_s=int(js.get("ttl_s") or 3600),
mount=str(js.get("mount") or "jwt-tenancy"), mount=str(js.get("mount") or "jwt-tenancy"),
role=str(js.get("role") or "see-env"), role=str(js.get("role") or "see-env"),
vault=Vault(addr=v.get("addr"), mount=v.get("mount"), prefix=v.get("prefix")), vault=Vault(addr=js.get("addr") or v.get("addr"), mount=v.get("mount"), prefix=v.get("prefix")),
) )
def report( def report(
+8
View File
@@ -104,6 +104,14 @@ class SdkTransport(Transport):
return socket.create_connection((host, port), timeout=timeout) return socket.create_connection((host, port), timeout=timeout)
except OSError as exc: except OSError as exc:
raise TransportError(f"transport sdk: dial {host}:{port} failed: {exc}") from exc raise TransportError(f"transport sdk: dial {host}:{port} failed: {exc}") from exc
except Exception as exc: # noqa: BLE001 - the SDK raises bare Exception((code, msg)) and TypeError
# openziti-sdk-py: an address with NO matching intercept falls through to
# PySocket.connect(tuple) → TypeError; a matching intercept the identity may not dial
# raises Exception((-18, 'service not available')) — env-qa-02 pilot, 2026-09-07.
raise TransportError(
f"transport sdk: dial {host}:{port} failed: {exc} — no intercept for that host:port, or this "
"identity has no dial policy for the service (check the intercept port and the identity's attrs)"
) from exc
def describe(self) -> str: def describe(self) -> str:
return f"sdk(identity={self.identity_path})" return f"sdk(identity={self.identity_path})"
+64 -21
View File
@@ -3,10 +3,10 @@
# #
# T=<read-only GitLab deploy token, scope read_package_registry> # from the install kit / OpenBao # T=<read-only GitLab deploy token, scope read_package_registry> # from the install kit / OpenBao
# curl -sSf -H "DEPLOY-TOKEN: $T" \ # curl -sSf -H "DEPLOY-TOKEN: $T" \
# https://scm.tikali.ai/api/v4/projects/69/packages/generic/monky-deployd/0.1.2/install.sh \ # https://scm.tikali.ai/api/v4/projects/69/packages/generic/monky-deployd/0.1.5/install.sh \
# | sudo bash -s -- --env env-qa-02 --site cbs --token "$T" --bootstrap-file bootstrap.jwt \ # | sudo bash -s -- --env env-qa-02 --site cbs --token "$T" --bootstrap-file bootstrap.jwt \
# [--transport sdk|proxy|system] [--version 0.1.2] [--enrol-jwt /path/monky-host.env-qa-02.jwt] \ # [--transport sdk|proxy|system] [--version 0.1.5] [--enrol-jwt /path/monky-host.env-qa-02.jwt] \
# [--laptop] [--source gitlab|gitea] # [--laptop] [--source gitlab|gitea] [--docker-data-root /home/docker-data]
# #
# --token / MONKY_DEPLOYD_TOKEN: the GitLab project is PRIVATE (its parent groups are private, so it # --token / MONKY_DEPLOYD_TOKEN: the GitLab project is PRIVATE (its parent groups are private, so it
# cannot be made public); every download from the generic package registry — this script included — # cannot be made public); every download from the generic package registry — this script included —
@@ -29,7 +29,7 @@
set -euo pipefail set -euo pipefail
umask 077 umask 077
DEFAULT_VERSION="0.1.2" DEFAULT_VERSION="0.1.5"
# Download source. PRIMARY is the GitLab project's generic package registry on scm.tikali.ai: inside # Download source. PRIMARY is the GitLab project's generic package registry on scm.tikali.ai: inside
# the estate gitea.cbs.tikali.net is split-horizon to jump1's RED EIP (10.10.0.175), which has no HTTP # the estate gitea.cbs.tikali.net is split-horizon to jump1's RED EIP (10.10.0.175), which has no HTTP
# ingress, so backend boxes cannot reach the Gitea mirror (cbs/iac#102); scm.tikali.ai they can. The # ingress, so backend boxes cannot reach the Gitea mirror (cbs/iac#102); scm.tikali.ai they can. The
@@ -45,7 +45,7 @@ TOKEN="${MONKY_DEPLOYD_TOKEN:-}"
OPENZITI_SUITE="${OPENZITI_SUITE:-jammy}" OPENZITI_SUITE="${OPENZITI_SUITE:-jammy}"
IDENTITY_DIR="/opt/openziti/etc/identities" IDENTITY_DIR="/opt/openziti/etc/identities"
ETC="/etc/monky-deployd" ETC="/etc/monky-deployd"
ENV_ID="" SITE="" TRANSPORT="sdk" VERSION="$DEFAULT_VERSION" ENROL_JWT="" BOOTSTRAP_FILE="" NO_RUN="" FORCE_CONFIG="" BAO_CA="" LAPTOP="false" ENV_ID="" SITE="" TRANSPORT="sdk" DOCKER_DATA_ROOT="" TUNNEL_FAILED="" DEPLOYD_VERSION="$DEFAULT_VERSION" ENROL_JWT="" BOOTSTRAP_FILE="" NO_RUN="" FORCE_CONFIG="" BAO_CA="" LAPTOP="false"
usage() { if [ -f "$0" ]; then sed -n '2,28p' "$0"; else echo "monky-deployd install.sh — see README.md (Install)"; fi; exit "${1:-0}"; } usage() { if [ -f "$0" ]; then sed -n '2,28p' "$0"; else echo "monky-deployd install.sh — see README.md (Install)"; fi; exit "${1:-0}"; }
die() { echo "install.sh: $*" >&2; exit 1; } die() { echo "install.sh: $*" >&2; exit 1; }
@@ -56,7 +56,8 @@ while [ $# -gt 0 ]; do
--env) ENV_ID="$2"; shift 2 ;; --env) ENV_ID="$2"; shift 2 ;;
--site) SITE="$2"; shift 2 ;; --site) SITE="$2"; shift 2 ;;
--transport) TRANSPORT="$2"; shift 2 ;; --transport) TRANSPORT="$2"; shift 2 ;;
--version) VERSION="$2"; shift 2 ;; --version) DEPLOYD_VERSION="$2"; shift 2 ;;
--docker-data-root) DOCKER_DATA_ROOT="$2"; shift 2 ;;
--enrol-jwt) ENROL_JWT="$2"; shift 2 ;; --enrol-jwt) ENROL_JWT="$2"; shift 2 ;;
--bootstrap-file) BOOTSTRAP_FILE="$2"; shift 2 ;; --bootstrap-file) BOOTSTRAP_FILE="$2"; shift 2 ;;
--bao-ca) BAO_CA="$2"; shift 2 ;; --bao-ca) BAO_CA="$2"; shift 2 ;;
@@ -86,9 +87,13 @@ IDENTITY="$IDENTITY_DIR/monky-host.$ENV_ID.json"
export DEBIAN_FRONTEND=noninteractive export DEBIAN_FRONTEND=noninteractive
if [ -r /etc/os-release ]; then if [ -r /etc/os-release ]; then
. /etc/os-release # read os-release in a SUBSHELL: it defines VERSION/NAME/ID/... and sourcing it inline clobbered
if [ "${ID:-}" != "ubuntu" ] || [ "${VERSION_ID:-}" != "26.04" ]; then # this script's own variables (0.1.2 → "26.04 LTS (Resolute Raccoon)" on the env-qa-02 pilot)
echo "WARNING: verified on Ubuntu 26.04; this is ${PRETTY_NAME:-unknown}. Continuing." >&2 OS_ID=$(. /etc/os-release; printf '%s' "${ID:-}")
OS_VERSION_ID=$(. /etc/os-release; printf '%s' "${VERSION_ID:-}")
OS_CODENAME=$(. /etc/os-release; printf '%s' "${VERSION_CODENAME:-${UBUNTU_CODENAME:-}}")
if [ "$OS_ID" != "ubuntu" ] || [ "$OS_VERSION_ID" != "26.04" ]; then
echo "WARNING: verified on Ubuntu 26.04 only (this is ${OS_ID:-?} ${OS_VERSION_ID:-?}); continuing" >&2
fi fi
fi fi
@@ -108,7 +113,34 @@ fi
if [ "$TRANSPORT" = "proxy" ] && ! command -v ziti >/dev/null 2>&1; then if [ "$TRANSPORT" = "proxy" ] && ! command -v ziti >/dev/null 2>&1; then
need_pkgs+=(openziti) # the `ziti` CLI (ziti tunnel proxy) from the same repo need_pkgs+=(openziti) # the `ziti` CLI (ziti tunnel proxy) from the same repo
fi fi
command -v docker >/dev/null 2>&1 || die "docker is not installed; install Docker Engine first (https://docs.docker.com/engine/install/ubuntu/)" if ! command -v docker >/dev/null 2>&1; then
# Docker Engine from Docker's apt repo (the suite for this Ubuntu codename); falls back to Ubuntu's
# docker.io if Docker has no suite for the codename yet. --docker-data-root puts the data-root on
# a dedicated disk BEFORE the daemon first starts (pilot VMs mount one at /home/docker-data).
log "docker is not installed — installing Docker Engine"
export DEBIAN_FRONTEND=noninteractive
install -d -m 0755 /etc/apt/keyrings
if curl -fsSL --max-time 20 "https://download.docker.com/linux/ubuntu/dists/${OS_CODENAME:-noble}/Release" -o /dev/null 2>/dev/null; then
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu ${OS_CODENAME:-noble} stable" > /etc/apt/sources.list.d/docker.list
apt-get update -qq
docker_pkgs="docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin"
else
log "Docker has no apt suite for '${OS_CODENAME:-?}' yet — using Ubuntu's docker.io"
apt-get update -qq
docker_pkgs="docker.io docker-compose-v2"
fi
if [ -n "$DOCKER_DATA_ROOT" ]; then
install -d -m 0710 "$DOCKER_DATA_ROOT"
install -d -m 0755 /etc/docker
[ -f /etc/docker/daemon.json ] || printf '{ "data-root": "%s" }\n' "$DOCKER_DATA_ROOT" > /etc/docker/daemon.json
fi
# shellcheck disable=SC2086
apt-get install -y -qq --no-install-recommends $docker_pkgs
systemctl enable --now docker
log "installed $(docker --version) (data-root $(docker info -f '{{.DockerRootDir}}' 2>/dev/null || echo default))"
fi
docker compose version >/dev/null 2>&1 || need_pkgs+=(docker-compose-plugin) docker compose version >/dev/null 2>&1 || need_pkgs+=(docker-compose-plugin)
python3 -c 'import sys; sys.exit(0 if sys.version_info >= (3, 12) else 1)' 2>/dev/null || die "python3 >= 3.12 is required" python3 -c 'import sys; sys.exit(0 if sys.version_info >= (3, 12) else 1)' 2>/dev/null || die "python3 >= 3.12 is required"
if [ "${#need_pkgs[@]}" -gt 0 ]; then if [ "${#need_pkgs[@]}" -gt 0 ]; then
@@ -119,23 +151,23 @@ fi
# --- 2. the pinned .deb ----------------------------------------------------------------------------- # --- 2. the pinned .deb -----------------------------------------------------------------------------
installed="$(dpkg-query -W -f='${Version}' monky-deployd 2>/dev/null || true)" installed="$(dpkg-query -W -f='${Version}' monky-deployd 2>/dev/null || true)"
if [ "$installed" = "$VERSION" ]; then if [ "$installed" = "$DEPLOYD_VERSION" ]; then
log "monky-deployd $VERSION already installed" log "monky-deployd $DEPLOYD_VERSION already installed"
else else
tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT
deb="monky-deployd_${VERSION}_amd64.deb" deb="monky-deployd_${DEPLOYD_VERSION}_amd64.deb"
# the deploy token travels in a 0600 curl config file (-K), never on the command line, so neither # the deploy token travels in a 0600 curl config file (-K), never on the command line, so neither
# `ps` nor an xtrace shows it; the file dies with $tmp # `ps` nor an xtrace shows it; the file dies with $tmp
curlrc="$tmp/curlrc"; : > "$curlrc"; chmod 0600 "$curlrc" curlrc="$tmp/curlrc"; : > "$curlrc"; chmod 0600 "$curlrc"
if [ "$SOURCE" = gitlab ]; then if [ "$SOURCE" = gitlab ]; then
url="${BASE_URL:-$GITLAB_BASE_URL}/${VERSION}" url="${BASE_URL:-$GITLAB_BASE_URL}/${DEPLOYD_VERSION}"
if [ -n "$TOKEN" ]; then if [ -n "$TOKEN" ]; then
printf 'header = "DEPLOY-TOKEN: %s"\n' "$TOKEN" > "$curlrc" printf 'header = "DEPLOY-TOKEN: %s"\n' "$TOKEN" > "$curlrc"
else else
echo "WARNING: no --token/MONKY_DEPLOYD_TOKEN: the scm.tikali.ai project is private, the download will 401 (use --source gitea off-estate)" >&2 echo "WARNING: no --token/MONKY_DEPLOYD_TOKEN: the scm.tikali.ai project is private, the download will 401 (use --source gitea off-estate)" >&2
fi fi
else else
url="${BASE_URL:-$GITEA_BASE_URL}/releases/download/v${VERSION}" url="${BASE_URL:-$GITEA_BASE_URL}/releases/download/v${DEPLOYD_VERSION}"
fi fi
log "downloading $deb from $url${TOKEN:+ (DEPLOY-TOKEN)}" log "downloading $deb from $url${TOKEN:+ (DEPLOY-TOKEN)}"
curl -fsSL -K "$curlrc" -o "$tmp/$deb" "$url/$deb" || die "download of $deb failed (private registry: is the deploy token set and valid?)" curl -fsSL -K "$curlrc" -o "$tmp/$deb" "$url/$deb" || die "download of $deb failed (private registry: is the deploy token set and valid?)"
@@ -162,14 +194,21 @@ chmod 0600 "$IDENTITY"
# ziti-edge-tunnel as a HOST (bind side, no tun/DNS) — the agent dials with the SDK or the proxy # ziti-edge-tunnel as a HOST (bind side, no tun/DNS) — the agent dials with the SDK or the proxy
install -d /etc/systemd/system/ziti-edge-tunnel.service.d install -d /etc/systemd/system/ziti-edge-tunnel.service.d
cat > /etc/systemd/system/ziti-edge-tunnel.service.d/run-host.conf <<'DROPIN' cat > /etc/systemd/system/ziti-edge-tunnel.service.d/run-host.conf <<'DROPIN'
# monky-deployd: run-host mode (no tun, no DNS); identities from the standard directory # monky-deployd: run-host mode (no tun, no DNS); identities from the standard directory.
# ExecStartPre is cleared: the package's ziti-edge-tunnel.sh (auto-enrol *.jwt in the identity dir)
# is not executable by user ziti on 1.18.x (203/EXEC, env-qa-02 pilot) and run-host does not need it.
[Service] [Service]
ExecStartPre=
ExecStart= ExecStart=
ExecStart=/opt/openziti/bin/ziti-edge-tunnel run-host --identity-dir=/opt/openziti/etc/identities ExecStart=/opt/openziti/bin/ziti-edge-tunnel run-host --identity-dir=/opt/openziti/etc/identities
DROPIN DROPIN
systemctl daemon-reload systemctl daemon-reload
systemctl enable --now ziti-edge-tunnel.service systemctl enable ziti-edge-tunnel.service >/dev/null 2>&1 || true
systemctl restart ziti-edge-tunnel.service || true if ! systemctl restart ziti-edge-tunnel.service; then
# do NOT abort here: config, ACLs, grant and timer below must land so a re-run needs no new kit
echo "WARNING: ziti-edge-tunnel.service failed to start — see: journalctl -u ziti-edge-tunnel; continuing" >&2
TUNNEL_FAILED=1
fi
# --- 4. config, ACLs, bootstrap grant ----------------------------------------------------------------- # --- 4. config, ACLs, bootstrap grant -----------------------------------------------------------------
install -d -m 0750 -o root -g monky-deployd "$ETC" install -d -m 0750 -o root -g monky-deployd "$ETC"
@@ -186,7 +225,7 @@ if [ -s "$ETC/config.yaml" ] && [ -z "$FORCE_CONFIG" ]; then
log "keeping existing $ETC/config.yaml (use --force-config to rewrite)" log "keeping existing $ETC/config.yaml (use --force-config to rewrite)"
else else
cat > "$ETC/config.yaml" <<CFG cat > "$ETC/config.yaml" <<CFG
# written by install.sh $(date -u +%FT%TZ) — monky-deployd $VERSION # written by install.sh $(date -u +%FT%TZ) — monky-deployd $DEPLOYD_VERSION
env_id: $ENV_ID env_id: $ENV_ID
site: $SITE site: $SITE
transport: $TRANSPORT transport: $TRANSPORT
@@ -194,7 +233,7 @@ identity: $IDENTITY
tenancy: tenancy:
service: monky.tenancy.deploy service: monky.tenancy.deploy
host: monky.tenancy.deploy host: monky.tenancy.deploy
port: 8081 port: 443 # the service's INTERCEPT port (host.v1 forwards to 8081 inside the pod); plain HTTP inside the mesh
scheme: http scheme: http
proxy_addr: 127.0.0.1:18443 proxy_addr: 127.0.0.1:18443
bao: bao:
@@ -246,7 +285,7 @@ rm -f "$ETC/enrol.jwt"
cat <<CHECK cat <<CHECK
monky-deployd $VERSION installed for $ENV_ID ($SITE, transport $TRANSPORT). Verification checklist (Ubuntu 26.04): monky-deployd $DEPLOYD_VERSION installed for $ENV_ID ($SITE, transport $TRANSPORT). Verification checklist (Ubuntu 26.04):
python3 --version # >= 3.12 python3 --version # >= 3.12
docker compose version # compose plugin present docker compose version # compose plugin present
systemctl is-active ziti-edge-tunnel # run-host mode (drop-in run-host.conf) systemctl is-active ziti-edge-tunnel # run-host mode (drop-in run-host.conf)
@@ -260,3 +299,7 @@ monky-deployd $VERSION installed for $ENV_ID ($SITE, transport $TRANSPORT). Veri
CHECK CHECK
CHECK_STATUS="$(monky-deployd status 2>&1 || true)" CHECK_STATUS="$(monky-deployd status 2>&1 || true)"
echo "$CHECK_STATUS" | sed 's/^/ | /' echo "$CHECK_STATUS" | sed 's/^/ | /'
if [ -n "$TUNNEL_FAILED" ] && ! systemctl is-active --quiet ziti-edge-tunnel.service; then
echo "ERROR: ziti-edge-tunnel.service is not running; the agent cannot reach the mesh until it is. Everything else is installed — fix the unit and 'systemctl restart ziti-edge-tunnel monky-deployd'." >&2
exit 1
fi
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "monky-deployd" name = "monky-deployd"
version = "0.1.2" version = "0.1.5"
description = "Monky backend pull agent: checkin -> bundle -> lease -> OpenBao -> docker compose -> report, over the ziti mesh (MONKY-ADR-0028)" description = "Monky backend pull agent: checkin -> bundle -> lease -> OpenBao -> docker compose -> report, over the ziti mesh (MONKY-ADR-0028)"
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
+2 -1
View File
@@ -354,7 +354,8 @@ class FakeTenancy:
"ttl_s": 3600, "ttl_s": 3600,
"mount": self.bao.mount, "mount": self.bao.mount,
"role": self.bao.role, "role": self.bao.role,
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": self.kv_mount}, # tenancy AgentLeaseOut: `addr` is top-level; KV mount/prefix come from checkin
"addr": "https://bao.cbs.tikali.net:8200",
}, },
headers={"Cache-Control": "no-store"}, headers={"Cache-Control": "no-store"},
) )
+13
View File
@@ -79,3 +79,16 @@ def test_disk_need_bytes_spellings():
assert b.parse(tar_bytes(make_files(meta={"agent": {"disk_need_bytes": 5}}))).disk_need_bytes == 5 assert b.parse(tar_bytes(make_files(meta={"agent": {"disk_need_bytes": 5}}))).disk_need_bytes == 5
assert b.parse(tar_bytes(make_files(meta={"disk": {"need_bytes": 7}}))).disk_need_bytes == 7 assert b.parse(tar_bytes(make_files(meta={"disk": {"need_bytes": 7}}))).disk_need_bytes == 7
assert b.parse(tar_bytes(make_files())).disk_need_bytes == 0 assert b.parse(tar_bytes(make_files())).disk_need_bytes == 0
def test_placeholders_in_comment_lines_are_not_references():
"""The renderer's .env.template header says '... substitutes every ${VAR} ...' — that must not
become an unresolved 'VAR' (env-qa-02 pilot: ENV_INCOMPLETE: unresolved: VAR)."""
from monky_deployd.bundle import defaulted_vars, referenced_vars
text = (
"# The on-box agent substitutes every ${VAR} from OpenBao per secrets.manifest.json.\n"
" # ${ALSO_COMMENT}\nGEMINI_API_KEY=${GEMINI_API_KEY}\nPG=${PGPASSWORD:-x}\n"
)
assert referenced_vars(text) == {"GEMINI_API_KEY", "PGPASSWORD"}
assert defaulted_vars(text) == {"PGPASSWORD"}